Skip to content
Aditya Prasad
Work

Project 02 / 04

MemoCore

Local-first search and answers over your own documentation.

QUERYDENSE · COSINESPARSE · BM25MERGERERANKTOP K

Context

Why it exists

Team knowledge tends to live in Markdown — wikis, runbooks, notes — and keyword search finds the words, not the answer. MemoCore is a retrieval engine for that content that runs on local models.

The backend is complete: a worker ingests repositories, a search endpoint returns the most relevant passages, and an ask endpoint streams a written answer built only from those passages. The web app is being built on top of it: sign-in and repository management work; sync controls and the chat interface are next.

Architecture

Question to answer, as implemented

  1. 01

    Ingest

    Worker clones a repository over SSH and finds every .md file

  2. 02

    Chunk

    Heading-aware sections, embedded with Ollama (nomic-embed-text, 768-d)

  3. 03

    Retrieve

    Dense cosine and sparse BM25 run concurrently, each over-fetching candidates

  4. 04

    Merge

    Union of both result sets, de-duplicated by chunk

  5. 05

    Rerank

    Cross-encoder (ms-marco-MiniLM-L6-v2) rescores every candidate; top k kept

  6. 06

    Answer

    Top passages go into the prompt; a local model in LM Studio streams an answer that cites file paths

Decisions

Key decisions

  1. 01

    Hybrid retrieval

    Dense embeddings match meaning; BM25 matches exact terms such as flags, error strings and names, which technical docs are full of. The two retrievers run concurrently and their results are merged.

  2. 02

    Retrieve wide, then rerank

    Each retriever returns four times the requested number of candidates. A cross-encoder then reads every query–passage pair and its scores, squashed to 0–1, decide the final order.

  3. 03

    CPU-only by intent

    PyTorch is pinned to the CPU-only wheel index, which skips several gigabytes of unused CUDA libraries — the target is a home server without a GPU.

  4. 04

    Locked-down data

    Row-level security is enabled on every table with no policies, so only server-side code holding the secret key (the backend and the web app's server) can read or write. The browser never talks to the database.

  5. 05

    Answers only from your documents

    The model is told to answer only from the retrieved passages, to say when the answer isn't in the knowledge base, and to cite the source file for each fact. Each passage is labelled with its file path and heading.

  6. 06

    Fail before the stream starts

    The connection to the model is opened, and checked, before the first token is sent. If LM Studio is down or returns an error, the client gets a proper error response instead of a half-written answer.

Progress

Built, and still to build

  • BuiltIngestion worker and sync jobs
  • BuiltHybrid search with cross-encoder reranking
  • BuiltStreamed answers from retrieved passages
  • BuiltWeb app: sign-in and repository management
  • NextWeb app: sync controls and live progress
  • NextWeb app: chat with source citations

Limitations

Known limitations

  • Embeddings are stored in Postgres with pgvector, but scoring currently runs in Python over every chunk per query; an indexed vector search would be needed at larger scale.