Pointers, Not Copies: Semantic Search Inside the Vault

The problem

A small vault of plain Markdown files needs no search layer. At session start, the chief of staff reads the few files that matter — a decisions digest, a feedback log, a dashboard — and everything else is a few grep calls away. That works until it doesn’t. Notes accumulate: project files, meeting notes, daily logs, a growing decisions record. Past a few dozen files, nobody can honestly say “I read everything that’s relevant.” Retrieval stops being optional.

The standard fix is RAG-style: chunk the corpus, embed it, and let the model answer questions directly from retrieved snippets. This is exactly the wrong shape for a vault whose entire value is trustworthiness. Snippet-answering recreates a bug family we already learned to fear with digests: text that reads complete but isn’t. A snippet blends sources, drops the caveat that appeared two paragraphs after the excerpt, and misses an amendment made later in the file. The answer comes back fluent, confident, and wrong — a silent lie, which is the most expensive kind, because nobody knows to challenge it.

A vault run on plain files + git lives or dies by provenance: every fact carries a source. An answer generated from retrieval snippets has no source. It has a query.

The philosophy behind the choice

The rule is one sentence: a pointer is not a source.

The search layer’s job is not to answer questions. Its job is to say, within a second, “the passage most likely relevant to this is at Atlas/decisions-raw.md:701-775” — a path plus a line range — and get out of the way. The assistant must then open the file and read the range before stating any fact, and cite the file-and-line location when it does.

Three sub-rules make this work in practice:

• Capped output. The tool returns at most ~8 pointers. This is a budget, not a limit: eight reads of ±20-line windows is a few kilobytes of context, versus the tens of kilobytes of full-file reads it replaces. Scarcity forces ranking quality.
• Read the range, then extend it. A matched passage can be cut off mid-sentence or sit near a note that was amended later. The assistant reads the pointed range, widens it when the passage is incomplete or the surrounding note may carry later edits, and only then claims the fact.
• Never answer from the snippet. The snippet text shown by search exists to decide whether to read, not what to say. The provenance rule — every fact gets a “ comment before it’s written down — extends naturally to retrieval: every fact spoken from a search must name the file and lines it came from.

This costs one extra read per question. That cost is the point: the friction is where honesty lives.

How it’s wired at a high level

The implementation (tools/vault-search/) is deliberately small — one Python file, a thin CLI wrapper, an eval harness — sitting inside the vault repo it searches.

Hybrid retrieval. Every Markdown file is hashed and chunked. Chunks get two representations: dense vectors from a self-hosted embedding model on an always-on home machine (reachable through a local proxy; the API key lives outside the vault), and a keyword index (SQLite FTS5, i.e. BM25). By default, ranking is vector-only, because the principal’s captures are paraphrase-shaped — spoken, casual questions that share few keywords with the notes that answer them. When the query contains an identifier-like token — a bead id, a reference number, a date, a full name — BM25 fuses into the ranking (reciprocal-rank fusion), because exact tokens should beat semantic similarity for lookups. An offline eval over 38 real-style questions justified the gating: paraphrased queries hit in the top-5 about 82% of the time with embeddings versus roughly half with keyword search alone, and the gated hybrid was best on every metric.

Gated, with a retry queue — never a fallback. Indexing runs in two places: a post-commit hook (so a triage commit updates the index instantly) and a small scheduled job every few minutes (so edits made directly in an editor, which never touch git, get picked up too). Embedding requests are deliberately tiny — a few chunks each, strictly sequential, with a per-request abort guard and bounded retries — because one oversized batch can push the home machine into memory pressure. If the embedding backend is unreachable, nothing fails: un-indexed files stay in a pending state and remain findable by keyword, and the tool says so in its output (“vector search unavailable — N files pending”). There is no local fallback model, on purpose: vectors from two different models cannot share one index, so a fallback would quietly corrupt search quality rather than visibly degrade it.

The index lives outside the repo. Content hashes, chunks, FTS5 tables and vectors sit in a cache directory, not in git. The chunk store is a plain-text copy of the vault; the vectors are tens of megabytes and growing. Rebuilding is cheap and scripted: hash every file, re-chunk and re-embed only what changed, drop deleted ones.

Scope is curated. Atlas, Calendar, and Efforts are searched by default. Delivery infrastructure and reports that hold other people’s work data are indexed but opt-in per query — they add noise, and noise in the pointer list is how the assistant ends up citing the wrong file with total confidence.

A note for operators: the eval harness is part of the tool, not an afterthought. After any change to chunking, the embedding model, or the fusion logic, a fixed question set must stay at or above its measured hit-rate floor. Retrieval changes should be measured, not vibes-reviewed.

What a reader’s chief of staff should be able to do afterwards

After handing this guide to their own system, the chief of staff should be able to:

1. Answer “what do we know about X” with file-and-line citations instead of confident paraphrase — every factual claim traceable to a range the assistant actually read, not a snippet it skimmed.
2. Use retrieval as context discovery — replacing multi-round grep and whole-file cat with ≤8 pointers plus targeted reads, cutting context cost per question substantially while improving the hit rate on paraphrased questions.
3. Run the same posture the production system runs: keyword-only degradation that announces itself, a retry queue that never silently substitutes a different model, and an index refreshed from the git-committed vault plus scheduled sweeps for out-of-band edits.
4. Say “I don’t know” when the pointers don’t actually contain the answer — the failure mode of snippet-answering is that it never leaves room for that sentence.

#vault · #search · #retrieval · #citations · #rag