Dense vector search is great until your agent asks for parseAuthHeader and gets back three chunks about "authentication token handling" — semantically close, functionally useless. Same story with file paths, error codes, and compliance clause numbers. These are lexical needles, and embeddings blur them.
This isn't a niche complaint. XERJ has been picking up steam on the strength of "stop making agents grep," and Volcengine's OpenViking has ~38k stars for treating agent context as structured, addressable storage rather than a vector dump. Both are good. Both are also new infrastructure you now operate. XERJ in particular already does hybrid BM25 + kNN with RRF — if you're greenfield and happy to run a dedicated engine, genuinely go look at it.
I had a constraint they don't solve for: the evidence had to live in the same transaction as the data it describes, in a database my team already backs up and already knows how to restore at 3am.
The usual fix is to bolt on BM25 from a dedicated search service, then fuse results in application code. That means a second stateful cluster: its own backups, its own failure modes, and no transactional guarantee that your index agrees with your source of truth.
I wanted to know how far Postgres 16 + pgvector could get on its own. Turns out: all the way.
Knowledge Fabric runs full-text search over tsvector and dense search over an HNSW index in the same database, then fuses the two ranked lists with Reciprocal Rank Fusion:
**score = 1 / (60 + rank_lexical) + 1 / (60 + rank_vector)**
RRF only needs ranks, not scores, so you skip the entire problem of normalizing BM25 against cosine similarity. A chunk that places top-3 on both paths wins. A chunk that's #1 lexically and invisible semantically still surfaces — which is exactly what you want when the query is a function name.
One query. One backup. One consistency model.
This part matters more than it sounds. In an agentic setup, retrieved text isn't just context — it's the authorization premise for a state-changing tool call. If the agent reads a policy chunk and then executes a deploy, something needs to prove that chunk wasn't tampered with.
Every chunk gets a deterministic SHA-256 hash and a composite provenance digest, canonicalized per RFC 8785 so byte-level serialization differences don't produce different hashes for identical content. A downstream policy layer can then verify the agent acted on authentic evidence before approving execution.
retrieve_evidence, get_document, explain_retrieval — bounded tools over stdio and HTTP via FastMCP. Works with Claude Code, Cursor, or your own harness. explain_retrieval exists because "why did it return that?" is a question you will ask roughly forty times in week one.
Docker Compose quickstart, benchmarks, and the full implementation: https://github.com/sagarv48/knowledge-fabric
If you've tuned RRF in production — did you keep k at 60, or did you find your corpus wanted something different? I'm curious whether the default holds up on codebases with heavy identifier repetition.