{"slug": "hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector", "title": "Hybrid retrieval in one Postgres query: RRF over tsvector + pgvector", "summary": "A developer built Knowledge Fabric, an open-source retrieval system that runs hybrid lexical (tsvector) and dense vector (pgvector HNSW) search inside a single Postgres 16 query, fusing the ranked lists with Reciprocal Rank Fusion. Each chunk is hashed with SHA-256 and a composite provenance digest canonicalized per RFC 8785, letting a downstream policy layer verify that an agent acted on authentic evidence before approving a state-changing tool call. The project exposes bounded retrieval tools over stdio and HTTP via FastMCP and ships with a Docker Compose quickstart.", "body_md": "**Dense vector search** is great until your agent asks for parseAuthHeader and gets back three chunks about \"authentication token handling\" — semantically close, functionally useless. Same story with file paths, error codes, and compliance clause numbers. These are lexical needles, and embeddings blur them.\n\nThis isn't a niche complaint. XERJ has been picking up steam on the strength of \"stop making agents grep,\" and Volcengine's OpenViking has ~38k stars for treating agent context as structured, addressable storage rather than a vector dump. Both are good. Both are also new infrastructure you now operate. XERJ in particular already does hybrid BM25 + kNN with RRF — if you're greenfield and happy to run a dedicated engine, genuinely go look at it.\n\nI had a constraint they don't solve for: the evidence had to live in the same transaction as the data it describes, in a database my team already backs up and already knows how to restore at 3am.\n\nThe usual fix is to bolt on BM25 from a dedicated search service, then fuse results in application code. That means a second stateful cluster: its own backups, its own failure modes, and no transactional guarantee that your index agrees with your source of truth.\n\nI wanted to know how far Postgres 16 + pgvector could get on its own. Turns out: all the way.\n\nKnowledge Fabric runs full-text search over tsvector and dense search over an HNSW index in the same database, then fuses the two ranked lists with Reciprocal Rank Fusion:\n\n```\n**score = 1 / (60 + rank_lexical) + 1 / (60 + rank_vector)**\n```\n\nRRF only needs ranks, not scores, so you skip the entire problem of normalizing BM25 against cosine similarity. A chunk that places top-3 on both paths wins. A chunk that's #1 lexically and invisible semantically still surfaces — which is exactly what you want when the query is a function name.\n\nOne query. One backup. One consistency model.\n\nThis part matters more than it sounds. In an agentic setup, retrieved text isn't just context — it's the authorization premise for a state-changing tool call. If the agent reads a policy chunk and then executes a deploy, something needs to prove that chunk wasn't tampered with.\n\nEvery chunk gets a deterministic SHA-256 hash and a composite provenance digest, canonicalized per RFC 8785 so byte-level serialization differences don't produce different hashes for identical content. A downstream policy layer can then verify the agent acted on authentic evidence before approving execution.\n\nretrieve_evidence, get_document, explain_retrieval — bounded tools over stdio and HTTP via FastMCP. Works with Claude Code, Cursor, or your own harness. explain_retrieval exists because \"why did it return that?\" is a question you will ask roughly forty times in week one.\n\nDocker Compose quickstart, benchmarks, and the full implementation: [https://github.com/sagarv48/knowledge-fabric](https://github.com/sagarv48/knowledge-fabric)\n\nIf you've tuned RRF in production — did you keep k at 60, or did you find your corpus wanted something different? I'm curious whether the default holds up on codebases with heavy identifier repetition.", "url": "https://wpnews.pro/news/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector", "canonical_source": "https://dev.to/vinay_kumarks_9d8ca4e45/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector-2hm", "published_at": "2026-09-21 12:15:58+00:00", "updated_at": "2026-09-21 12:25:35.069971+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools", "agent-protocols", "ai-tools"], "entities": ["Knowledge Fabric", "Postgres", "pgvector", "FastMCP", "XERJ", "Volcengine", "OpenViking", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector", "markdown": "https://wpnews.pro/news/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector.md", "text": "https://wpnews.pro/news/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector.txt", "jsonld": "https://wpnews.pro/news/hybrid-retrieval-in-one-postgres-query-rrf-over-tsvector-pgvector.jsonld"}}