Combining Vector and Full-Text Search with Reciprocal Rank Fusion

A developer demonstrated Reciprocal Rank Fusion (RRF) as a way to combine vector and full-text search results for RAG retrieval, using the open-source project vaultrag as an example. The technique discards incomparable raw scores like cosine distance and BM25, fusing results purely by rank position with the formula 1/(k + rank) and a default k of 60 from the original 2009 Cormack, Clarke, and Buettcher paper. The author shows the approach in a short Python function and notes vaultrag performs the fusion inside a single Postgres query using pgvector's distance operator.

Every time I build retrieval for a RAG system, I run into the same wall. Vector search is wonderful at understanding meaning. Ask it "what is our policy on working from home" and it happily finds the paragraph titled "Remote Work Guidelines" even though the words do not match. But ask it for "ERR 4021" or "Policy 7.3" and it flounders, because an error code has no meaningful embedding neighborhood. Keyword search is the mirror image: it nails the exact token and misses the paraphrase. Real questions contain both kinds of signal. So the honest answer is to run both searches and combine them. The problem is that the two searches produce scores on completely different scales. A cosine distance of 0.18 and a BM25 score of 7.4 are not comparable numbers. You cannot just add them. This is exactly the problem Reciprocal Rank Fusion solves, and I want to teach you the technique here. I will use my own project, vaultrag, as the running example, but the method transfers to any two rankers you have. The insight behind RRF is almost rude in its simplicity. Ignore the raw scores entirely. They are not comparable, so stop trying to compare them. Instead, look only at the position a document holds in each list. Rank 1 is rank 1 whether it came from a vector index or a keyword index, and those you can combine. Here is the formula. For a document d , its fused score is the sum over every ranked list of one divided by a constant k plus the document's rank in that list: RRF d = sum over lists L of 1 / k + rank L d Rank is 1-based best result is rank 1 . If a document does not appear in a given list at all, it simply contributes nothing from that list. The constant k is a damping term. A larger k flattens the curve so the top result of any single list does not dominate; a smaller k lets the top hits win harder. The value from the original 2009 paper by Cormack, Clarke, and Buettcher is k = 60 , and it is a perfectly reasonable default that I have never had a strong reason to change. Why 1 / k + rank ? Because it is steeply decreasing but never zero. Moving from rank 1 to rank 2 costs a lot; moving from rank 40 to rank 41 costs almost nothing. That matches intuition: the difference between the best and second-best result matters far more than the difference between the fortieth and forty-first. Before wiring it into a database, here is the whole technique in a few lines you can drop into a notebook and reason about: python def reciprocal rank fusion ranked lists, k=60 : """Fuse several ranked lists of ids into one. ranked lists: list of lists, each already ordered best-first. Returns: list of id, score sorted best-first. """ scores = {} for ranked in ranked lists: for rank, doc id in enumerate ranked, start=1 : scores doc id = scores.get doc id, 0.0 + 1.0 / k + rank return sorted scores.items , key=lambda kv: kv 1 , reverse=True Notice what this does and does not need. It never sees a cosine distance or a BM25 score. It only needs each list in the correct order. That is the entire contract, which is why RRF works with any rankers you can name, including a lexical index, a dense retriever, and a reranker all at once. A quick sanity check: vec = "a", "b", "c" vector search order kw = "c", "a", "d" keyword search order for doc id, score in reciprocal rank fusion vec, kw : print doc id, round score, 5 Document a appears at rank 1 in vec and rank 2 in kw, so it scores 1/61 + 1/62 . Document c is rank 3 in vec and rank 1 in kw. Both landing near the top of some list beats b and d , which each show up in only one list. Agreement between the two searches is rewarded, which is precisely the behavior we want. In vaultrag I do not fuse in Python. I fuse in the same SQL query that runs both searches, so Postgres hands me one already-fused list. The two arms are a vector search using pgvector's <= distance operator and a full-text search using ts rank cd . ROW NUMBER turns each arm's ordering into an explicit rank, and then a FULL OUTER JOIN lets me add the two reciprocal terms: js vec AS SELECT id, ROW NUMBER OVER ORDER BY embedding <= % embedding s::vector AS rank FROM visible WHERE embedding IS NOT NULL ORDER BY embedding <= % embedding s::vector LIMIT % candidates s , kw AS SELECT id, ROW NUMBER OVER ORDER BY ts rank cd tsv, websearch to tsquery 'english', % q s DESC AS rank FROM visible WHERE tsv @@ websearch to tsquery 'english', % q s LIMIT % candidates s , fused AS SELECT COALESCE vec.id, kw.id AS id, COALESCE 1.0 / % k s + vec.rank , 0 + COALESCE 1.0 / % k s + kw.rank , 0 AS score FROM vec FULL OUTER JOIN kw ON kw.id = vec.id Two details are load-bearing. The FULL OUTER JOIN is what allows a document to appear in one arm but not the other, and COALESCE ..., 0 is the "contributes nothing when absent" rule from the formula made literal. A document found only by keyword search has a NULL vector rank, so its vector term collapses to zero, and only its keyword term counts. That is RRF working exactly as designed. I also limit each arm to a candidate pool I use 50 before fusing, then return the top few. Fusing the whole corpus would be pointless work; anything ranked fiftieth in both arms is not going to win. RRF is deliberately blind to how confident each search was. A document that is a near-perfect keyword match at rank 1 and a document that is a lukewarm match at rank 1 contribute the identical 1/ k+1 , because the rank is the same and the score was discarded. Usually that robustness is a feature, since it stops one loud arm from steamrolling the other. But when one of your rankers is genuinely much more trustworthy than the other for a given query, RRF cannot express that. If you need to weight the arms or preserve calibrated confidence, you will have to reach for weighted fusion or a learned reranker instead. RRF is the strong, simple baseline, not the ceiling. Reciprocal Rank Fusion earns its keep because it demands so little: no shared score scale, no training, no tuning beyond one constant that has a sensible default. Give it two lists in the right order and it gives you back one better list. For hybrid search that is very often all you need. If you want to see the full ACL-scoped hybrid query this snippet came from, including how both search arms start from the same authorized candidate set, the code is at github.com/AgentPostmortem/vaultrag https://github.com/AgentPostmortem/vaultrag . Clone it, read app/retrieval.py , and try changing k to see the ranking shift for yourself.