Hybrid search for RAG: BM25 plus vectors, merged with reciprocal rank fusion A developer published a tested plain-Python implementation of hybrid search for RAG that runs BM25 keyword search and vector embedding search on every query, then merges the two rankings with reciprocal rank fusion (RRF, default k=60). The approach targets embeddings' weakness with exact tokens such as invoice IDs, error codes and names, which semantic search blurs, while BM25 covers exact matches and vectors cover paraphrases. The writeup includes BM25 scoring code with identifier-preserving tokenization and an RRF merge function that needs no score normalization or training. Hybrid search runs two retrievers on every query, BM25 keyword search and vector embedding search, and merges their results. It fixes the best-known weakness of embeddings: they capture meaning but blur exact tokens, so a query for invoice INV-20931 , error ECONNRESET or a person's name can return documents that are merely about invoices, network errors or people. The usual way to merge the two rankings is reciprocal rank fusion RRF : each document scores 1 / k + rank in each list, the scores are added, and k = 60 is the common default. It needs no score normalisation and no training. Below is a complete, tested implementation in plain Python. An embedding model compresses a passage into a few hundred numbers that represent what it's about. Two passages about disputed invoices land close together whether the invoice number is INV-20931 or INV-55102 . That's what makes semantic search good at paraphrases, and bad at identifiers. I covered the mechanics in embeddings and vector search, explained https://aliahmed.si/blog/embeddings-vector-search-explained/ . Keyword search has the opposite profile: it matches exact tokens and knows nothing about meaning. Together they cover each other's gaps. | Query type | Vector search | BM25 | |---|---|---| | "how do I get my money back" | finds the refund policy | misses it no shared words | | "INV-20931" | returns any invoice | finds that invoice | | "ECONNRESET when proxying" | finds network errors | finds that error | | A customer's name | unreliable | exact | BM25 scores each query term by how rare it is across all documents inverse document frequency and how often it appears in this one, with diminishing returns for repetition k1 and a penalty for long documents b . The tokenizer matters as much as the formula: keep hyphenated identifiers like inv-20931 as one token, or the code is split into two common pieces. python title="hybrid.py: BM25" import math import re from collections import Counter def tokenize text: str - list str : return re.findall r" a-z0-9 + ?:- a-z0-9 + ", text.lower class BM25: def init self, docs: dict str, str , k1: float = 1.5, b: float = 0.75 : self.docs = {doc id: tokenize text for doc id, text in docs.items } self.k1, self.b = k1, b self.avg len = sum len t for t in self.docs.values / len self.docs df = Counter term for tokens in self.docs.values for term in set tokens n = len self.docs self.idf = {term: math.log 1 + n - f + 0.5 / f + 0.5 for term, f in df.items } php def search self, query: str, k: int = 10 - list str : scores = {} for doc id, tokens in self.docs.items : tf = Counter tokens norm = self.k1 1 - self.b + self.b len tokens / self.avg len score = sum self.idf.get t, 0 tf t self.k1 + 1 / tf t + norm for t in tokenize query if t in tf if score 0: scores doc id = score return sorted scores, key=scores.get, reverse=True :k In production you'd use the BM25 built into your database or search engine Postgres full-text search, OpenSearch, Elasticsearch, or a vector database with sparse vectors rather than scanning every document. The scoring is the same idea. How does reciprocal rank fusion merge the results? BM25 scores and cosine similarities live on different scales, so adding them directly is meaningless without careful normalisation. RRF sidesteps that by using only ranks : python title="hybrid.py: RRF" def rrf rankings: list list str , k: int = 60 - list str : scores: dict str, float = {} for ranking in rankings: for rank, doc id in enumerate ranking, start=1 : scores doc id = scores.get doc id, 0 + 1 / k + rank return sorted scores, key=scores.get, reverse=True A document near the top of both lists beats one at the top of only one. With four small documents and the query INV-20931 dispute : python title="Example" docs = { "refunds": "Refunds are issued within 14 days to the original payment method.", "inv-20931": "Invoice INV-20931 was disputed by the customer in March.", "disputes": "How to handle a customer dispute about a charge or an invoice.", "shipping": "Orders ship within two business days.", } bm25 = BM25 docs bm25.search "INV-20931 dispute" 'inv-20931', 'disputes' vector = "disputes", "refunds", "shipping", "inv-20931" embeddings ranked the exact invoice last rrf bm25.search "INV-20931 dispute" , vector The invoice moves from rank 4 vectors alone to rank 2 , inside any reasonable top-k. Notice it doesn't reach rank 1: the dispute-handling guide ranked well in both lists, and RRF rewards agreement. Whether that's right depends on the question, which is why you measure instead of guessing. How do you choose k and weights? k = 60 is the value from the original RRF paper and a sensible default. A smaller k gives more weight to the very top of each list; a larger one flattens the ranks. Weighted RRF multiplies each list's contribution, for example 1.5 / k + rank for BM25 when users mostly search by ID. Only do this if your evaluation shows it helps. Retrieve more than you return. Fetch the top 20โ€“50 from each retriever, fuse, then keep the top 5โ€“10 for the prompt. A reranker can sit after fusion when the order of the top results matters most. How do you know hybrid search helped? Run the same labelled questions through vector-only, BM25-only and hybrid retrieval, and compare recall\@k and MRR . Split the results by query type: hybrid should lift questions containing names, codes and exact phrases, and shouldn't hurt the paraphrased ones. The metrics and a small harness are in how to evaluate a RAG system https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/ , and the chunk boundaries you're retrieving matter too: see RAG chunking strategies https://aliahmed.si/blog/rag-chunking-strategies/ . Summary Embeddings miss exact identifiers; BM25 misses paraphrases. Hybrid search runs both. Merge with reciprocal rank fusion : 1 / k + rank per list, summed, k = 60 . No score normalisation needed. Tokenize identifiers as single tokens, retrieve 20โ€“50 per retriever, fuse, then trim. Confirm the gain with recall\@k and MRR, split by query type. Related: building a fully offline RAG chatbot with Ollama https://aliahmed.si/blog/offline-rag-chatbot-ollama/ and single agent vs multi-agent systems https://aliahmed.si/blog/single-agent-vs-multi-agent/ . --- Written by Ali Ahmed ยท Original article https://aliahmed.si/blog/hybrid-search-bm25-reciprocal-rank-fusion/