{"slug": "hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion", "title": "Hybrid search for RAG: BM25 plus vectors, merged with reciprocal rank fusion", "summary": "A developer published a tested plain-Python implementation of hybrid search for RAG that runs BM25 keyword search and vector embedding search on every query, then merges the two rankings with reciprocal rank fusion (RRF, default k=60). The approach targets embeddings' weakness with exact tokens such as invoice IDs, error codes and names, which semantic search blurs, while BM25 covers exact matches and vectors cover paraphrases. The writeup includes BM25 scoring code with identifier-preserving tokenization and an RRF merge function that needs no score normalization or training.", "body_md": "**Hybrid search** runs two retrievers on every query, **BM25** keyword search and **vector** (embedding) search, and merges their results. It fixes the best-known weakness of embeddings: they capture meaning but blur exact tokens, so a query for invoice `INV-20931`, error `ECONNRESET` or a person's name can return documents that are merely *about* invoices, network errors or people. The usual way to merge the two rankings is **reciprocal rank fusion (RRF)**: each document scores `1 / (k + rank)` in each list, the scores are added, and `k = 60` is the common default. It needs no score normalisation and no training. Below is a complete, tested implementation in plain Python.\n\nAn embedding model compresses a passage into a few hundred numbers that represent what it's about. Two passages about disputed invoices land close together whether the invoice number is `INV-20931` or `INV-55102`. That's what makes semantic search good at paraphrases, and bad at identifiers. I covered the mechanics in [embeddings and vector search, explained](https://aliahmed.si/blog/embeddings-vector-search-explained/).\n\nKeyword search has the opposite profile: it matches exact tokens and knows nothing about meaning. Together they cover each other's gaps.\n\n| Query type | Vector search | BM25 | \n|---|---|---|\n| \"how do I get my money back\" | finds the refund policy | misses it (no shared words) | \n| \"INV-20931\" | returns any invoice | finds that invoice | \n| \"ECONNRESET when proxying\" | finds network errors | finds that error | \n| A customer's name | unreliable | exact | \n\nBM25 scores each query term by how **rare** it is across all documents (inverse document frequency) and how **often** it appears in this one, with diminishing returns for repetition (`k1`) and a penalty for long documents (` b`). The tokenizer matters as much as the formula: keep hyphenated identifiers like `inv-20931` as one token, or the code is split into two common pieces.\n\n``` python title=\"hybrid.py: BM25\"\n\nimport math\n\nimport re\n\nfrom collections import Counter\n\ndef tokenize(text: str) -> list[str]:\n\n    return re.findall(r\"[a-z0-9]+(?:-[a-z0-9]+)*\", text.lower())\n\nclass BM25:\n\n    def **init**(self, docs: dict[str, str], k1: float = 1.5, b: float = 0.75):\n\n        self.docs = {doc_id: tokenize(text) for doc_id, text in docs.items()}\n\n        self.k1, self.b = k1, b\n\n        self.avg_len = sum(len(t) for t in self.docs.values()) / len(self.docs)\n\n        df = Counter(term for tokens in self.docs.values() for term in set(tokens))\n\n        n = len(self.docs)\n\n        self.idf = {term: math.log(1 + (n - f + 0.5) / (f + 0.5)) for term, f in df.items()}\n\n``` php\ndef search(self, query: str, k: int = 10) -> list[str]:\n    scores = {}\n    for doc_id, tokens in self.docs.items():\n        tf = Counter(tokens)\n        norm = self.k1 * (1 - self.b + self.b * len(tokens) / self.avg_len)\n        score = sum(self.idf.get(t, 0) * tf[t] * (self.k1 + 1) / (tf[t] + norm) for t in tokenize(query) if t in tf)\n        if score > 0:\n            scores[doc_id] = score\n    return sorted(scores, key=scores.get, reverse=True)[:k]\nIn production you'd use the BM25 built into your database or search engine (Postgres full-text search, OpenSearch, Elasticsearch, or a vector database with sparse vectors) rather than scanning every document. The scoring is the same idea.\n\n## How does reciprocal rank fusion merge the results?\n\nBM25 scores and cosine similarities live on different scales, so adding them directly is meaningless without careful normalisation. RRF sidesteps that by using only **ranks**:\n\n``` python title=\"hybrid.py: RRF\"\ndef rrf(rankings: list[list[str]], k: int = 60) -> list[str]:\n    scores: dict[str, float] = {}\n    for ranking in rankings:\n        for rank, doc_id in enumerate(ranking, start=1):\n            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)\n    return sorted(scores, key=scores.get, reverse=True)\n```\n\nA document near the top of **both** lists beats one at the top of only one. With four small documents and the query `INV-20931 dispute`:\n\n``` python title=\"Example\"\n\ndocs = {\n\n    \"refunds\": \"Refunds are issued within 14 days to the original payment method.\",\n\n    \"inv-20931\": \"Invoice INV-20931 was disputed by the customer in March.\",\n\n    \"disputes\": \"How to handle a customer dispute about a charge or an invoice.\",\n\n    \"shipping\": \"Orders ship within two business days.\",\n\n}\n\nbm25 = BM25(docs)\n\nbm25.search(\"INV-20931 dispute\")       # ['inv-20931', 'disputes']\n\nvector = [\"disputes\", \"refunds\", \"shipping\", \"inv-20931\"]  # embeddings ranked the exact invoice last\n\nrrf([bm25.search(\"INV-20931 dispute\"), vector])\n\n```\nThe invoice moves from **rank 4** (vectors alone) to **rank 2**, inside any reasonable top-k. Notice it doesn't reach rank 1: the dispute-handling guide ranked well in both lists, and RRF rewards agreement. Whether that's right depends on the question, which is why you measure instead of guessing.\n\n## How do you choose k and weights?\n\n* **k = 60** is the value from the original RRF paper and a sensible default. A smaller `k` gives more weight to the very top of each list; a larger one flattens the ranks.\n* **Weighted RRF** multiplies each list's contribution, for example `1.5 / (k + rank)` for BM25 when users mostly search by ID. Only do this if your evaluation shows it helps.\n* **Retrieve more than you return.** Fetch the top 20–50 from each retriever, fuse, then keep the top 5–10 for the prompt. A reranker can sit after fusion when the order of the top results matters most.\n\n## How do you know hybrid search helped?\n\nRun the same labelled questions through vector-only, BM25-only and hybrid retrieval, and compare **recall\\@k** and **MRR**. Split the results by query type: hybrid should lift questions containing names, codes and exact phrases, and shouldn't hurt the paraphrased ones. The metrics and a small harness are in [how to evaluate a RAG system](https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/), and the chunk boundaries you're retrieving matter too: see [RAG chunking strategies](https://aliahmed.si/blog/rag-chunking-strategies/).\n\n## Summary\n\n* Embeddings miss exact identifiers; BM25 misses paraphrases. Hybrid search runs both.\n* Merge with **reciprocal rank fusion**: `1 / (k + rank)` per list, summed, `k = 60`. No score normalisation needed.\n* Tokenize identifiers as single tokens, retrieve 20–50 per retriever, fuse, then trim.\n* Confirm the gain with recall\\@k and MRR, split by query type.\n\nRelated: [building a fully offline RAG chatbot with Ollama](https://aliahmed.si/blog/offline-rag-chatbot-ollama/) and [single agent vs multi-agent systems](https://aliahmed.si/blog/single-agent-vs-multi-agent/).\n\n---\n\nWritten by Ali Ahmed · [Original article](https://aliahmed.si/blog/hybrid-search-bm25-reciprocal-rank-fusion/)\n```\n\n", "url": "https://wpnews.pro/news/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion", "canonical_source": "https://dev.to/quantumbreakz/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion-4ap", "published_at": "2026-11-18 18:37:23+00:00", "updated_at": "2026-11-18 18:42:07.948800+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "artificial-intelligence"], "entities": [], "also_reported_by": [], "takeaway": "Merging BM25 and vector rankings with reciprocal rank fusion at k=60 needs no score normalization and recovers exact-token matches like INV-20931 or ECONNRESET that embeddings blur.", "recommendation": {"advice": "Run BM25 and vector search on every query and merge with RRF at k=60, and keep hyphenated identifiers as single tokens so codes like inv-20931 aren't split.", "playbook": "rag-retrieval", "playbook_url": "https://wpnews.pro/playbooks/rag-retrieval"}, "alternates": {"html": "https://wpnews.pro/news/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion", "markdown": "https://wpnews.pro/news/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion.md", "text": "https://wpnews.pro/news/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion.txt", "jsonld": "https://wpnews.pro/news/hybrid-search-for-rag-bm25-plus-vectors-merged-with-reciprocal-rank-fusion.jsonld"}}