Stop Choosing Between BM25 and Vector Search: Implement Hybrid Search with RRF

A developer has published a guide to implementing hybrid search in retrieval-augmented generation (RAG) pipelines, combining BM25 keyword search with dense vector embeddings using Reciprocal Rank Fusion (RRF). The approach addresses failure modes where pure vector search misses exact tokens like error codes and pure keyword search fails on paraphrased queries. The developer provides a minimal Python implementation that merges ranked lists from both search methods to improve retrieval accuracy.

Combine dense embeddings and sparse keyword search using Reciprocal Rank Fusion to eliminate retrieval failure modes in production RAG systems. Most production RAG pipelines start with pure vector search. It works reliably during initial demos, but fails quietly once users begin searching for exact product IDs, error codes, or domain-specific identifiers. Vector search translates text into semantic coordinate spaces. Because dense models optimize for high-level concepts, they tend to blur fine-grained details. A search for ERR-502-BAD-GATEWAY might retrieve general network troubleshooting docs instead of the exact runbook for error 502 . Conversely, relying purely on keyword search BM25 breaks when users paraphrase. A query for "reduce database memory footprint" completely misses an article titled "Mitigating PostgreSQL RAM Saturation" if there is no direct keyword overlap. The Naive Approach: Semantic-only retrieval misses exact tokens def get context query: str, vector db - list str : Fails when query contains exact SKUs, UUIDs, or specific error logs return vector db.similarity search query, k=5 Relying on a single retrieval strategy creates hard blind spots that degrade downstream LLM generation quality. The fix is running sparse BM25 and dense vector searches concurrently and merging their ranked outputs using Reciprocal Rank Fusion RRF . ┌─── BM25 Keyword Search ─── Sparse Ranked List ───┐ User Query ──────┤ ├─── RRF Fusion ─── Top-K Documents ─── LLM └─── Dense Vector Search ─── Dense Ranked List ───┘ RRF avoids the core challenge of hybrid search: score normalization . BM25 produces unbounded positive floating-point scores, while vector search usually outputs cosine similarities between -1 and 1 . Merging raw scores requires fragile heuristic scaling factors. Instead of comparing raw scores, RRF scores documents based strictly on their relative rank position across both search lists: $$RRF d = \sum {m \in M} \frac{1}{k + r m d }$$ Where: 60 that prevents low-ranking outliers from disproportionately skewing results.Here is a minimal, production-ready implementation combining BM25, dense embeddings, and RRF in under 25 lines of Python: python import numpy as np from rank bm25 import BM25Okapi from sentence transformers import SentenceTransformer from sklearn.metrics.pairwise import cosine similarity def hybrid rrf search query: str, corpus: list str , top n: int = 3, k: int = 60 - list str : 1. Sparse BM25 scoring & ranking bm25 = BM25Okapi doc.lower .split for doc in corpus bm25 rank = np.argsort bm25.get scores query.lower .split ::-1 2. Dense Vector scoring & ranking model = SentenceTransformer 'all-MiniLM-L6-v2' doc embs, query emb = model.encode corpus , model.encode query dense rank = np.argsort cosine similarity query emb, doc embs 0 ::-1 3. Reciprocal Rank Fusion rrf scores = {} for rank idx, doc idx in enumerate bm25 rank : rrf scores doc idx = rrf scores.get doc idx, 0.0 + 1.0 / k + rank idx + 1 for rank idx, doc idx in enumerate dense rank : rrf scores doc idx = rrf scores.get doc idx, 0.0 + 1.0 / k + rank idx + 1 sorted docs = sorted rrf scores.items , key=lambda x: x 1 , reverse=True return corpus doc idx for doc idx, in sorted docs :top n This pattern guarantees resilience. If a user inputs a technical keyword match, BM25 pushes the correct document to the top. If a user describes a high-level conceptual problem, dense