cd /news/ai-tools/hybrid-search-for-rag-bm25-plus-vect… · home › topics › ai-tools › article
[ARTICLE · art-172018] src=dev.to ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

Hybrid search for RAG: BM25 plus vectors, merged with reciprocal rank fusion

A developer published a tested plain-Python implementation of hybrid search for RAG that runs BM25 keyword search and vector embedding search on every query, then merges the two rankings with reciprocal rank fusion (RRF, default k=60). The approach targets embeddings' weakness with exact tokens such as invoice IDs, error codes and names, which semantic search blurs, while BM25 covers exact matches and vectors cover paraphrases. The writeup includes BM25 scoring code with identifier-preserving tokenization and an RRF merge function that needs no score normalization or training.

recommendation

Run BM25 and vector search on every query and merge with RRF at k=60, and keep hyphenated identifiers as single tokens so codes like inv-20931 aren't split.

more in playbook: Retrieval and RAG
by read5 min views3 publishedNov 18, 2026

Hybrid search runs two retrievers on every query, BM25 keyword search and vector (embedding) search, and merges their results. It fixes the best-known weakness of embeddings: they capture meaning but blur exact tokens, so a query for invoice INV-20931, error ECONNRESET or a person's name can return documents that are merely about invoices, network errors or people. The usual way to merge the two rankings is reciprocal rank fusion (RRF): each document scores 1 / (k + rank) in each list, the scores are added, and k = 60 is the common default. It needs no score normalisation and no training. Below is a complete, tested implementation in plain Python.

An embedding model compresses a passage into a few hundred numbers that represent what it's about. Two passages about disputed invoices land close together whether the invoice number is INV-20931 or INV-55102. That's what makes semantic search good at paraphrases, and bad at identifiers. I covered the mechanics in embeddings and vector search, explained.

Keyword search has the opposite profile: it matches exact tokens and knows nothing about meaning. Together they cover each other's gaps.

Query type Vector search BM25
"how do I get my money back" finds the refund policy misses it (no shared words)
"INV-20931" returns any invoice finds that invoice
"ECONNRESET when proxying" finds network errors finds that error
A customer's name unreliable exact

BM25 scores each query term by how rare it is across all documents (inverse document frequency) and how often it appears in this one, with diminishing returns for repetition (k1) and a penalty for long documents ( b). The tokenizer matters as much as the formula: keep hyphenated identifiers like inv-20931 as one token, or the code is split into two common pieces.


import math

import re

from collections import Counter

def tokenize(text: str) -> list[str]:

    return re.findall(r"[a-z0-9]+(?:-[a-z0-9]+)*", text.lower())

class BM25:

    def **init**(self, docs: dict[str, str], k1: float = 1.5, b: float = 0.75):

        self.docs = {doc_id: tokenize(text) for doc_id, text in docs.items()}

        self.k1, self.b = k1, b

        self.avg_len = sum(len(t) for t in self.docs.values()) / len(self.docs)

        df = Counter(term for tokens in self.docs.values() for term in set(tokens))

        n = len(self.docs)

        self.idf = {term: math.log(1 + (n - f + 0.5) / (f + 0.5)) for term, f in df.items()}

``` php
def search(self, query: str, k: int = 10) -> list[str]:
    scores = {}
    for doc_id, tokens in self.docs.items():
        tf = Counter(tokens)
        norm = self.k1 * (1 - self.b + self.b * len(tokens) / self.avg_len)
        score = sum(self.idf.get(t, 0) * tf[t] * (self.k1 + 1) / (tf[t] + norm) for t in tokenize(query) if t in tf)
        if score > 0:
            scores[doc_id] = score
    return sorted(scores, key=scores.get, reverse=True)[:k]
In production you'd use the BM25 built into your database or search engine (Postgres full-text search, OpenSearch, Elasticsearch, or a vector database with sparse vectors) rather than scanning every document. The scoring is the same idea.

## How does reciprocal rank fusion merge the results?

BM25 scores and cosine similarities live on different scales, so adding them directly is meaningless without careful normalisation. RRF sidesteps that by using only **ranks**:

``` python title="hybrid.py: RRF"
def rrf(rankings: list[list[str]], k: int = 60) -> list[str]:
    scores: dict[str, float] = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

A document near the top of both lists beats one at the top of only one. With four small documents and the query INV-20931 dispute:


docs = {

    "refunds": "Refunds are issued within 14 days to the original payment method.",

    "inv-20931": "Invoice INV-20931 was disputed by the customer in March.",

    "disputes": "How to handle a customer dispute about a charge or an invoice.",

    "shipping": "Orders ship within two business days.",

}

bm25 = BM25(docs)

bm25.search("INV-20931 dispute")       # ['inv-20931', 'disputes']

vector = ["disputes", "refunds", "shipping", "inv-20931"]  # embeddings ranked the exact invoice last

rrf([bm25.search("INV-20931 dispute"), vector])

The invoice moves from rank 4 (vectors alone) to rank 2, inside any reasonable top-k. Notice it doesn't reach rank 1: the dispute-handling guide ranked well in both lists, and RRF rewards agreement. Whether that's right depends on the question, which is why you measure instead of guessing.

How do you choose k and weights? #

  • k = 60 is the value from the original RRF paper and a sensible default. A smaller k gives more weight to the very top of each list; a larger one flattens the ranks.
  • Weighted RRF multiplies each list's contribution, for example 1.5 / (k + rank) for BM25 when users mostly search by ID. Only do this if your evaluation shows it helps.
  • Retrieve more than you return. Fetch the top 20–50 from each retriever, fuse, then keep the top 5–10 for the prompt. A reranker can sit after fusion when the order of the top results matters most.

How do you know hybrid search helped? #

Run the same labelled questions through vector-only, BM25-only and hybrid retrieval, and compare recall@k and MRR. Split the results by query type: hybrid should lift questions containing names, codes and exact phrases, and shouldn't hurt the paraphrased ones. The metrics and a small harness are in how to evaluate a RAG system, and the chunk boundaries you're retrieving matter too: see RAG chunking strategies.

Summary #

  • Embeddings miss exact identifiers; BM25 misses paraphrases. Hybrid search runs both.
  • Merge with reciprocal rank fusion: 1 / (k + rank) per list, summed, k = 60. No score normalisation needed.
  • Tokenize identifiers as single tokens, retrieve 20–50 per retriever, fuse, then trim.
  • Confirm the gain with recall@k and MRR, split by query type.

Related: building a fully offline RAG chatbot with Ollama and single agent vs multi-agent systems.


Written by Ali Ahmed · Original article


── more in #ai-tools 4 stories · sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/hybrid-search-for-ra…] indexed:0 read:5min 2026-11-18 · —