Why vector search misses exact terms, and what hybrid search fixed (measured)

A team building in-browser knowledge bases fixed vector search returning wrong exact strings — such as ERR_4512 instead of ERR_4521 — by adding a BM25 keyword index alongside all-MiniLM-L6-v2 (384-d) embeddings in SurrealDB 2.x compiled to WebAssembly on IndexedDB, fusing the two ranked lists with reciprocal-rank fusion at k = 60. Each list fetches fetchK = min(200, max(k, 4k, 20)) candidates, and because fused scores are rank-based, a relevance floor is applied to each hit's cosine similarity, with exact keyword matches (BM25 > 0 and more than half the query terms present) counted as relevant regardless. The team documented two bugs: SurrealDB 2.x's @@ full-text operator is AND-only with no OR form, worked around by giving each of up to 12 query terms its own match reference and summing clamped search::score values, and a filtering bug that made small collections return zero hits.

Why vector search misses exact terms, and what hybrid search fixed measured Ask a vector search for ERR 4521 and you may get ERR 4512. How we added BM25 next to vectors with reciprocal-rank fusion in an in-browser knowledge base, what we measured, a SurrealDB full-text gotcha, and a filtering bug that gave small collections zero hits. You search your own docs for an error code and the AI hands you a different one. You ask for the v2.3 release notes and v3.2 shows up. The search is not broken in the usual sense: it is doing exactly what a meaning-based search does. We fixed this in our in-browser knowledge bases with hybrid search: BM25 keywords plus vectors, fused with reciprocal-rank fusion. This post covers how it works, the numbers we measured, a database gotcha, and a second bug that made small collections return nothing. This is part 2. Part 1, We rebuilt RAG in a browser tab, tells the whole rebuild, starting with the bug where every vector was empty. Why vectors blur exact strings To a sentence-embedding model, near-identical strings mean near-identical things. That’s exactly what you don’t want for error codes, SKUs, versions, names and IDs. Meaning is the right tool for “how do I get my money back?” and the wrong one for ERR 4521. The setup, all on the user’s machine: Embeddings: all-MiniLM-L6-v2 384-d , running in a Web Worker Store: SurrealDB 2.x compiled to WebAssembly, on IndexedDB how that works: WebAssembly, explained Indexes per chunk table: HNSW cosine for vectors, and a BM25 full-text index with a simple analyzer lowercase, ascii, English snowball stemming How the hybrid query works The query runs two searches and fuses their ranked lists: query ─┬─ embed ───────────────► vector list fetchK nearest └─ keyword terms ───────► BM25 list fetchK best │ reciprocal-rank fusion k = 60 │ optional rerank │ top k Two lists for one question The query is split into keyword terms: words with punctuation trimmed, English, French and German stopwords dropped, at most 12 terms. Each list fetches fetchK = min 200, max k, 4k, 20 candidates, so fusion has enough overlap to work with. Fused by rank, not by score Each document scores Σ 1 / 60 + rank over the lists it appears in, normalised to 0..1. RRF ignores the raw scores, which is the point: BM25 scores and cosine similarities aren’t on the same scale, and RRF never has to compare them. Then a relevance floor Only passages that are actually relevant make the final top k. How that floor works is the subtle part, below. Here’s that subtlety. Because the fused score is rank-based, you can’t put a “minimum relevance” cut-off on it. So we apply the floor to each hit’s cosine similarity computed from the stored vector for keyword-only hits . A hit also counts as relevant by itself if it’s a real keyword match: BM25 0 and more than half of the query terms present. That way an exact code survives even when its embedding scores below the cut-off. The SurrealDB gotcha: @@ is AND-only SurrealDB 2.x’s full-text match operator only matches rows that contain all the tokens, and there’s no OR form. A 6-word question would match almost nothing. The workaround: give each term its own match reference and sum the scores. SELECT id, text, metadata, vector, math::max search::score 1 ?? 0, 0 + math::max search::score 2 ?? 0, 0 + math::max search::score 3 ?? 0, 0 AS bm25 FROM type::table $table WHERE text @1@ $t1 OR text @2@ $t2 OR text @3@ $t3 AND knowledge id = $knowledgeId ORDER BY bm25 DESC LIMIT $k; Summing is legitimate, because BM25 is itself a sum over terms. A missing term adds 0. A term that appears in almost every row can score below 0, so each score is clamped at 0. A nice side effect: a term with several tokens ERR 4521, v2.3, e-mail still needs all of its own tokens, which keeps codes precise. The numbers Our test set: 20 questions over 27 documents, deliberately full of near-duplicate error codes and version families. 12 questions ask for an exact term, 8 are paraphrases. With the real MiniLM model: Vector Hybrid Hybrid + rerank recall@1, all 0.90 0.95 0.95 recall@3 / @5, all 1.00 / 1.00 1.00 / 1.00 1.00 / 1.00 recall@1, exact terms 0.83 0.92 1.00 recall@1, paraphrases 1.00 1.00 0.88 Turn the dial to see what each step bought and what it cost, using the recall@1 figures from the table: What we took from it: Hybrid never did worse than vector, and it fixed exact-term misses at rank 1. It’s the default now. The cross-encoder rerank ms-marco-MiniLM-L-6-v2, on device fixed the last exact-term miss but lost a paraphrase. It’s an opt-in switch “Rerank on this device” , not the default. At @3 everything is already perfect on this set. Rank 1 matters most when only the top few passages fit in a small model’s context. To be clear, this is 20 questions, not a benchmark. We kept it because it encodes the failures our users actually hit, and it runs in CI with a synthetic embedder. That synthetic embedder is much weaker vector 0.35 → hybrid 0.90 at @1 , which makes it a good canary when the keyword path regresses. The second bug: filtered KNN that returns nothing All knowledge bases that use the same embedding model share one table, filtered by knowledge id. In SurrealDB 2.x, the bare KNN operator <|k| on an HNSW index runs a top-k over the whole table and filters afterwards. So imagine a 60-chunk knowledge base next to a 2,940-chunk one in the same table: Ask for the top 4, and the global top-4 can be all from the big one. Filter to the small one, and you get 0 hits. The fix has three parts: Filter during the walk. The <|K,ef| form applies the knowledge id condition while it traverses the HNSW graph. Over-fetch.fetchK = min max 3k, k+8 , 200 , with the search effort ef scaled to it 64 to 800 . An exact fallback. If the approximate query errors or returns fewer than k rows, run an exact filter-first scan: WHERE knowledge id = $kb, cosine similarity, ORDER BY, LIMIT k. We measured both paths on 3,000 × 384-d vectors: the approximate query takes about 14 ms, the exact scan 160 to 556 ms depending on knowledge base size. So the fallback is a safety net, not the main path. A low ef 40 had poor recall on the big knowledge base, hence the scaling. Test this in your own stack: create a tiny collection next to a big one in the same index, and query the tiny one. Many vector stores post-filter by default. Takeaways These questions apply to any RAG stack, whatever store you use. If your models run on device as ours do, running an LLM in the browser explains the runtimes. Have you measured a reranker that hurt? I’d like to know if paraphrases were the casualty for you too. The knowledge bases described here are part of AWFlow, a browser extension for workflows and AI agents that runs locally.