# Why Pure Vector Search Fails on Kannada Literature — And How Hybrid RRF Fixed It > Source: > Published: 2026-08-18 07:40:22+00:00 *A field note on why your RAG app doesn't have a model problem — it has a retrieval problem.* *Built on a 346-page scanned Kannada novel: OCR, hybrid retrieval, reranking, deterministic routing — and the numbers that proved it worked.* The moment I stopped trusting my own system was the moment it confidently answered a question about **page 120** with a passage from an entirely different chapter. I was building a RAG agent for *Heli Hogu Kaarana*, a Kannada novel by Ravi Belagere, digitized from a scanned 346-page PDF. The v1 was the standard recipe: chunk the text, embed with a multilingual model, store in ChromaDB, retrieve top-5, prompt Gemini. It demoed well. It evaluated terribly. This post is everything I learned rebuilding that naive pipeline into a grounded, RAGAS-validated retrieval system — and why **the LLM was never the bottleneck. Retrieval was.** TL;DR - Multilingual embeddings underperform on agglutinative Kannada — hybrid BM25 + dense retrieval fused with Reciprocal Rank Fusion fixed the retrieval failures. - A deterministic regex router bypasses semantic search entirely for page-level queries: 100% precision, zero hallucination surface. - Final system: 0.92 RAGAS faithfulness / 0.89 context recallon a 50-query golden set, at 2.8s P50 end-to-end on serverless. Three things make this a hard problem: **1. The book doesn't exist digitally.** It's a scanned physical book. Kannada OCR is genuinely hard — ligatures, conjunct characters, noisy print. Your retrieval is only as good as your ingestion. **2. Kannada is agglutinative.** A character's name like *Himavant* appears as ಹಿಮವಂತ, ಹಿಮವಂತನ, or ಹಿಮವಂತನಿಗೆ depending on grammatical case. Multilingual embeddings trained on mostly high-resource data compress these into a weak, inconsistent semantic space. **3. Literary text is sparse and specific.** Rare colloquialisms, proper nouns, and page-level references are exactly the queries where cosine similarity goes to die. Generic LLMs have essentially never seen this book. Without perfect retrieval, they don't retrieve — they **invent**. V1 architecture: `chunk → embed (multilingual MiniLM) → ChromaDB → top-5 → Gemini` . Failure modes, in order of severity: **Query:** *"What does Himavant say on page 120?"* **V1 (dense-only):** A fluent paragraph about Himavant's conflict — drawn from a different chapter. No citation. Wrong page. Full confidence. **V2:** The regex router classifies it as an exact-page query. A metadata lookup returns the page-120 chunk in ~milliseconds. The answer quotes it, cited `[Page 120]` . Same LLM behind both. The difference was entirely in retrieval architecture. Five phases, one principle: *never let a single retrieval signal make the final decision.* ``` ┌──────────────────────────┐ │ User Query │ └────────────┬─────────────┘ ▼ ┌──────────────────────────┐ │ Regex Query Router │ └───────┬──────────┬───────┘ exact page │ │ semantic ▼ ▼ ┌───────────────┐ ┌───────────────────────────┐ │ Page Lookup │ │ Dense (ChromaDB) + BM25 │ │ (metadata) │ └─────────────┬─────────────┘ └───────┬───────┘ ▼ │ ┌───────────────────────────┐ │ │ RRF Fusion (k = 60) │ │ └─────────────┬─────────────┘ │ ▼ │ ┌───────────────────────────┐ │ │ Cross-Encoder Rerank │ │ │ + confidence guardrail │ │ └─────────────┬─────────────┘ ▼ ▼ │ ┌───────────────────────────┐ └───────────►│ Context Builder │ └─────────────┬─────────────┘ ▼ ┌───────────────────────────┐ │ Gemini → Groq fallback │ └─────────────┬─────────────┘ ▼ ┌───────────────────────────┐ │ Sarvam TTS / gTTS │ └───────────────────────────┘ ``` Garbage in, hallucination out. The ingestion pipeline: That last point is the quiet hero. Page metadata on every chunk is what makes citations possible and what powers the deterministic router below. If your chunks don't carry provenance, your RAG system can't be held accountable. Here's the uncomfortable truth about multilingual embeddings on Indic languages: the semantic space is undertrained. For exact lexical items — names, places, rare colloquialisms — **BM25 catches what embeddings miss**, because it matches the surface forms that actually appear in the text. To be precise about what that means: BM25 rescues **exact lexical overlap**. It does *not* solve cross-inflection matching — ಹಿಮವಂತನ and ಹಿಮವಂತ are different tokens unless you stem, and my pipeline normalizes Unicode and ligatures, not morphology. Cross-inflection matching remains a partially open problem here; hybrid redundancy and the reranker are what compensate for it. Exhibit B(paraphrased):The query uses a rare colloquialism from chapter 7. Dense-only returns a thematically similar passage from chapter 3 — cosine loved the vibe. Hybrid returns chapter 7, because BM25 matched the exact surface form. That's the whole thesis in one retrieval. But BM25 alone fails at paraphrase. "Explain the protagonist's internal conflict" has zero lexical overlap with the passage that answers it. So neither wins. **Both run, in parallel, on every semantic query.** The problem then is merging. Cosine similarity lives in roughly `[-1, 1]` . BM25 scores are unbounded `[0, ∞)` . You cannot add them. You cannot even meaningfully normalize them. Enter **Reciprocal Rank Fusion** — fuse the *ranks*, not the scores: `RRF_score(d) = Σ_m 1 / (k + rank_m(d))` with `k = 60` . Scale-invariant, dead simple, brutally effective: ``` python def rrf_merge(dense_ranks, sparse_ranks, k=60): scores = {} for rank, doc in enumerate(dense_ranks, 1): scores[doc] = scores.get(doc, 0) + 1 / (k + rank) for rank, doc in enumerate(sparse_ranks, 1): scores[doc] = scores.get(doc, 0) + 1 / (k + rank) return sorted(scores, key=scores.get, reverse=True) ``` Bi-encoders encode query and passage *independently* — fast, but shallow. A **cross-encoder** reads them *together* with full cross-attention. Much more accurate, much more expensive. The compromise: RRF narrows the field to a small candidate set; the cross-encoder (`mmarco-mMiniLMv2-L12-H384-v1` ) reranks only those. Cross-encoder precision at bi-encoder cost. One more guardrail: if the top reranked score falls below a confidence threshold — deliberately conservative, tuned against the golden dataset — **the system refuses to answer.** A graceful "I can't ground this in the text" beats a fluent fabrication, every time. Some queries should never touch semantic search. If the user asks about **page 42**, the correct answer is a metadata lookup — full stop. ``` python import re def route(query: str): m = re.search(r"\b(?:page|ಪುಟ)\s*(\d{1,3})", query) if m: return ("exact_page", int(m.group(1))) return ("semantic", None) ``` A regex intercepts page-intent queries and fetches chunks by page metadata directly: **100% precision, ~5–12ms routing latency, zero hallucination surface** for that entire class of queries. This is the cheapest "AI win" in the whole system. Not every query needs a neural network. Some need a hash lookup. I'm not going to pretend each component contributed equally. Here's the honest failure taxonomy that drove each addition: I've deliberately kept the component-level benchmarks (per-retriever recall@k, reranker MRR lift) in the repo's eval suite — `scripts/eval/eval_hybrid.py` and `eval_reranking.py` — so you can reproduce them on **your** corpus rather than trust mine. The end-to-end numbers below are what the full pipeline scores. I built a 50-query golden dataset from the novel — exact-fact, thematic, multi-hop, and rare-colloquialism queries — and evaluated with **RAGAS**. I treat it as a regression suite, not a benchmark — it catches regressions across query classes; it makes no claim to statistical power. | Metric | Score | Target | |---|---|---| | Faithfulness | 0.92 | > 0.85 | | Answer Relevancy | 0.88 | > 0.80 | | Context Recall | 0.89 | > 0.80 | | Context Precision | 0.85 | > 0.75 | One honesty note: these scores reflect the **primary Gemini path**. Fallback tiers trade marginal quality for availability and were spot-checked, not benchmarked. Latency profile *(warm path; serverless cold start adds ~1–2s on first invocation)*: | Stage | P50 | P95 | |---|---|---| | Query routing | 5ms | 12ms | | BM25 search | 120ms | 250ms | | Dense search | 300ms | 450ms | | RRF + rerank | 450ms | 800ms | | LLM generation | 1.2s | 2.5s | | TTS first byte | 0.8s | 2.0s | End-to-end | 2.8s | 5.0s | The jump from v1 to v2 wasn't a better model. It was better retrieval, measured. `torch` /`transformers` imports are mocked out of the serving path, models load lazily, and peak usage stays under ~600MB RAM.No system postmortem is credible without the scars: Everyone is prompt-engineering. Almost nobody is retrieval-engineering. The LLM was the easiest component in this entire system. The real work — and the real wins — lived in OCR cleanup, rank fusion math, routing logic, and evaluation discipline. If your RAG app hallucinates, don't blame the model. **Audit your retrieval.** **Links:** [GitHub repo](https://github.com/Amruth011/kannada-rag-agent) · [Live demo](https://kannada-rag-agent.vercel.app/) · [Full system design doc](https://github.com/Amruth011/kannada-rag-agent/blob/main/docs/system_design.md) *Built as a non-commercial research/educational demonstration on a scanned personal copy.* *🟢 P.S. I'm currently open to AI/ML engineering roles. If you're building applied AI systems and want someone who ships past the demo stage, my DMs are open.* *If this saved you a hallucination, share it with someone still averaging cosine and BM25 scores.*