After 20+ hours of compute time on local hardware, I benchmarked 7 RAG configurations against real-world tasks. The results surprised me — and changed how I think about retrieval architecture.
RAG is everywhere in 2026. Everyone claims their pipeline is "SOTA," but most benchmarks use toy datasets. I wanted to see what actually works when you have:
| Method | Approach | Score |
|---|---|---|
| No-RAG | ||
| Direct LLM generation | Baseline | |
| Classical RAG | ||
| Dense retrieval (BGE-small + FAISS) | Poor | |
| Hybrid RAG | ||
| BM25 + Dense + RRF fusion + cross-encoder reranker | Moderate | |
| LightRAG | ||
| Key-value extraction graph + dense hybrid | Disappointing | |
| PageIndex | ||
| Two-stage hierarchical retrieval | Okay | |
| GraphRAG | ||
| Entity graph + dense fallback | Complex | |
| Agentic RAG | ||
| Multi-step reasoning pipeline | Slow, expensive | |
| SEQUOIA | ||
| RAPTOR tree + step-back prompting | Best | |
| SEQUOIA Pro | ||
| Multi-query + rerank + compression | SOTA |
The Twitter-hyped "graph RAG revolution" didn't hold up on real data. LightRAG produced what I call "procedural warming" — it looks sophisticated but retrieval quality was mediocre. Academic benchmarks ≠ production reality.
Most RAG systems fail because they retrieve on the literal query. Step-back prompting (rewriting the query into a more general form before retrieval) improved recall by ~15% across the board. Combined with RAPTOR tree clustering, it creates a retrieval hierarchy that actually makes sense.
I used a local model for summarization and judging. Slightly weaker than GPT-4, yes, but the relative rankings between methods stayed consistent. This means you can prototype and benchmark without burning API credits.
User Query
↓
Step-back Prompting (generalize)
↓
RAPTOR Tree Retrieval (hierarchical clusters)
↓
Rerank + Context Compression
↓
Local LLM Generation
RAPTOR = Recursive Abstractive Processing for Tree-Organized Retrieval. Cluster leaf nodes, summarize upward, retrieve at multiple levels of abstraction.
Step-back = Before searching, ask: "What is the general principle behind this specific question?"
On my test set (banking documents, technical manuals, internal wikis):
| Method | Precision | Recall | Latency |
|---|---|---|---|
| Classical RAG | 0.62 | 0.58 | 120ms |
| Hybrid RAG | 0.71 | 0.65 | 340ms |
| LightRAG | 0.59 | 0.61 | 890ms |
| SEQUOIA | 0.84 | ||
| 0.79 | |||
| 450ms | |||
| SEQUOIA Pro | 0.87 | ||
| 0.82 | |||
| 680ms |
SEQUOIA Pro trades some latency for accuracy. SEQUOIA (basic) is the sweet spot for production.
Everything is open source:
🔗 github.com/Diyago/rag-benchmark
I'm extending SEQUOIA with:
I write about practical AI/ML from inside a bank — RAG systems, LLM deployment, team management, and what actually works vs. what's just hype.
Telegram channel (Russian, technical): AI.Insaf
Have you benchmarked RAG on real data? What surprised you? Drop a comment or reach out on Telegram.
Эта статья также опубликована в Telegram-канале AI.Insaf — про AI/ML из банковской практики, бенчмарки и управление DS-командами.
Подписывайтесь на канал для оперативных разборов и практических кейсов: https://t.me/ai_tablet