[Read Part 1 ↗](https://medium.com/@victormocioiu/production-grade-hybrid-search-rag-on-116-month-of-compute-part-1-the-pipeline-the-deede53d461d)·
[GitHub](https://github.com/victormocioiu/rag-api)
Hybrid retrieval with receipts.
#9
on EnterpriseRAG-Bench — officially scored 512K docs benchmarked
88ms BM25 at 2M chunks
€116 a month of Hetzner… yes, that is the whole cluster
00 · 01
The platform, answering #
actual screenshots of the live thing… the 512K-document playground, streamed answers, citations you can open. no mockups — mockups would have been easier.
01 · 02
What runs under the hood #
BM25 inside PostgresNo search cluster to babysit.
pg_textsearch gives real IDF ranking with Block-Max WAND: 88ms over 2 million chunks, where plain tsquery took 7.5 seconds.
Hybrid fusion, weightedFusion is not a democracy.
Vector and lexical arms fuse with a measured 0.3 vector weight — equal weighting scored worse than turning one arm off.
Cross-encoder rerankerThe right eight chunks.
A fourth service reads query and chunk together and reorders the window. Worth +3.9 on the public benchmark… and 16 seconds of latency on 2 vCPUs, which is why it ships as a checkbox and not a default. Both numbers are published.
Grounded answersRefusal as a feature.
Empty retrieval refuses without calling the model. On the benchmark's info-not-found questions: 100% correct while others hallucinate.
Tenants by row-level securityIsolation is the floor, not a filter.
Every query runs inside a tenant transaction Postgres enforces. The benchmark corpus and your sandbox are neighbors that cannot meet.
Streaming citations[n] means something. Sources arrive before the first token; every bracket in an answer opens to the exact chunk that backs it.
02 · 03
How the numbers stay honest #
- 1 Measure before building The reranker cost $1 to prototype offline before a line of service code existed. - 2 Publish the noes Multi-query rewriting, wider context, prompt-only fixes, HyDE — measured, failed, documented next to the wins. The noes cost real money; hiding them would waste it twice. - 3 One database Vectors, BM25, text, tenants, and usage all live in Postgres. One thing to operate, one thing to trust. - 4 Small models, priced honestly A 118M embedder and budget answerers, benchmarked against clouds that cost 100× more.
03 · 04
What it costs #
- Five-node Kubernetes cluster on Hetzner
- Postgres with vectors, BM25, and row-level security
- Four services: ingest, embed, rerank, answer
- A public benchmark score you can reproduce
- All code MIT, all numbers receipted
€116
Answers cost tenths of a cent; the benchmark that proved them cost about $60 once.
Try it now — no login The 512K-document playground is open to everyone, no login. Sign in with Google or GitHub only for your own private sandbox.
per month — the entire five-node cluster
04 · 05
Fair questions #
- What am I actually chatting with?
- EnterpriseRAG-Bench: 512,000 simulated company documents — Slack threads, emails, wikis, tickets… the same corpus the platform is publicly scored on. what you see is exactly what the benchmark saw.
- Can I bring my own documents?
- Yes — sign in with Google or GitHub and you get a private sandbox: 10 documents, 20 pages each, a daily token budget, and meters that show exactly what's left. A page is counted in tokens, so no… one huge .txt file is not "one page", we checked. Row-level security keeps your docs yours.
- How good is it, honestly?
- Officially scored on EnterpriseRAG-Bench: #9, overall 44.74 (correctness 52.6, completeness 54.38, document recall 69.65) — ahead of Vertex AI Search and NVIDIA, one spot below Azure. On document recall we beat both Azure (64.25) and Vertex (61.76): the retrieval competes with the managed clouds outright, on a €116/month box. Thelive leaderboard is on Hugging Face; per-category tables are in the repo. - Why should I trust these numbers?
- You shouldn't — not on our word alone. Every run's raw results are committed to the repo, and when the benchmark team re-scored our submission with their own judge, their recall landed within 0.05 points of ours (69.65 vs 69.6). Retrieval validated to a rounding error. Verify, then trust.
- Can I run this myself?
- That is the whole point… the articles walk through every deployment click and every benchmark, and the repos build the platform from scratch. Copy freely, measure everything.