# Sanity check on my hybrid RAG settings

> Source: <https://discuss.huggingface.co/t/sanity-check-on-my-hybrid-rag-settings/190412#post_2>
> Published: 2026-10-12 19:17:22+00:00

Hi all, I’d love a quick review of my RAG retrieval settings and some advice on the vector store.

**Current stack**

- Backend: FastAPI (Python) with Docker workers for ingestion
- Vector DB: ChromaDB (one collection per project)
- Keyword search: PostgreSQL full-text search (`tsvector` ,`ts_rank_cd` )
- Cache: Redis
- Embeddings: Mistral, OpenAI, Jina (via Ollama)
- LLMs: OpenAI, Mistral, Gemini
- Reranker: `BAAI/bge-reranker-base`
- Chunking: LlamaIndex `SentenceSplitter`

**Indexing**

- Chunks of 1,000–1,500 characters with 200 overlap. Tables are split by whole rows.
- A title/breadcrumb header (up to 200 characters) is added before embedding.
- Normalized embeddings with cosine distance.

**Retrieval**

- sim=1−dsim=1−*d* , with threshold τ=0.3*τ* =0.3 (range 0.2–0.45). If nothing passes, retry at 0.5τ0.5*τ* .
- Small collections (300 chunks or fewer): score=0.65⋅sim+0.35⋅kwscore=0.65⋅sim+0.35⋅kw
- Large collections: RRF, score=∑w/(60+rank)score=∑*w* /(60+rank). Vector, keyword and exact-ID lists each have w=1*w* =1, and structured records have w=2*w* =2.
- Results found only by keyword search are kept if sim≥0.75×sim≥0.75× the best vector similarity.

**Reranking**

- Reranks the top 30 within an 800 ms budget, reading up to 256 tokens per chunk, with d=1−σ(s)*d* =1−*σ* (*s* ).
- Skipped when the top result has sim≥0.75sim≥0.75 and leads the second by at least 0.10.

**Final context**

- top-k = 5, with near-duplicates removed and at most 3 chunks per page.
- Confidence requires top-1 similarity of at least 0.16 and a top-3 average of at least 0.20. Otherwise it retries with the original query.

**Questions**

1. Is a weighted mix for small collections and RRF for large ones sensible, or should I use one method everywhere?
2. Are 256 tokens per chunk too few for `bge-reranker-base` when chunks are about 1,400 characters?
3. Are the confidence floors (0.16 / 0.20) too low for production?
4. Since keyword search already runs in Postgres, would you move vectors to **Postgres + pgvector** , or switch to**OpenSearch / Elasticsearch** for native hybrid search (BM25 plus vectors) at multi-tenant scale? Which works better in production?

Thanks!

 
 
honestly your stack is fine, the part I’d worry about is the glue - Chroma + Postgres FTS + Redis workers means 3 places that can drift when a doc gets re-ingested. pgvector is the easy win if you already run Postgres, keeps keyword + vector in one place. we hit the same thing and ended up keeping chunks, embeddings and indexes in one table that updates incrementally in Pixeltable.. we wrote up a side by side vs Celery+Postgres+pgvector here if useful: [github.com/pixeltable/showcase-call-intelligence](https://github.com/pixeltable/showcase-call-intelligence)