Sanity check on my hybrid RAG settings

Hi all, I’d love a quick review of my RAG retrieval settings and some advice on the vector store.

Current stack

- Backend: FastAPI (Python) with Docker workers for ingestion
- Vector DB: ChromaDB (one collection per project)
- Keyword search: PostgreSQL full-text search (`tsvector` ,`ts_rank_cd` )
- Cache: Redis
- Embeddings: Mistral, OpenAI, Jina (via Ollama)
- LLMs: OpenAI, Mistral, Gemini
- Reranker: `BAAI/bge-reranker-base`

• Chunking: LlamaIndex SentenceSplitter

Indexing

• Chunks of 1,000–1,500 characters with 200 overlap. Tables are split by whole rows.
• A title/breadcrumb header (up to 200 characters) is added before embedding.
• Normalized embeddings with cosine distance.

Retrieval

- sim=1−dsim=1−*d* , with threshold τ=0.3*τ* =0.3 (range 0.2–0.45). If nothing passes, retry at 0.5τ0.5*τ* .
- Small collections (300 chunks or fewer): score=0.65⋅sim+0.35⋅kwscore=0.65⋅sim+0.35⋅kw
- Large collections: RRF, score=∑w/(60+rank)score=∑*w* /(60+rank). Vector, keyword and exact-ID lists each have w=1*w* =1, and structured records have w=2*w* =2.

• Results found only by keyword search are kept if sim≥0.75×sim≥0.75× the best vector similarity.

Reranking

• Reranks the top 30 within an 800 ms budget, reading up to 256 tokens per chunk, with d=1−σ(s)d =1−σ (s ).
• Skipped when the top result has sim≥0.75sim≥0.75 and leads the second by at least 0.10.

Final context

• top-k = 5, with near-duplicates removed and at most 3 chunks per page.
• Confidence requires top-1 similarity of at least 0.16 and a top-3 average of at least 0.20. Otherwise it retries with the original query.

Questions

1. Is a weighted mix for small collections and RRF for large ones sensible, or should I use one method everywhere?
2. Are 256 tokens per chunk too few for bge-reranker-base when chunks are about 1,400 characters?
3. Are the confidence floors (0.16 / 0.20) too low for production?
4. Since keyword search already runs in Postgres, would you move vectors to Postgres + pgvector , or switch toOpenSearch / Elasticsearch for native hybrid search (BM25 plus vectors) at multi-tenant scale? Which works better in production?

Thanks!

honestly your stack is fine, the part I’d worry about is the glue - Chroma + Postgres FTS + Redis workers means 3 places that can drift when a doc gets re-ingested. pgvector is the easy win if you already run Postgres, keeps keyword + vector in one place. we hit the same thing and ended up keeping chunks, embeddings and indexes in one table that updates incrementally in Pixeltable.. we wrote up a side by side vs Celery+Postgres+pgvector here if useful: github.com/pixeltable/showcase-call-intelligence