Sanity check on my hybrid RAG settings

A developer running a hybrid RAG stack of FastAPI, ChromaDB, PostgreSQL full-text search, Redis and BAAI/bge-reranker-base asked for a review of retrieval settings including a 0.3 similarity threshold, RRF scoring with w=2 for structured records, an 800 ms rerank budget at 256 tokens per chunk, and confidence floors of 0.16 top-1 and 0.20 top-3. A respondent said the stack is fine but flagged the three-way glue between Chroma, Postgres FTS and Redis workers as a drift risk on re-ingestion, recommending pgvector to keep keyword and vector in one place and citing a Pixeltable showcase repository comparing that approach with Celery plus Postgres plus pgvector.

Hi all, I’d love a quick review of my RAG retrieval settings and some advice on the vector store. Current stack - Backend: FastAPI Python with Docker workers for ingestion - Vector DB: ChromaDB one collection per project - Keyword search: PostgreSQL full-text search tsvector , ts rank cd - Cache: Redis - Embeddings: Mistral, OpenAI, Jina via Ollama - LLMs: OpenAI, Mistral, Gemini - Reranker: BAAI/bge-reranker-base - Chunking: LlamaIndex SentenceSplitter Indexing - Chunks of 1,000–1,500 characters with 200 overlap. Tables are split by whole rows. - A title/breadcrumb header up to 200 characters is added before embedding. - Normalized embeddings with cosine distance. Retrieval - sim=1−dsim=1− d , with threshold τ=0.3 τ =0.3 range 0.2–0.45 . If nothing passes, retry at 0.5τ0.5 τ . - Small collections 300 chunks or fewer : score=0.65⋅sim+0.35⋅kwscore=0.65⋅sim+0.35⋅kw - Large collections: RRF, score=∑w/ 60+rank score=∑ w / 60+rank . Vector, keyword and exact-ID lists each have w=1 w =1, and structured records have w=2 w =2. - Results found only by keyword search are kept if sim≥0.75×sim≥0.75× the best vector similarity. Reranking - Reranks the top 30 within an 800 ms budget, reading up to 256 tokens per chunk, with d=1−σ s d =1− σ s . - Skipped when the top result has sim≥0.75sim≥0.75 and leads the second by at least 0.10. Final context - top-k = 5, with near-duplicates removed and at most 3 chunks per page. - Confidence requires top-1 similarity of at least 0.16 and a top-3 average of at least 0.20. Otherwise it retries with the original query. Questions 1. Is a weighted mix for small collections and RRF for large ones sensible, or should I use one method everywhere? 2. Are 256 tokens per chunk too few for bge-reranker-base when chunks are about 1,400 characters? 3. Are the confidence floors 0.16 / 0.20 too low for production? 4. Since keyword search already runs in Postgres, would you move vectors to Postgres + pgvector , or switch to OpenSearch / Elasticsearch for native hybrid search BM25 plus vectors at multi-tenant scale? Which works better in production? Thanks honestly your stack is fine, the part I’d worry about is the glue - Chroma + Postgres FTS + Redis workers means 3 places that can drift when a doc gets re-ingested. pgvector is the easy win if you already run Postgres, keeps keyword + vector in one place. we hit the same thing and ended up keeping chunks, embeddings and indexes in one table that updates incrementally in Pixeltable.. we wrote up a side by side vs Celery+Postgres+pgvector here if useful: github.com/pixeltable/showcase-call-intelligence https://github.com/pixeltable/showcase-call-intelligence