{"slug": "sanity-check-on-my-hybrid-rag-settings", "title": "Sanity check on my hybrid RAG settings", "summary": "A developer running a hybrid RAG stack of FastAPI, ChromaDB, PostgreSQL full-text search, Redis and BAAI/bge-reranker-base asked for a review of retrieval settings including a 0.3 similarity threshold, RRF scoring with w=2 for structured records, an 800 ms rerank budget at 256 tokens per chunk, and confidence floors of 0.16 top-1 and 0.20 top-3. A respondent said the stack is fine but flagged the three-way glue between Chroma, Postgres FTS and Redis workers as a drift risk on re-ingestion, recommending pgvector to keep keyword and vector in one place and citing a Pixeltable showcase repository comparing that approach with Celery plus Postgres plus pgvector.", "body_md": "Hi all, I’d love a quick review of my RAG retrieval settings and some advice on the vector store.\n\n**Current stack**\n\n- Backend: FastAPI (Python) with Docker workers for ingestion\n- Vector DB: ChromaDB (one collection per project)\n- Keyword search: PostgreSQL full-text search (`tsvector` ,`ts_rank_cd` )\n- Cache: Redis\n- Embeddings: Mistral, OpenAI, Jina (via Ollama)\n- LLMs: OpenAI, Mistral, Gemini\n- Reranker: `BAAI/bge-reranker-base`\n- Chunking: LlamaIndex `SentenceSplitter`\n\n**Indexing**\n\n- Chunks of 1,000–1,500 characters with 200 overlap. Tables are split by whole rows.\n- A title/breadcrumb header (up to 200 characters) is added before embedding.\n- Normalized embeddings with cosine distance.\n\n**Retrieval**\n\n- sim=1−dsim=1−*d* , with threshold τ=0.3*τ* =0.3 (range 0.2–0.45). If nothing passes, retry at 0.5τ0.5*τ* .\n- Small collections (300 chunks or fewer): score=0.65⋅sim+0.35⋅kwscore=0.65⋅sim+0.35⋅kw\n- Large collections: RRF, score=∑w/(60+rank)score=∑*w* /(60+rank). Vector, keyword and exact-ID lists each have w=1*w* =1, and structured records have w=2*w* =2.\n- Results found only by keyword search are kept if sim≥0.75×sim≥0.75× the best vector similarity.\n\n**Reranking**\n\n- Reranks the top 30 within an 800 ms budget, reading up to 256 tokens per chunk, with d=1−σ(s)*d* =1−*σ* (*s* ).\n- Skipped when the top result has sim≥0.75sim≥0.75 and leads the second by at least 0.10.\n\n**Final context**\n\n- top-k = 5, with near-duplicates removed and at most 3 chunks per page.\n- Confidence requires top-1 similarity of at least 0.16 and a top-3 average of at least 0.20. Otherwise it retries with the original query.\n\n**Questions**\n\n1. Is a weighted mix for small collections and RRF for large ones sensible, or should I use one method everywhere?\n2. Are 256 tokens per chunk too few for `bge-reranker-base` when chunks are about 1,400 characters?\n3. Are the confidence floors (0.16 / 0.20) too low for production?\n4. Since keyword search already runs in Postgres, would you move vectors to **Postgres + pgvector** , or switch to**OpenSearch / Elasticsearch** for native hybrid search (BM25 plus vectors) at multi-tenant scale? Which works better in production?\n\nThanks!\n\n \n \nhonestly your stack is fine, the part I’d worry about is the glue - Chroma + Postgres FTS + Redis workers means 3 places that can drift when a doc gets re-ingested. pgvector is the easy win if you already run Postgres, keeps keyword + vector in one place. we hit the same thing and ended up keeping chunks, embeddings and indexes in one table that updates incrementally in Pixeltable.. we wrote up a side by side vs Celery+Postgres+pgvector here if useful: [github.com/pixeltable/showcase-call-intelligence](https://github.com/pixeltable/showcase-call-intelligence)", "url": "https://wpnews.pro/news/sanity-check-on-my-hybrid-rag-settings", "canonical_source": "https://discuss.huggingface.co/t/sanity-check-on-my-hybrid-rag-settings/190412#post_2", "published_at": "2026-10-12 19:17:22+00:00", "updated_at": "2026-10-12 19:20:27.751360+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "ai-tools"], "entities": ["ChromaDB", "PostgreSQL", "pgvector", "Redis", "FastAPI", "BAAI/bge-reranker-base", "Pixeltable", "LlamaIndex"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/sanity-check-on-my-hybrid-rag-settings", "markdown": "https://wpnews.pro/news/sanity-check-on-my-hybrid-rag-settings.md", "text": "https://wpnews.pro/news/sanity-check-on-my-hybrid-rag-settings.txt", "jsonld": "https://wpnews.pro/news/sanity-check-on-my-hybrid-rag-settings.jsonld"}}