{"slug": "production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and", "title": "Production RAG at Scale: HMAC Cookies, Workspace Isolation, Hybrid Retrieval, and Citation Validation", "summary": "A developer built a production RAG system addressing four key challenges: stateless HMAC-signed guest cookies for zero-friction onboarding, layered workspace isolation for multi-tenancy, hybrid retrieval combining BM25 and vector search with Reciprocal Rank Fusion, and a citation validation pipeline achieving 74.6% precision. The system, built with Next.js, PostgreSQL, pgvector, and local Ollama models, reports 66.7% retrieval recall, 80% answer correctness, and zero cross-workspace data leaks in a 15-case evaluation.", "body_md": "#\nProduction RAG at Scale: HMAC Guest Cookies, Workspace Isolation, Hybrid Retrieval, and Citation Validation\n\n##\nExecutive Summary\n\nI built a production RAG system that solves four hard problems:\n\n-\n**Zero-friction onboarding:** Stateless HMAC-signed guest cookies (no database overhead, 1h TTL, timing-safe validation)\n-\n**Enterprise multi-tenancy:** Layered permission checks (retrieval-layer, session-layer, mutation-layer isolation)\n-\n**Hybrid search at scale:** Keyword (BM25 via PostgreSQL `tsvector`\n\n) + vector (pgvector) merged with Reciprocal Rank Fusion\n-\n**Grounded citations:** Citation validation pipeline (extract, verify, deduplicate) achieving 74.6% precision\n\n**Metrics (15-case evaluation):**\n\n- Retrieval recall: 66.7%\n- Citation precision: 74.6%\n- Answer correctness: 80%\n- No-answer accuracy: 100%\n- Cross-workspace data leaks: 0 (verified with SQL injection tests)\n\n**Tech stack:** Next.js, PostgreSQL + pgvector, Ollama (local Mistral + nomic-embed-text), Tailscale Funnel, 170+ tests.\n\n**Live demo:** [https://rag-system-ashen.vercel.app](https://rag-system-ashen.vercel.app)\n\n**Source:** [https://github.com/KasaVarun/rag-system](https://github.com/KasaVarun/rag-system)\n\n##\nPart 1: The Problem Space\n\n###\nWhy RAG Systems Fail in Production\n\nMost RAG implementations I've seen in the wild have one or more of these issues:\n\n**1. Hallucination without accountability**\n\nFixing this requires citations that are *verified against the source*, not just string-searched in the output.\n\n**2. No multi-tenancy isolation**\n\nThis is catastrophic. It needs defense-in-depth: permission checks at retrieval, at session resolution, at mutation. Not just a role check.\n\n**3. Demo friction**\n\nMost systems require full signup. We flip the model: try first, authenticate later.\n\n**4. No quality measurement**\n\nWithout evaluation, you optimize for the wrong things. We built a 15-case framework with recall/precision/latency metrics.\n\n##\nPart 2: Stateless Guest Authentication with HMAC Cookies\n\n###\nWhy Not Traditional Sessions?\n\nTraditional approach for guests:\n\n- Click \"Try Now\"\n- Server generates session ID\n- Store session in database (user_id, expiration, permissions)\n- Return session cookie to client\n- On every request: database lookup to validate session\n\n**Problems:**\n\n-\n**Database overhead:** Millions of demo users = bloated sessions table\n-\n**State management:** Session invalidation, cleanup, TTL expiration requires cron jobs\n-\n**Complexity:** Session store now needs clustering, replication, cache invalidation\n\n###\nHMAC-Signed Cookie Design\n\nInstead, encode everything in the cookie itself and sign it cryptographically.\n\n###\nCookie Setting and Validation\n\n###\nSession Resolution Logic\n\nThe critical part: how do we resolve a session when a request comes in?\n\n###\nWhy This Design Works\n\n**Advantages:**\n\n-\n**No database writes for guests:** Session table doesn't bloat. Millions of demo users = zero overhead.\n-\n**Stateless:** Cookie contains all information. Can scale horizontally without session replication.\n-\n**Tamper-proof:** HMAC signature cryptographically prevents guest from modifying expiration.\n-\n**Timing-safe:** Comparison takes constant time regardless of where mismatch occurs (prevents timing attacks).\n-\n**Self-contained:** Single cookie lookup, then one workspace membership check. ~2ms total.\n-\n**Revocable:** Rotate SESSION_SECRET and all existing guest cookies invalidate immediately.\n\n**Edge Cases Handled:**\n\n- Expired token:\n`parseGuestCookieValue`\n\nreturns `{ isValid: false }`\n\n→ rejected\n- Modified expiration: HMAC signature won't match → rejected\n- Real session exists: Real session wins, guest cookie ignored → no conflict\n- CSRF attack: Origin header checked before minting → blocked\n- XSS attack: Cookie is\n`HttpOnly`\n\n→ inaccessible to JavaScript\n\n##\nPart 3: Workspace Isolation - Defense in Depth\n\n###\nThe Three-Layer Permission Model\n\nMost systems check permissions once. We check at three independent layers. If one layer has a bug, the others catch it.\n\n###\nLayer 1: Retrieval Permission Check\n\nWhen retrieving chunks, only return chunks from the user's workspace.\n\n**Key insight:** The `WHERE d.workspace_id = $2`\n\nclause is in the *database query itself*. Even if the application layer has a bug and forgets to check permissions, the database enforces isolation.\n\n###\nLayer 2: Session Permission Check\n\nBefore processing a request, verify the user can access the requested workspace.\n\n###\nLayer 3: Mutation Permission Check\n\nGuests and viewers can't modify data. Only owners can.\n\n###\nVerification: SQL Injection Tests\n\nWe prove this works with SQL injection tests:\n\n**Result:** 0 cross-workspace data leaks verified across 170+ test cases.\n\n##\nPart 4: Hybrid Retrieval with Reciprocal Rank Fusion\n\n###\nThe Problem: Single-Modality Search Blindness\n\nPure vector search:\n\n- ✓ Great for semantic similarity (\"vacation days\" ~ \"time off\")\n- ✗ Fails on exact keywords (\"Q3 revenue\" returns nothing if corpus says \"third quarter revenue\")\n- ✗ Fails on rare terms (uncommon acronyms, specific product names)\n\nPure keyword search (BM25):\n\n- ✓ Excellent for exact matches and rare terms\n- ✗ Misses semantic relationships (can't connect \"vacation\" to \"PTO\")\n- ✗ No semantic ranking (all exact matches scored equally)\n\n###\nSolution: Hybrid Search with RRF\n\nCombine both approaches and merge rankings using Reciprocal Rank Fusion.\n\n###\nWhy RRF Works\n\nRRF doesn't normalize scores (which vary by modality). Instead, it uses reciprocal ranks:\n\nThis balances both signals naturally without manual weighting.\n\n###\nPerformance Characteristics\n\nHybrid search remains sub-100ms even at scale.\n\n##\nPart 5: Citation Validation Pipeline\n\n###\nThe Citation Problem\n\nRaw LLM output:\n\nThe model cited [2] but fabricated it. We need to *verify* every citation exists in the source chunks.\n\n###\nCitation Validation Pipeline\n\n###\nIntegration in Answer Generation\n\n###\nCitation Metrics\n\nFrom 15-case evaluation:\n\nTrade-off: Remove some valid citations to eliminate hallucinations.\n\n##\nPart 6: Evaluation Framework\n\n###\nWhy Metrics Matter\n\nWithout measurement, you're guessing:\n\nWith metrics, you *know*:\n\n###\n15-Case Evaluation Suite\n\n###\nResults\n\n##\nPart 7: Deployment & Operations\n\n###\nLocal Ollama via Tailscale Funnel\n\nFor production, expose local Ollama securely without opening ports:\n\nThen set environment variable:\n\n**Why Tailscale Funnel instead of ngrok/CloudFlare Tunnel?**\n\n- ✓ End-to-end encryption (device-to-device via Tailscale mesh)\n- ✓ Identity-based access (only authenticated users)\n- ✓ Built-in certificate management\n- ✓ Free tier is generous\n- ✓ No token rotation needed\n\n###\nDatabase Schema\n\n###\nTesting\n\n##\nConclusion\n\nBuilding production RAG means solving:\n\n-\n**Friction:** HMAC-signed stateless guest cookies eliminate signup overhead\n-\n**Security:** Layered permission checks prevent data leaks\n-\n**Search quality:** Hybrid retrieval (keyword + vector) beats either alone\n-\n**Trust:** Citation validation makes hallucinations detectable\n-\n**Measurement:** Evaluation frameworks replace guessing\n\nThe architecture scales to enterprise workloads while remaining interpretable and maintainable.\n\n**Questions? Code issues? Open an issue on GitHub.**\n\n**Varun Kasa**\n\nML/AI Engineer\n\n[GitHub](https://github.com/KasaVarun) | [LinkedIn](https://linkedin.com/in/varunkasa) | [Portfolio](https://varunkasa.vercel.app)", "url": "https://wpnews.pro/news/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and", "canonical_source": "https://dev.to/kasavarun/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and-citation-4blc", "published_at": "2026-08-31 01:09:20+00:00", "updated_at": "2026-08-31 01:51:45.037431+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["Next.js", "PostgreSQL", "pgvector", "Ollama", "Mistral", "nomic-embed-text", "Tailscale Funnel", "KasaVarun"], "alternates": {"html": "https://wpnews.pro/news/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and", "markdown": "https://wpnews.pro/news/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and.md", "text": "https://wpnews.pro/news/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and.txt", "jsonld": "https://wpnews.pro/news/production-rag-at-scale-hmac-cookies-workspace-isolation-hybrid-retrieval-and.jsonld"}}