#
Production RAG at Scale: HMAC Guest Cookies, Workspace Isolation, Hybrid Retrieval, and Citation Validation
#
Executive Summary
I built a production RAG system that solves four hard problems:
Zero-friction onboarding: Stateless HMAC-signed guest cookies (no database overhead, 1h TTL, timing-safe validation) #
Enterprise multi-tenancy: Layered permission checks (retrieval-layer, session-layer, mutation-layer isolation) #
Hybrid search at scale: Keyword (BM25 via PostgreSQL tsvector
) + vector (pgvector) merged with Reciprocal Rank Fusion #
Grounded citations: Citation validation pipeline (extract, verify, deduplicate) achieving 74.6% precision
**Metrics (15-case evaluation):**
- Retrieval recall: 66.7%
- Citation precision: 74.6%
- Answer correctness: 80%
- No-answer accuracy: 100%
- Cross-workspace data leaks: 0 (verified with SQL injection tests)
Tech stack: Next.js, PostgreSQL + pgvector, Ollama (local Mistral + nomic-embed-text), Tailscale Funnel, 170+ tests.
**Live demo:** [https://rag-system-ashen.vercel.app](https://rag-system-ashen.vercel.app)
**Source:** [https://github.com/KasaVarun/rag-system](https://github.com/KasaVarun/rag-system)
#
Part 1: The Problem Space
Why RAG Systems Fail in Production
Most RAG implementations I've seen in the wild have one or more of these issues:
1. Hallucination without accountability
Fixing this requires citations that are verified against the source, not just string-searched in the output.
2. No multi-tenancy isolation
This is catastrophic. It needs defense-in-depth: permission checks at retrieval, at session resolution, at mutation. Not just a role check.
3. Demo friction
Most systems require full signup. We flip the model: try first, authenticate later.
4. No quality measurement
Without evaluation, you optimize for the wrong things. We built a 15-case framework with recall/precision/latency metrics.
#
Part 2: Stateless Guest Authentication with HMAC Cookies
Why Not Traditional Sessions?
Traditional approach for guests:
- Click "Try Now"
- Server generates session ID
- Store session in database (user_id, expiration, permissions)
- Return session cookie to client
- On every request: database lookup to validate session
Problems:
Database overhead: Millions of demo users = bloated sessions table #
State management: Session invalidation, cleanup, TTL expiration requires cron jobs #
Complexity: Session store now needs clustering, replication, cache invalidation
HMAC-Signed Cookie Design
Instead, encode everything in the cookie itself and sign it cryptographically.
Cookie Setting and Validation
Session Resolution Logic
The critical part: how do we resolve a session when a request comes in?
Why This Design Works
Advantages:
No database writes for guests: Session table doesn't bloat. Millions of demo users = zero overhead. #
Stateless: Cookie contains all information. Can scale horizontally without session replication. #
Tamper-proof: HMAC signature cryptographically prevents guest from modifying expiration. #
Timing-safe: Comparison takes constant time regardless of where mismatch occurs (prevents timing attacks). #
Self-contained: Single cookie lookup, then one workspace membership check. ~2ms total. #
Revocable: Rotate SESSION_SECRET and all existing guest cookies invalidate immediately.
Edge Cases Handled:
- Expired token:
parseGuestCookieValue
returns { isValid: false }
→ rejected
- Modified expiration: HMAC signature won't match → rejected
- Real session exists: Real session wins, guest cookie ignored → no conflict
- CSRF attack: Origin header checked before minting → blocked
- XSS attack: Cookie is
HttpOnly
→ inaccessible to JavaScript
#
Part 3: Workspace Isolation - Defense in Depth
The Three-Layer Permission Model
Most systems check permissions once. We check at three independent layers. If one layer has a bug, the others catch it.
Layer 1: Retrieval Permission Check
When retrieving chunks, only return chunks from the user's workspace.
Key insight: The WHERE d.workspace_id = $2
clause is in the database query itself. Even if the application layer has a bug and forgets to check permissions, the database enforces isolation.
Layer 2: Session Permission Check
Before processing a request, verify the user can access the requested workspace.
Layer 3: Mutation Permission Check
Guests and viewers can't modify data. Only owners can.
Verification: SQL Injection Tests
We prove this works with SQL injection tests:
Result: 0 cross-workspace data leaks verified across 170+ test cases.
#
Part 4: Hybrid Retrieval with Reciprocal Rank Fusion
The Problem: Single-Modality Search Blindness
Pure vector search:
- ✓ Great for semantic similarity ("vacation days" ~ "time off")
- ✗ Fails on exact keywords ("Q3 revenue" returns nothing if corpus says "third quarter revenue")
- ✗ Fails on rare terms (uncommon acronyms, specific product names)
Pure keyword search (BM25):
-
✓ Excellent for exact matches and rare terms
-
✗ Misses semantic relationships (can't connect "vacation" to "PTO")
-
✗ No semantic ranking (all exact matches scored equally)
Solution: Hybrid Search with RRF
Combine both approaches and merge rankings using Reciprocal Rank Fusion.
Why RRF Works
RRF doesn't normalize scores (which vary by modality). Instead, it uses reciprocal ranks:
This balances both signals naturally without manual weighting.
Performance Characteristics
Hybrid search remains sub-100ms even at scale.
#
Part 5: Citation Validation Pipeline
The Citation Problem
Raw LLM output:
The model cited [2] but fabricated it. We need to verify every citation exists in the source chunks.
Citation Validation Pipeline
Integration in Answer Generation
Citation Metrics
From 15-case evaluation: Trade-off: Remove some valid citations to eliminate hallucinations.
#
Part 6: Evaluation Framework
Why Metrics Matter
Without measurement, you're guessing:
With metrics, you know:
15-Case Evaluation Suite
Results
#
Part 7: Deployment & Operations
Local Ollama via Tailscale Funnel
For production, expose local Ollama securely without opening ports: Then set environment variable:
Why Tailscale Funnel instead of ngrok/CloudFlare Tunnel?
- ✓ End-to-end encryption (device-to-device via Tailscale mesh)
- ✓ Identity-based access (only authenticated users)
- ✓ Built-in certificate management
- ✓ Free tier is generous
- ✓ No token rotation needed
Database Schema
Testing
#
Conclusion
Building production RAG means solving:
Friction: HMAC-signed stateless guest cookies eliminate signup overhead #
Security: Layered permission checks prevent data leaks #
Search quality: Hybrid retrieval (keyword + vector) beats either alone #
Trust: Citation validation makes hallucinations detectable #
Measurement: Evaluation frameworks replace guessing
The architecture scales to enterprise workloads while remaining interpretable and maintainable.
Questions? Code issues? Open an issue on GitHub.
Varun Kasa
ML/AI Engineer