Most "AI chatbot" projects stop at a demo: a widget, an OpenAI/Gemini call, a canned prompt. Getting one into production for real businesses — across voice calls, web chat, and WhatsApp, in two languages, without hallucinating — is a different problem. Here's how I approached it.
The Core Challenge
Three requirements shaped the whole architecture:
Multi-channel, single brain — phone calls, web chat, WhatsApp, and Instagram all needed to hit the same knowledge base and produce consistent answers.
No hallucination tolerance — a wrong answer on a live sales call is worse than no answer.
Non-technical operators — the business owner using the admin panel should never need to touch a config file, an API key in code, or a terminal.
The RAG Pipeline
The knowledge layer runs on a fairly standard but carefully tuned RAG stack:
Ingestion: a website crawler pulls headers, paragraphs, and lists from a given URL (or a pasted/uploaded document), and normalizes it into clean text chunks.
Embedding: chunks are embedded using gemini-embedding-001. Storage: vectors are indexed into ChromaDB for fast cosine-similarity retrieval.
Retrieval-time logic: on every incoming query, the system computes similarity scores against the knowledge base. If the top match clears a confidence threshold, it answers from the KB (source: knowledge_base). If not, it falls back to a live, relevant web search (source: search_fallback) rather than letting the LLM freestyle an answer from parametric memory.
That fallback tagging turned out to be one of the most useful design decisions — every logged interaction carries a source field (knowledge_base or search_fallback), which means you get a real-time KB Hit Rate metric for free: how often the system is confidently answering from your own data vs. reaching outside it.
Observability Was Not an Afterthought
A lot of RAG demos skip this, and it's the first thing that breaks trust once you hand a system to a real business. I built a diagnostic layer that surfaces, per query:
Cosine distance / similarity scores against retrieved chunks
Which knowledge base sections were actually matched
End-to-end latency (ms)
Source attribution (KB vs. fallback)
This is exposed directly in a "Diagnostic Playground" in the admin UI — type a query, and see exactly what the retrieval layer matched and why, before it ever reaches a real customer.
Aggregate metrics (Total Conversations, Average Latency, KB Hit Rate, Search Fallback %, Error Rate) roll up from the same interaction logs into an overview dashboard — no separate analytics pipeline needed.
Lead Extraction Without a Structured Form
Because conversations happen in free text (and voice-to-text), lead capture couldn't rely on form fields. The system parses conversational turns for identifiers — name, phone number, email — as they're mentioned naturally ("my name is X and my number is Y"), writes them to a leads table, deduplicates against existing entries, and assigns the lead to a sales rep via round-robin rotation. This runs as a lightweight side-effect of the main conversation loop, not a separate workflow the user has to trigger.
Multi-Tenant Branding Without Multi-Tenant Infra Complexity
Rather than spinning up separate deployments per client, business identity (company name, agent persona, tone, contact details, tagline) is stored as a set of "Business Variables" that get interpolated into the system prompt and voice/chat responses at runtime. One codebase, many brands — which matters a lot if you're an agency or planning to white-label this.
No-Code Data Layer
The default store is local (SQLite) for simplicity, but the admin UI also supports connecting an external Postgres-compatible database (Supabase, Neon, or vanilla Postgres) via a connection string pasted directly into the UI — no backend redeploy required. This was a deliberate trade-off: less "clever" than an ORM migration system, but it means a non-engineer can point the whole system at their own cloud database in under a minute.
Since the target users span Hindi and English speakers, language handling isn't a toggle — the model detects and responds in whichever language the user used, per message, in both voice (STT/TTS) and text channels.
What I'd Do Differently Next
Real-time in-call escalation to a human when confidence is low (currently: unresolved queries become a lead for post-call follow-up — live handoff is the next milestone)
Per-channel confidence thresholds (a WhatsApp typo tolerance vs. a live voice transcript need different tuning)
Try It
If you're building something similar or want to see this running on a real business's data, I opened up a free 10-day trial — happy to walk through the architecture in more depth too.
📧 [nexopersupport@gmail.com](mailto:nexopersupport@gmail.com) · 🌐 nexoper.in
Would genuinely love feedback from anyone who's tuned RAG confidence thresholds for production voice use cases — what's worked for you?