{"slug": "how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops", "title": "How I Built a Bilingual RAG-Powered AI Calling & Chat Agent (With a Full Admin Ops Center)", "summary": "A developer built a bilingual, multi-channel AI calling and chat agent that routes phone, web chat, WhatsApp, and Instagram traffic through a single RAG pipeline backed by ChromaDB and gemini-embedding-001. The system uses a confidence threshold to answer from the knowledge base or fall back to live web search, tagging each interaction with its source to produce a real-time KB Hit Rate metric. It also includes a no-code admin operations center with a diagnostic playground, aggregate dashboards, conversational lead extraction, and runtime business-variable branding for multi-tenant deployments.", "body_md": "Most \"AI chatbot\" projects stop at a demo: a widget, an OpenAI/Gemini call, a canned prompt. Getting one into production for real businesses — across voice calls, web chat, and WhatsApp, in two languages, without hallucinating — is a different problem. Here's how I approached it.\n\nThe Core Challenge\n\nThree requirements shaped the whole architecture:\n\nMulti-channel, single brain — phone calls, web chat, WhatsApp, and Instagram all needed to hit the same knowledge base and produce consistent answers.\n\nNo hallucination tolerance — a wrong answer on a live sales call is worse than no answer.\n\nNon-technical operators — the business owner using the admin panel should never need to touch a config file, an API key in code, or a terminal.\n\nThe RAG Pipeline\n\nThe knowledge layer runs on a fairly standard but carefully tuned RAG stack:\n\nIngestion: a website crawler pulls headers, paragraphs, and lists from a given URL (or a pasted/uploaded document), and normalizes it into clean text chunks.\n\nEmbedding: chunks are embedded using gemini-embedding-001.\n\nStorage: vectors are indexed into ChromaDB for fast cosine-similarity retrieval.\n\nRetrieval-time logic: on every incoming query, the system computes similarity scores against the knowledge base. If the top match clears a confidence threshold, it answers from the KB (source: knowledge_base). If not, it falls back to a live, relevant web search (source: search_fallback) rather than letting the LLM freestyle an answer from parametric memory.\n\nThat fallback tagging turned out to be one of the most useful design decisions — every logged interaction carries a source field (knowledge_base or search_fallback), which means you get a real-time KB Hit Rate metric for free: how often the system is confidently answering from your own data vs. reaching outside it.\n\nObservability Was Not an Afterthought\n\nA lot of RAG demos skip this, and it's the first thing that breaks trust once you hand a system to a real business. I built a diagnostic layer that surfaces, per query:\n\nCosine distance / similarity scores against retrieved chunks\n\nWhich knowledge base sections were actually matched\n\nEnd-to-end latency (ms)\n\nSource attribution (KB vs. fallback)\n\nThis is exposed directly in a \"Diagnostic Playground\" in the admin UI — type a query, and see exactly what the retrieval layer matched and why, before it ever reaches a real customer.\n\nAggregate metrics (Total Conversations, Average Latency, KB Hit Rate, Search Fallback %, Error Rate) roll up from the same interaction logs into an overview dashboard — no separate analytics pipeline needed.\n\nLead Extraction Without a Structured Form\n\nBecause conversations happen in free text (and voice-to-text), lead capture couldn't rely on form fields. The system parses conversational turns for identifiers — name, phone number, email — as they're mentioned naturally (\"my name is X and my number is Y\"), writes them to a leads table, deduplicates against existing entries, and assigns the lead to a sales rep via round-robin rotation. This runs as a lightweight side-effect of the main conversation loop, not a separate workflow the user has to trigger.\n\nMulti-Tenant Branding Without Multi-Tenant Infra Complexity\n\nRather than spinning up separate deployments per client, business identity (company name, agent persona, tone, contact details, tagline) is stored as a set of \"Business Variables\" that get interpolated into the system prompt and voice/chat responses at runtime. One codebase, many brands — which matters a lot if you're an agency or planning to white-label this.\n\nNo-Code Data Layer\n\nThe default store is local (SQLite) for simplicity, but the admin UI also supports connecting an external Postgres-compatible database (Supabase, Neon, or vanilla Postgres) via a connection string pasted directly into the UI — no backend redeploy required. This was a deliberate trade-off: less \"clever\" than an ORM migration system, but it means a non-engineer can point the whole system at their own cloud database in under a minute.\n\nSince the target users span Hindi and English speakers, language handling isn't a toggle — the model detects and responds in whichever language the user used, per message, in both voice (STT/TTS) and text channels.\n\nWhat I'd Do Differently Next\n\nReal-time in-call escalation to a human when confidence is low (currently: unresolved queries become a lead for post-call follow-up — live handoff is the next milestone)\n\nPer-channel confidence thresholds (a WhatsApp typo tolerance vs. a live voice transcript need different tuning)\n\nTry It\n\nIf you're building something similar or want to see this running on a real business's data, I opened up a free 10-day trial — happy to walk through the architecture in more depth too.\n\n📧 [nexopersupport@gmail.com](mailto:nexopersupport@gmail.com) · 🌐 nexoper.in\n\nWould genuinely love feedback from anyone who's tuned RAG confidence thresholds for production voice use cases — what's worked for you?", "url": "https://wpnews.pro/news/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops", "canonical_source": "https://dev.to/nexoper/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops-center-573o", "published_at": "2026-09-13 19:56:29+00:00", "updated_at": "2026-09-13 20:20:38.580427+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-products", "ai-tools"], "entities": ["ChromaDB", "gemini-embedding-001", "WhatsApp", "Instagram", "SQLite", "Supabase", "Neon", "Postgres"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops", "markdown": "https://wpnews.pro/news/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops.md", "text": "https://wpnews.pro/news/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops.txt", "jsonld": "https://wpnews.pro/news/how-i-built-a-bilingual-rag-powered-ai-calling-chat-agent-with-a-full-admin-ops.jsonld"}}