{"slug": "how-i-built-maya-a-real-time-voice-ai-clinic-receptionist", "title": "How I Built Maya: A Real-Time Voice AI Clinic Receptionist", "summary": "A developer built Maya, a real-time voice AI receptionist for a fictional Bengaluru clinic, using LiveKit, Groq, and Cartesia to handle appointment bookings with sub-second latency. The project showcases a full voice pipeline and addresses challenges like rate limits by compressing prompts to cut token usage by 73%.", "body_md": "Most AI demos you see online are either simple text chatbots or basic wrappers around an API.\n\nI wanted to build something that felt like a real product solving a real problem. So I built Maya—a real-time voice AI receptionist for a fictional clinic in Bengaluru called SwasthyaCare Clinic.\n\nInstead of typing into a chat box, you just talk to your microphone like you're on a real phone call. Maya answers your questions, checks doctor schedules, books appointments, reschedules or cancels existing visits, and gives clinic\n\ninformation—all in real-time with sub-second voice latency.\n\nHere is a breakdown of how the architecture works, the real issues I faced during development (rate limits, background noise, Docker container paths), and how I solved them.\n\nWhen someone calls a clinic, they don't want to navigate a robotic IVR menu (\"Press 1 for appointments...\"). They want to talk to a human receptionist.\n\nMaya handles that conversational flow:\n\n• Checks live doctor availability: Understands dates and doctor specialties (General Physician vs. Dentist).\n\n• Gathers details naturally: Asks for missing information (name, 10-digit mobile number, preferred slot) across multiple turns.\n\n• Explicit confirmation before booking: Summarizes the appointment details and only commits the booking once the user says \"Yes\".\n\n• Enforces business rules: Only books up to 14 days in advance, checks for slot collisions, and enforces a strict 2-hour cancellation/rescheduling policy.\n\n• Medical triage guardrails: If someone mentions severe symptoms like acute chest pain or breathing issues, Maya immediately instructs them to dial 112 or visit an emergency room rather than booking a routine outpatient slot.\n\nTo make voice feel natural, latency has to be as close to human conversational speed as possible. A typical turn looks like this:\n\n```\nUser speaks into Browser Mic\n       ↓ (WebRTC Audio Stream)\nLiveKit Cloud (ap-south / Mumbai)\n       ↓\nSilero VAD (Voice Activity Detection on-device)\n       ↓\nGroq Whisper (Speech-to-Text)\n       ↓\nGroq LLM (Reasoning + Function/Tool Calling)\n       ↓\nSQLite Database (Appointment State Engine)\n       ↓\nCartesia TTS (Streaming Indian English Voice)\n       ↓ (WebRTC Audio)\nBrowser Speakers (User hears response)\n```\n\n• Audio Transport: LiveKit (WebRTC). Handles real-time, low-latency audio streaming between the browser and backend worker.\n\n• VAD (Voice Activity Detection): Silero VAD. Runs locally inside the worker to detect when the user starts and stops speaking.\n\n• STT (Speech-to-Text): Groq Whisper (whisper-large-v3-turbo). Transcribes speech into text in ~100–200ms.\n\n• LLM Reasoning & Tool Calling: Groq (openai/gpt-oss-120b). Fast reasoning and function calling.\n\n• TTS (Text-to-Speech): Cartesia (sonic-turbo). Uses the \"Priya\" voice profile—a natural, clear Indian English tone suited for a Bengaluru clinic.\n\n• State & Database: SQLite + Python. Manages appointments, availability checks, and audit trails.\n\n• Frontend: React + Vite with @livekit/components-react and Tailwind CSS.\n\nBuilding the basic happy path is easy; making real-time voice work reliably is where the real learning happened. Here are 4 specific problems I ran into:\n\nWhen I first ran voice tests, everything would work for 2 or 3 turns, and then suddenly crash with an HTTP 429 Too Many Requests (Rate Limit Exceeded).\n\nWhy it happened:\n\nOn Groq's free tier, there is an 8,000 Tokens Per Minute (TPM) limit. My initial system prompt combined with the JSON schemas for 5 function tools was taking ~1,900 tokens per single LLM call. If the user spoke 4 times in a minute,\n\nthat was 1,900 * 4 = 7,600+ tokens, immediately blowing through the 8,000 token limit.\n\nHow I fixed it:\n\nI completely compressed the system prompt and tool docstrings. I removed repetitive instructions, used dense bullet points, and kept the tool parameters minimal while preserving all clinical guardrails. This brought the request size\n\ndown from ~1,900 tokens to ~500 tokens (a 73% reduction). Suddenly, I could have 15+ turns a minute without hitting rate limits.\n\nWhile testing with my laptop mic, I noticed the agent would randomly trigger and start speaking even when I hadn't said anything.\n\nWhy it happened:\n\nDefault VAD sensitivity was picking up subtle background sounds—fan noise, keyboard typing, and breathing. Each micro-sound triggered LiveKit's turn detector, which immediately dispatched an STT call, an LLM call, and a TTS synthesis\n\ncall. This was burning API quota and interrupting the conversation.\n\nHow I fixed it:\n\nI tuned the Silero VAD parameters and turn handling in LiveKit:\n\n```\nvad_instance = silero.VAD.load(\n    min_speech_duration=0.25,   # Ignore clicks and breath sounds under 250ms\n    min_silence_duration=0.65,  # Wait for a clean pause before marking end-of-turn\n    prefix_padding_duration=0.3,\n    activation_threshold=0.6,   # Require clearer vocal energy over ambient noise\n)\n\nsession = AgentSession(\n    turn_handling={\n        \"endpointing\": {\"min_delay\": 0.6, \"max_delay\": 3.0},\n        \"preemptive_generation\": {\"enabled\": False},  # Only generate audio when speech is complete\n        \"interruption\": {\"enabled\": True, \"min_duration\": 0.5},\n    }\n)\n```\n\nDisabling preemptive generation and requiring at least 250ms of vocal energy eliminated the false triggers completely.\n\nInitially, I used Groq's built-in TTS. While it worked, it only had US and Arabic voice profiles, and the free-tier rate limits were strict for voice generation.\n\nFor a clinic located in HSR Layout, Bengaluru, a North American voice felt out of place. I integrated Cartesia's sonic-turbo model with their Priya voice (an Indian English female voice profile).\n\nThe difference was night and day:\n\n• Latency dropped below 100ms.\n\n• The cadence and pronunciation of Indian names sounded authentic.\n\nWhen I tried deploying the Python backend, I quickly realized you can't just throw a voice agent onto serverless platforms like Vercel or AWS Lambda.\n\nA text chatbot handles a quick HTTP request and terminates in 1 second. A voice agent, on the other hand, is a persistent WebRTC worker daemon. It maintains an active bidirectional audio socket to LiveKit Cloud 24/7.\n\nThe Solution:\n\n• React Frontend: Deployed on Vercel with a serverless token endpoint (/api/token) that generates short-lived LiveKit JWT access tokens without exposing LIVEKIT_API_SECRET to the browser.\n\n• Python Agent Worker: Containerized via Docker and deployed to LiveKit Cloud Agent Hosting in the ap-south (Mumbai) region for ultra-low ping.\n\nWhen deploying the Docker container, I ran into a ModuleNotFoundError: No module named 'agent.prompts'. The container entrypoint was running python agent/agent.py start, which put /app/agent into Python's sys.path instead of the root\n\n/app. I fixed this by adding ENV PYTHONPATH=\"/app\" in the Dockerfile and adding a defensive path resolution in Python.\n\nThis project gave me a massive appreciation for what it takes to build reliable real-time AI systems.\n\nI'd love to connect with other engineers and builders working in Voice AI, WebRTC, and LLM tool calling:\n\n• How are you handling ambient noise and interruption handling in your voice agents?\n\n• What TTS providers have you found best for regional accents?\n\nCheck out the code and feel free to share your thoughts or suggestions!\n\n🔗 GitHub Repo: [https://github.com/CosmosTechy/maya-ai-voice-receptionist](https://github.com/CosmosTechy/maya-ai-voice-receptionist)\n\n🌐 Live Demo: [https://maya-ai-voice-receptionist.vercel.app/](https://maya-ai-voice-receptionist.vercel.app/)", "url": "https://wpnews.pro/news/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist", "canonical_source": "https://dev.to/cosmostechy/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist-45m", "published_at": "2026-09-04 08:12:17+00:00", "updated_at": "2026-09-04 08:23:53.760810+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["Maya", "SwasthyaCare Clinic", "LiveKit", "Groq", "Cartesia", "Silero", "Whisper"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist", "markdown": "https://wpnews.pro/news/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist.md", "text": "https://wpnews.pro/news/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist.txt", "jsonld": "https://wpnews.pro/news/how-i-built-maya-a-real-time-voice-ai-clinic-receptionist.jsonld"}}