# Navigating Floods Without Data: Building Sentinel Voice Agent in 10 Days > Source: > Published: 2026-08-15 12:21:54+00:00 It was during the peak monsoon season when I read a distress report from a family stranded on their rooftop. Power was flickering, rain was hammering against the walls, and cellular data was down to a crawling 2G edge. They had a phone with 14% battery, but opening an emergency app or downloading heavy government disaster PDFs was impossible. All they could do was place a direct phone call. That moment stayed with me. When panic sets in and water is rising inside your living room, you don't navigate drop-down menus or type search queries into a browser. You need to speak, and you need a voice that answers immediately with verified life-saving relief info. That became the driving mission behind **Sentinel** — an autonomous, real-time Voice AI emergency dispatcher that I built over 10 days during the **#VoiceForBharat** challenge. ## The Problem I Wanted to Solve In emergency response across India, victims and disaster managers face three immediate hurdles: - **Information Fragmentation:** Emergency guidelines, live rainfall alerts, and shelter capacities exist across different departments. A caller in panic needs instant answers (e.g., *"Is there a shelter in Guwahati with medical support and space for pets?"*). - **The Friction of Touch UIs:** Wet screens, low digital literacy, and high adrenaline make text interfaces fail. Voice is the most natural, accessible lifeline. - **Context Collapse:** When standard chatbots escalate a user to another team or system, they drop the context and force the distressed victim to repeat their story from scratch. ## How Sentinel Works Under the Hood To make Sentinel feel like a natural human dispatcher, every millisecond of latency had to be eliminated. The system streams voice bidirectionally through a unified WebRTC pipeline: - **Speech-to-Text (STT):** Deepgram Nova-3 transcribes incoming audio streams in real time with multilingual code-mixing support (English & Hindi). - **Brain & Reasoning (LLM):** Google Gemini handles real-time disaster triage, safety guardrails, and dynamic tool orchestration. - **Ultra-Fast Voice (TTS):** **Murf Falcon (Anisha voice)** delivers human-like, expressive Indian English speech synthesis with near-instant time-to-first-byte. - **Transport & Turn Detection:** LiveKit WebRTC combined with Silero Voice Activity Detection (VAD) to allow natural user interruptions. - **Memory & Persistence:** SQLite database storing caller check-in history, household details, and escalation tickets. ## Seeing Sentinel in Action Here is what the interface and voice response looks like when a caller connects: ## markdown ## System Architecture & Data Flow To ensure sub-second response times during high-stress disaster triage, Sentinel replaces traditional polling with a unified, bi-directional WebRTC pipeline: plaintext ### Core Architecture Layers: - **Real-Time Audio Transport (LiveKit):** Manages ultra-low-latency WebRTC streams between the caller's browser and the backend agent worker. Integrated with **Silero VAD**, it dynamically tracks speech boundaries to allow natural, uninterrupted voice turns. - **Speech-to-Text Layer (Deepgram Nova-3):** Provides fast speech transcription capable of handling diverse Indian English accents and rapid code-switching without breaking connection. - **Reasoning & Tool Orchestrator (Google Gemini):** Acts as Sentinel's decision engine. It evaluates incoming transcripts against disaster guardrails, decides whether to fetch external telemetry or execute SQLite lookups, and triggers the `ShelterInformationSpecialist` when domain-specific shelter queries arise. - **Low-Latency Speech Synthesis (Murf Falcon 2):** Converts raw LLM output tokens into high-fidelity, expressive Indian English speech (`Anisha` voice). Leveraging Murf's streaming API, audio chunks begin playing back immediately without waiting for the full LLM sentence to generate. - **State & Persistence Layer (SQLite):** Maintains caller profiles, past check-in coordinates, household sizes, and emergency escalation logs with built-in user consent controls. ## What Sentinel Can Do - **Stateful Memory with Strict Privacy:** Sentinel remembers returning callers. If someone calls back hours later, Sentinel greets them by name, recalls their last reported location, and checks if their safety status has changed. - Caller data is saved only after explicit verbal consent. - Built-in privacy commands let callers say *"Forget my data"* to instantly purge their records from SQLite. - **Live Weather Telemetry & Fault Tolerance:** Queries real-time rainfall, temperature, and flood alerts via Open-Meteo geocoding and forecast APIs. If third-party APIs fail or experience latency, Sentinel gracefully falls back to cached emergency advisories without breaking conversational flow. - **Human Escalation Workflows:** When a caller reports being trapped or in immediate danger, Sentinel verifies their safety, asks for confirmation, logs an emergency escalation ticket (Critical, Moderate, Low), generates a traceable reference ID, and alerts dispatch webhooks. ## Why I Built the Shelter Specialist Agent General emergency bots often struggle when deep domain details are required under time pressure. When a caller specifically requests relief shelter availability, pet accommodations, or on-site medical care, Sentinel executes a dynamic **Multi-Agent Handoff** to the **Shelter Information Specialist**. ## The Hardest Roadblock: The "Silent Handoff" The toughest challenge occurred during multi-agent delegation. Sentinel would announce *"I will connect you to our shelter information specialist"*, the badge on the UI would change, but the call would go completely silent. LiveKit treated the initial turn as finished, leaving the new agent waiting for the caller to speak again. **The Solution:** I leveraged the `on_enter()` lifecycle method in LiveKit Agents. The moment the specialist agent assumes ownership of the session, `on_enter()` immediately triggers a direct `session.say(...)` audio stream via Murf Falcon. This eliminated dead air and delivered real-time shelter data instantly upon transfer. ## Run It Yourself ### Prerequisites - Python 3.10 to 3.13 - Node.js 18+ and pnpm - LiveKit Cloud account, Murf AI API key, Google Gemini API key, Deepgram API key ### 1. Clone & Configure Environment Create `backend/.env.local` and add your keys: ### 2. Start the Backend & Frontend Open `http://localhost:3000` in your browser and start your emergency call session. ## What I Would Build Next - **SIP Telephony Integration:** Connecting Sentinel directly to toll-free disaster hotlines (NDRF/SDMA) for citizens without active smartphone data plans. - **Regional Language Code-Switching:** Expanding support for seamless bilingual Kannada, Assamese, and Bengali synthesis pipelines for hyper-local relief hubs. *Special thanks to Murf AI and the VoiceForBharat mentors for hosting this intensive 10-day challenge!*