{"slug": "how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers", "title": "How I Built Kisan Vaani: A Multi-Agent Voice AI Assistant for Indian Farmers", "summary": "A developer built Kisan Vaani, a multi-agent voice AI assistant for Indian farmers, during Murf AI's 10 Days of AI Voice Agents challenge. The system uses LiveKit for WebRTC, Deepgram for speech-to-text, Murf Falcon 2 for Hindi TTS, and Groq's Llama 3.1 for the LLM, enabling farmers to get market prices, weather updates, and crop doctor advice via natural voice calls. It features caller profile memory, human escalation, and a multi-agent specialist handoff.", "body_md": "Over the last 10 days, I participated in the **10 Days of AI Voice Agents (Voice for Bharat Edition)** challenge by Murf AI. My goal was to build a real-world voice application that solves an actual problem in India.\n\nI chose the **Farm & Field Track and built Kisan Vaani (किसान वाणी)** — an AI-powered voice assistant designed to help Indian farmers get instant market prices, weather updates, and agricultural doctor advice through natural Hindi voice calls.\n\nHere is a breakdown of what I built, the architecture behind it, the hardest bugs I ran into, and how you can run the project yourself.\n\n**The Problem I Set Out to Solve**\n\nIn rural India, most agricultural information is locked behind text-heavy websites or mobile apps. For many farmers, navigating complex menus or reading long texts on smartphones is a major barrier.\n\nWhen a farmer needs to know today's mandi price for wheat in Noida, or needs urgent medicine advice for yellow rust disease on their crop, a 30-second voice call in native Devanagari Hindi is much more natural and effective than searching through an app.\n\nI designed Kisan Vaani to act like a personal phone call helper:\n\n**System Architecture**\n\nHere is how real-time audio and data flow through the application:\n\n``` php\nflowchart TD\n    Browser[\"Next.js Web Frontend & Audio Visualizer\"] <-->|WebRTC Stream| LiveKit[\"LiveKit WebRTC Cloud\"]\n    LiveKit <--> STT[\"Deepgram Nova-3 Speech-to-Text\"]\n    LiveKit <--> TTS1[\"Murf Falcon 2 TTS ('Anisha' Main Voice)\"]\n    LiveKit <--> TTS2[\"Murf Falcon 2 TTS ('Samar' Doctor Voice)\"]\n\n    subgraph Agent Engine [\"Python Agent Backend\"]\n        MainAgent[\"KisanVaaniMainAgent\"]\n        SpecAgent[\"CropDoctorSpecialistAgent\"]\n        LLM[\"Groq Llama 3.1 LLM\"]\n    end\n\n    STT --> MainAgent\n    MainAgent <--> LLM\n    MainAgent -->|Agent Transfer| SpecAgent\n    MainAgent --> TTS1\n    SpecAgent --> TTS2\n\n    subgraph Storage [\"SQLite Database\"]\n        DB1[(\"farmers table\")]\n        DB2[(\"escalations table\")]\n        DB3[(\"call_logs table\")]\n    end\n\n    MainAgent <--> Storage\n```\n\n**The Tech Stack**\n\n**Key Features Built Over the 10 Days**\n\n**1. Natural Hindi Voice with Ultra-Low Latency**\n\nUsing Murf Falcon 2 (Anisha voice), the agent responds almost instantly with a time-to-first-byte under 100 ms. The voice sounds warm and natural, making the conversation feel like talking to a real helper on the phone.\n\n**2. Caller Profile Memory (SQLite)**\n\nInstead of asking for information repeatedly, Kisan Vaani remembers the farmer across calls. When Ramesh from Noida calls back, the agent loads his saved profile from the farmer's SQLite table and tailors its responses to his location and crop.\n\n**3. Live Mandi Prices & Weather Alerts**\n\nI integrated real-time tools to fetch current market prices per quintal for crops like wheat, paddy, and mustard (via e-NAM price data), as well as district weather forecasts and rain probabilities (via the Open-Meteo API).\n\n**4. Human Escalation & Call Analytics Dashboard**\n\nIf a farmer faces an emergency or financial dispute, Kisan Vaani creates an escalation ticket in SQLite (escalations table) for a Krishi Vigyan Kendra (KVK) officer callback. All call outcomes, duration metrics, and success rates are rendered on a glassmorphism dashboard built with Next.js.\n\n**5. Multi-Agent Specialist Handoff**\n\nWhen a farmer asks about crop diseases like yellow rust, the main agent (Anisha) announces a transfer: \"I am connecting you with our senior crop doctor specialist.\" The session updates to CropDoctorSpecialistAgent, and the TTS voice dynamically switches to Murf Falcon Samar (a male doctor voice), who provides precise pesticide treatment remedies.\n\n**Real Engineering Challenges & How I Solved Them**\n\nBuilding real-time voice agents comes with unique edge cases. Here are the three main technical hurdles I worked through:\n\n**1. Stopping the LLM from Speaking Raw Tool Tags**\n\nEarly in testing, Llama 3.1 occasionally outputted text tags like {\"name_or_id\": \"रमेश\"} directly into the chat box.\n\n**The Cause:** Mentioning literal python function names inside the prompt text tricked the LLM into thinking it should output them as text instead of executing native background tools.\n\n**The Fix:** I removed all literal function names from the prompt text and added strict rules ordering the LLM to run background tools silently without outputting code tags.\n\n**2. Overcoming Groq 429 Rate Limits During Agent Transfers**\n\nDuring multi-turn calls, Groq returned 429 Token-Per-Minute rate limit errors.\n\n**The Cause:** The accumulated conversation history sent to Groq grew too large on long calls.\n\n**The Fix:** When transferring the caller to the specialist doctor, I cleared the old chat context (session.chat_ctx.messages.clear()). This dropped the request payload back down to ~100 tokens, completely eliminating 429 rate limit errors.\n\n**3. Fixing LiveKit Read-Only Property Error**\n\nAttempting session.tts = specialist_tts inside the handoff tool threw an AttributeError because session.tts is read-only in LiveKit Agents 1.4.5.\n\n**The Fix:** I passed the TTS instance directly into the agent class constructor:\n\nclass CropDoctorSpecialistAgent(Agent):\n\ndef **init**(self, specialist_tts, specialist_tools: list, farmer_name: str, district: str, crop_issue: str):\n\nsuper().**init**(\n\ninstructions=get_crop_doctor_prompt(farmer_name, district, crop_issue),\n\ntts=specialist_tts,\n\ntools=specialist_tools,\n\n)\n\nNow when session.update_agent(doctor_specialist) runs, LiveKit automatically updates both the agent logic and the Murf Falcon voice seamlessly.\n\n**How to Set Up and Run Kisan Vaani**\n\nIf you want to run this project on your local machine, follow these steps:\n\n**1. Clone the Code**\n\ngit clone [https://github.com/Aaddii17/murf-livekit-starter.git](https://github.com/Aaddii17/murf-livekit-starter.git)\n\ncd murf-livekit-starter\n\n**2. Install Dependencies**\n\ncd backend\n\nuv venv\n\nuv sync\n\ncd ../frontend\n\nnpm install\n\n**3. Configure API Keys**\n\nCreate a .env.local file in both backend and frontend folders with your credentials:\n\nLIVEKIT_URL=wss://your-project.livekit.cloud\n\nLIVEKIT_API_KEY=your_livekit_key\n\nLIVEKIT_API_SECRET=your_livekit_secret\n\nMURF_API_KEY=your_murf_key\n\nDEEPGRAM_API_KEY=your_deepgram_key\n\nGROQ_API_KEY=your_groq_key\n\n**4. Launch the Dev Servers**\n\ncd backend\n\nuv run python src/agent.py dev\n\ncd frontend\n\nnpm run dev -- -p 3000\n\nOpen [http://localhost:3000](http://localhost:3000) in your browser to start talking with Kisan Vaani\n\n**Conclusion & Code Links**\n\nBuilding Kisan Vaani showed me how powerful combining WebRTC, fast TTS, and multi-agent LLM systems can be for solving accessibility problems in India.\n\n**GitHub Repository:** [https://github.com/Aaddii17/murf-livekit-starter](https://github.com/Aaddii17/murf-livekit-starter)\n\n**LiveKit Agents Framework:** LiveKit Voice AI Docs\n\n**Murf Falcon Documentation:** Murf Falcon 2 Docs\n\nThanks to Murf AI for hosting the #VoiceForBharat challenge!", "url": "https://wpnews.pro/news/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers", "canonical_source": "https://dev.to/_adi_17/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers-550c", "published_at": "2026-08-15 05:05:19+00:00", "updated_at": "2026-08-15 05:41:05.227780+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "natural-language-processing", "ai-products"], "entities": ["Kisan Vaani", "Murf AI", "LiveKit", "Deepgram", "Groq", "Llama 3.1", "Open-Meteo", "e-NAM"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers", "markdown": "https://wpnews.pro/news/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers.md", "text": "https://wpnews.pro/news/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers.txt", "jsonld": "https://wpnews.pro/news/how-i-built-kisan-vaani-a-multi-agent-voice-ai-assistant-for-indian-farmers.jsonld"}}