How I Built an AI Agricultural Advisor That Talks to Indian Farmers — 10 Days of Voice Agents

An engineer built Samar, an AI agricultural advisor voice agent for Indian farmers, in 10 days as part of Murf AI's VoiceForBharat challenge. The agent speaks Hindi and English, provides weather forecasts, crop prices, and pest advice, and can make outbound calls to warn farmers of weather alerts. It uses Deepgram for speech-to-text, Google Gemini for language processing, and Murf Falcon for text-to-speech.

How I Built an AI Agricultural Advisor That Talks to Indian Farmers — 10 Days of Voice Agents The Problem and the Users Over 60% of India's workforce depends on agriculture, yet accessing timely agricultural advice remains a massive challenge for rural farmers. Language barriers, low digital literacy, and poor internet connectivity mean that text-based apps simply don't work for most of them. What if farmers could just pick up the phone and talk to an AI expert — in Hindi, in their own language, like talking to a friend? That's exactly what I built over the last 10 days as part of the 10 Days of Voice Agents — VoiceForBharat Edition challenge by Murf AI. Meet Samar — an AI Agricultural Advisor for Farm & Field, a voice-first helpline that provides real-time weather forecasts, mandi crop prices, pest diagnosis, and farming advice to Indian farmers through natural conversation. What the Voice Agent Does Samar is a fully functional voice AI agent that: 🗣️ Speaks naturally in Hindi Devanagari script and English using Murf Falcon TTS 🌦️ Fetches live weather data for any Indian city using the OpenWeatherMap API 💰 Looks up real-time crop prices from government mandi data 🧠 Remembers returning farmers — their name, location, and crops across sessions 📞 Makes outbound phone calls via SIP to proactively warn farmers about weather alerts 🎫 Escalates to human experts when it encounters problems it can't solve, creating trackable support tickets 📊 Tracks call analytics — logging every call's success or failure in real-time on a live dashboard 🤝 Hands off to a specialist agent — if a farmer asks about crop diseases, Samar transfers the call to "Pooja", a crop disease expert with a completely different voice How the System Works The architecture follows a standard voice AI pipeline: Farmer's Voice → Deepgram STT → Google Gemini LLM → Murf Falcon TTS → Farmer Hears Response Core Components: Component Technology Role Speech-to-Text STT Deepgram Nova-3 Converts farmer's speech to text Large Language Model LLM Google Gemini via LiveKit inference Processes input, decides actions, generates responses Text-to-Speech TTS Murf Falcon Converts text responses to natural Indian voice Real-time Transport LiveKit Agents SDK Manages audio streaming, WebRTC, SIP Voice Activity Detection Silero VAD + LiveKit Turn Detector Detects when the farmer is speaking Database SQLite Stores farmer profiles, tickets, call logs Frontend Next.js + Tailwind CSS Browser UI with dashboards Backend: Python with livekit-agents SDK — everything lives in a single agent.py file. Frontend: Next.js with LiveKit's Agents UI components, plus custom dashboards for escalation tickets and call analytics. The Most Important Features python tts=murf.TTS voice="Samar", style="Conversation", tokenizer=tokenize.basic.SentenceTokenizer min sentence len=2 , text pacing=True, LANGUAGE & SCRIPT CRITICAL : Always write every language in its own native script. Hindi → Devanagari नमस्ते , never romanized never "namaste" . Persistent Memory with SQLite Day 4 Samar remembers every farmer she talks to. When a farmer calls back, she greets them by name, knows their location, and remembers their crops. This is powered by a simple SQLite database with lookup farmer and save farmer tools. Live Tool Integration Day 5 Samar doesn't guess — she fetches real data: Weather: Calls the OpenWeatherMap API to get temperature, humidity, rain chance, and wind speed for any Indian location. Crop Prices: Queries government mandi market data to give farmers the latest prices for their crops. Outbound SIP Phone Calls Day 6 This was the most exciting feature. Samar can proactively call farmers on their real phone numbers via SIP trunking to warn them about incoming storms or pest outbreaks. The agent waits for the farmer to answer before speaking, and always offers an opt-out option. Human Escalation Dashboard Day 7 When Samar encounters a problem she can't solve e.g., the farmer explicitly requests a human , she creates a support ticket with a unique reference ID. These tickets appear in a real-time dashboard overlay on the website, accessible via a notification bell icon. Call Analytics Dashboard Day 8 Every call is tracked. If Samar successfully provides weather or crop price data, she silently logs the call as "successful". If the farmer hangs up before getting help, it's logged as "failed". A live Next.js dashboard at /analytics shows Total Calls, Success Rate, Failure Types, Channel Mix, and a Recent Calls table — all updating in real-time. Multi-Agent Handoff Day 9 Samar knows her limits. If a farmer asks about crop diseases or fungus, she doesn't try to answer — she transfers the call to Pooja, a specialized Crop Problem Expert. Pooja has her own voice, her own prompt, and receives the full conversation history so the farmer never has to repeat themselves. python @function tool async def transfer to crop specialist self, context: RunContext - tuple Agent, str : specialist = CropSpecialist chat ctx=self.chat ctx.copy exclude instructions=True return specialist, "Transferring you to our crop specialist." Challenges and How I Overcame Them Challenge 1: Hindi Romanization Problem: Despite explicit instructions, the LLM would sometimes output "Dhanyavaad" instead of "धन्यवाद". Solution: I added the language rule in multiple places in the system prompt and marked it as CRITICAL . Repeating the instruction at the top AND bottom of the prompt significantly improved compliance. Challenge 2: SIP Outbound Call Timing Problem: When making outbound phone calls, the agent would start speaking before the farmer picked up the phone, wasting the greeting. Solution: I added logic to wait for the farmer's audio track to be published before generating the greeting. The agent polls for the participant's audio track and only speaks once detected: python if is outbound: Wait for user to answer before speaking while not any t.kind == rtc.TrackKind.KIND AUDIO for t in participant.track publications.values : await asyncio.sleep 0.5 Challenge 3: Native SQLite Build Errors on Windows Problem: The better-sqlite3 npm package requires native C++ build tools, which caused compilation errors on Windows. Solution: I switched to the sqlite + sqlite3 JavaScript packages which don't require native compilation, and the backend Python agent handles all heavy database operations directly. How to Build and Run It Yourself Prerequisites Python 3.10+ Node.js 18+ uv Python package manager pnpm Node package manager API keys for: LiveKit, Murf AI, Deepgram, Google AI Step 1: Clone the Repository bash git clone https://github.com/SatishBoya07/murf-livekit-starter.git https://github.com/SatishBoya07/murf-livekit-starter.git cd murf-livekit-starter Step 2: Set Up the Backend bash cd backend cp .env.example .env.local uv sync uv run python src/agent.py download-files first time only uv run python src/agent.py dev Step 3: Set Up the Frontend bash cd frontend cp .env.example .env.local pnpm install pnpm dev Step 4: Open and Talk Open http://localhost:3000 http://localhost:3000 in your browser, click "Talk to Samar", and start a conversation ⚠️ Important: Never commit your .env.local files or expose API keys publicly. The .gitignore file is already configured to exclude them. What I Would Improve Next Regional Language Support: Add support for Tamil, Telugu, Kannada, and other Indian languages beyond Hindi and English. Voice Cloning: Train custom voices that sound even more natural and region-specific. WhatsApp Integration: Many Indian farmers use WhatsApp — integrating voice messages through WhatsApp would massively increase accessibility. Offline Mode: Build a lightweight offline model for areas with poor connectivity. Production Deployment: Deploy on cloud infrastructure with proper scaling, monitoring, and security. Links 🔗 GitHub Repository: github.com/SatishBoya07/murf-livekit-starter 🎙️ Murf Falcon TTS: murf.ai 🔊 LiveKit Agents: docs.livekit.io/agents This project was built as part of the 10 Days of Voice Agents — VoiceForBharat Edition challenge by Murf AI. The fastest TTS API — Murf Falcon — made it possible to deliver natural, low-latency Indian voices that feel real to the farmers who need them most.