{"slug": "hackhazards-26", "title": "HACKHAZARDS'26", "summary": "At HackHazards'26, the team Git Push Pray built Dial2AI, a multilingual voice-based AI assistant that makes generative AI accessible through a regular phone call without requiring internet, a smartphone, or an app. The system uses Exotel for telephony, Grok for speech recognition and AI reasoning, and supports English, Hindi, and Hinglish with features like conversational memory and live data fetching.", "body_md": "Dial2AI: Making Generative AI Accessible Through a Phone Call\n\nNo Internet. No Smartphone. No App. Just a Phone Call.\n\nArtificial Intelligence is transforming education, healthcare, agriculture, finance, and access to public information. However, most AI tools require a smartphone, a stable internet connection, and enough digital literacy to use websites or applications.\n\nFor HackHazards’26, our team, Git Push Pray, built Dial2AI, a multilingual voice-based AI assistant that makes Generative AI accessible through a regular phone call.\n\nA user can call an Exotel-powered number from any mobile phone, including a basic keypad phone, ask a question naturally, and receive an AI-generated voice response in English, Hindi, or Hinglish.\n\nThe Problem\n\nMillions of people still have limited internet access, low digital literacy, or no modern smartphone. This especially affects farmers, workers, senior citizens, students, and people living in rural or underserved areas.\n\nImportant information about weather, government schemes, agriculture, jobs, education, healthcare, and public services may be available online, but not everyone can easily access it.\n\nTraditional IVR systems are also limited. They depend on fixed options such as “Press 1 for weather” or “Press 2 for market prices.” These systems cannot understand natural or open-ended questions.\n\nWe wanted to replace complicated applications and rigid IVR menus with something almost everyone already understands: a phone call.\n\nOur Solution\n\nDial2AI transforms an ordinary phone call into a natural conversation with an AI assistant.\n\nThe user calls the Dial2AI number and asks a question. Exotel sends the caller’s audio to our FastAPI backend through a WebSocket connection. The speech is converted into text, and the system detects the user’s language and intent.\n\nWhen the question requires current information, Dial2AI fetches data from external APIs. The AI then generates an appropriate response, which is converted back into speech and played to the caller during the same phone call.\n\nThe user does not need an application, a browser, mobile data, or typing skills.\n\nKey Features\n\nDial2AI supports conversations in English, Hindi, and Hinglish. It responds in the same language used by the caller.\n\nThe assistant also maintains conversational memory, allowing users to ask follow-up questions without repeating the complete context.\n\nIt can fetch live weather, news, agricultural, and civic information through external APIs instead of relying only on the AI model’s existing knowledge.\n\nDial2AI also supports barge-in interruption. If the user starts speaking while the assistant is answering, the system can stop the response and listen to the new question.\n\nSmart silence detection identifies when the user has finished speaking and starts processing the request without waiting for a long fixed timeout.\n\nAn amplitude-based noise gate filters background noise and telecom static so that low-level disturbances are not treated as speech.\n\nWhile the AI prepares an answer, Dial2AI plays hold music to prevent the user from thinking that the call has disconnected.\n\nThe system can also generate conversation transcripts, detect intent and sentiment, display call analytics on a dashboard, and prepare SMS summaries for the caller.\n\nTechnology Stack\n\nDial2AI uses Exotel and Passthru Applets for telephony and audio streaming.\n\nThe backend is built using Python, FastAPI, Uvicorn, WebSockets, HTTPX, and Pydantic.\n\nGrok STT is used for speech recognition, while Grok 4.1 Fast is used for AI reasoning and response generation.\n\nGoogle Text-to-Speech and gTTS are used to convert generated responses into audio. FFmpeg and Python audio-processing utilities are used to manage audio formats.\n\nSQLite is used for structured call records, while Neo4j AuraDB is used for graph-based memory and relationships.\n\nThe dashboard is built using Next.js, React, Tailwind CSS, Recharts, and Lucide React. Base44 is used to support and accelerate the application-management and dashboard layer.\n\nPartner Tracks\n\nBase44 helped us build and manage the dashboard surrounding Dial2AI. The dashboard allows us to view call records, transcripts, detected intent, sentiment, user interaction patterns, and backend configurations.\n\nThis allowed us to spend more time improving the real-time voice pipeline while still creating a usable operational interface.\n\nWe also used Neo4j AuraDB to explore graph-based conversational memory and personalisation.\n\nNeo4j can connect callers with their locations, interests, previously discussed topics, and earlier answers. These relationships can help Dial2AI understand returning callers and provide more contextual responses during future conversations.\n\nImplementation Challenges\n\nOne of the biggest challenges was real-time audio streaming. A phone call produces continuous audio rather than a single API request. We built a WebSocket-based pipeline that receives, processes, and returns audio without blocking the active call.\n\nAnother challenge was latency. Speech recognition, external API requests, AI generation, and text-to-speech conversion take time. Silence during this delay can make users think that the call has ended. We solved this by playing hold music asynchronously until the final response was ready.\n\nTelecom noise was another major issue. Static and background sounds sometimes caused the speech model to detect words that were never spoken. We implemented a custom amplitude gate to ignore audio below a selected threshold.\n\nSilence detection also required careful adjustment. Processing too early could cut off the caller, while waiting too long made the conversation feel slow. Our system begins processing only after meaningful speech is followed by a sufficient silent period.\n\nBarge-in interruption created an echo problem because the assistant’s own voice could return through the phone channel and be recognised as user speech. We used playback timestamps and echo-guard windows to separate genuine interruptions from audio feedback.\n\nUnderstanding Hinglish was also challenging because real conversations often mix Hindi, English, accents, Urdu-origin words, and imperfect transcriptions. We designed prompts that focus on the overall meaning of the sentence rather than depending entirely on exact word recognition.\n\nFuture Improvements\n\nIn future versions, we plan to add a missed-call callback system so that users can access Dial2AI without paying for the outgoing call.\n\nWe also plan to add location-based weather and Mandi information, two-way SMS conversations, more Indian regional languages, and integrations with Agmarknet, IMD, railway, healthcare, and government datasets.\n\nNeo4j-based personalisation can also be expanded to provide more relevant responses to returning callers.\n\nImpact\n\nDial2AI demonstrates that advanced AI does not need to remain limited to modern smartphones and applications.\n\nA basic phone can become a weather assistant for a farmer, a learning assistant for a student, a government-scheme discovery tool, or a support platform for senior citizens and rural communities.\n\nOur goal is simple:\n\nIf someone can make a phone call, they should be able to access AI.\n\nTeam Git Push Pray\n\nDial2AI was developed for HackHazards’26 by Rudrakshi Agarwal and Prabhav Agrawal.\n\nConclusion\n\nDial2AI brings Generative AI to one of the simplest and most familiar communication interfaces: the telephone.\n\nNo internet.\n\nNo smartphone.\n\nNo application.\n\nJust dial, speak, and receive an answer.\n\nDial2AI: Making Artificial Intelligence as accessible as a phone call.", "url": "https://wpnews.pro/news/hackhazards-26", "canonical_source": "https://dev.to/rudrakshi_agarwal/hackhazards26-dh0", "published_at": "2026-07-12 18:12:33+00:00", "updated_at": "2026-07-12 18:45:19.622435+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "natural-language-processing", "ai-products", "developer-tools"], "entities": ["Git Push Pray", "Dial2AI", "Exotel", "Grok", "Google Text-to-Speech", "Neo4j AuraDB", "HackHazards'26", "Base44"], "alternates": {"html": "https://wpnews.pro/news/hackhazards-26", "markdown": "https://wpnews.pro/news/hackhazards-26.md", "text": "https://wpnews.pro/news/hackhazards-26.txt", "jsonld": "https://wpnews.pro/news/hackhazards-26.jsonld"}}