{"slug": "how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls", "title": "How We Got Voice AI Response Times Under 1.8 Seconds on Real Phone Calls", "summary": "Minions.AI, a voice-based service dispatch platform for trade contractors, reduced voice AI response times on real phone calls from ~2,900ms to an average of 1,450ms (sub-1.8s in 99% of live calls). Developer Parvej Shah replaced the sequential pipeline with overlapping event-driven streams, using neural voice activity detection on 20ms frames, streaming token-based TTS, and edge WebSocket termination to cut latency.", "body_md": "*By Parvej Shah — Full-Stack Web Developer & AI Systems Engineer based in Dhaka, Bangladesh.*\n\nThere's a specific type of frustration that's hard to explain unless you've experienced it. You call a business. An automated voice picks up. You ask your question. And then — silence. Not a brief pause. A real silence. Long enough that you start wondering if the call dropped, long enough that you pull the phone away from your ear to check the signal bars.\n\nThat silence is what we were trying to eliminate when building the telephony dispatcher for **Minions.AI**, a voice-based service dispatch platform for trade contractors.\n\nIn human conversation, the natural gap between one person finishing a sentence and the other beginning a response is around 200 to 300 milliseconds. Anything beyond 600ms starts to feel awkward. At 2,500ms — which was where the original prototype sat — callers would repeat themselves, raise their voice, or hang up. The call experience was technically functional and practically unusable.\n\nThe first design was a completely natural one: record audio, run transcription, generate a response, synthesize speech, play it back. Each stage waited for the previous one to finish. The latency budget looked like this:\n\n| Stage | Time |\n|---|---|\n| Voice Activity Detection (end-of-turn) | 800ms |\n| Speech-to-Text transcription | 400ms |\n| LLM generation (full response) | 1,200ms |\n| Text-to-Speech synthesis | 500ms |\nTotal |\n~2,900ms |\n\nThat math is catastrophic for a phone call. And it gets worse in real conditions: cellular networks introduce jitter, LLM response times have variance, TTS output buffering adds overhead.\n\nThe solution wasn't to make each stage faster in isolation. It was to stop treating them as stages at all.\n\nThe rewrite changed the mental model from a sequential pipeline to an overlapping set of event-driven streams. Nothing waits for anything it doesn't strictly have to.\n\nThe original design waited for 800ms of audio silence before assuming the caller had finished speaking. We replaced this with a WebRTC-compatible neural Voice Activity Detection model running on 20ms audio frames. It detects speech completion at the prosodic level — reading the natural falling intonation of a completed sentence — rather than just measuring decibels.\n\nInstead of waiting for the full LLM completion before initiating TTS, we stream tokens into a buffer that fires TTS synthesis the moment a punctuation mark (period, comma, question mark) is encountered. The user hears the first sentence while the LLM is still reasoning about the second.\n\nAudio packets are processed over raw WebSockets terminating at edge nodes closest to the telecom provider, stripping out over 120ms of round-trip latency.\n\nThe resulting latency dropped from **~2,900ms to an average of 1,450ms (sub-1.8s in 99% of live calls)**.\n\n*Written by **Parvej Shah**, Full-Stack Web Developer & AI Systems Engineer based in Dhaka, Bangladesh. Explore more case studies and engineering insights at parvejshah.com.*", "url": "https://wpnews.pro/news/how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls", "canonical_source": "https://dev.to/parvejshah/how-we-got-voice-ai-response-times-under-18-seconds-on-real-phone-calls-2eei", "published_at": "2026-08-26 10:37:59+00:00", "updated_at": "2026-08-26 11:15:35.192493+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "ai-infrastructure", "ai-products"], "entities": ["Minions.AI", "Parvej Shah", "WebRTC"], "alternates": {"html": "https://wpnews.pro/news/how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls", "markdown": "https://wpnews.pro/news/how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls.md", "text": "https://wpnews.pro/news/how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls.txt", "jsonld": "https://wpnews.pro/news/how-we-got-voice-ai-response-times-under-1-8-seconds-on-real-phone-calls.jsonld"}}