{"slug": "lessons-from-running-ai-voice-calls-at-scale", "title": "Lessons from Running AI Voice Calls at Scale", "summary": "A developer shared lessons from scaling an AI voice-call system from test calls to thousands per day, arguing that voice AI is mostly distributed-systems engineering with a language model in the middle. The account covers keeping conversation state in Redis outside the audio worker, enforcing per-campaign concurrency limits before dialing, deduplicating retried telephony webhooks with idempotency keys, and tuning per-turn latency so silence doesn't exceed about a second. The developer also stressed logging how each call ended and where it dropped out of the flow, since edge cases like voicemail and mid-sentence hang-ups are roughly half the work.", "body_md": "A chatbot request takes a few seconds. A phone call takes minutes, and during those minutes it holds a telephony connection, a speech-to-text stream, LLM calls and text-to-speech output, all of which have their own limits and failure modes.\n\nHere are the lessons that matter most once an AI voice system goes from a handful of test calls to thousands a day.\n\nWeb backends are built around short requests. Voice calls are long-lived sessions with state: who's on the line, what's been said, which step of the conversation you're in. Keep that state outside the process handling the audio (Redis works well), so a worker restart or a reconnect doesn't lose the conversation.\n\nYour telephony provider, your speech services and your LLM provider all cap how much you can run at once. For outbound campaigns, enforce a per-campaign limit before you dial:\n\n``` php\nasync def acquire_call_slot(redis, campaign_id: str, limit: int) -> bool:\n    key = f\"active_calls:{campaign_id}\"\n    active = await redis.incr(key)\n    if active > limit:\n        await redis.decr(key)\n        return False\n    return True\n\nasync def release_call_slot(redis, campaign_id: str) -> None:\n    await redis.decr(f\"active_calls:{campaign_id}\")\n```\n\nRelease the slot when the call ends, whatever the reason. Then put calls that can't get a slot back in the queue instead of dropping them.\n\nTelephony providers retry webhooks when they don't get a fast response, so the same \"call answered\" or \"call ended\" event can arrive twice. Processing it twice means double billing, duplicate records or a confused conversation state.\n\n``` php\nasync def handle_call_event(event, redis) -> None:\n    first_time = await redis.set(f\"event:{event.id}\", 1, nx=True, ex=86_400)\n    if not first_time:\n        return  # provider retried an event we already processed\n    await process(event)\n```\n\nRespond to the webhook quickly, then do the slow work in the background.\n\nOn a call, silence feels broken after about a second. Each turn adds up: speech-to-text finishing, the LLM's first token, and text-to-speech starting to play. What helps most:\n\nVoicemail, no answer, the caller hanging up mid-sentence, background noise, someone saying \"stop calling me\". Each needs a defined behaviour and a logged outcome. The happy path is maybe half the work.\n\nFor every call, record how it ended, how long it lasted and where in the flow it stopped. Those outcomes, more than error logs, show where conversations go wrong and what to fix next.\n\nVoice AI is mostly distributed-systems engineering with a language model in the middle. Get state, concurrency, idempotency and latency right, and the AI part gets much easier to improve.\n\nHave you built anything with voice AI? I'd love to compare notes in the comments.", "url": "https://wpnews.pro/news/lessons-from-running-ai-voice-calls-at-scale", "canonical_source": "https://dev.to/deependra_rajput_360d4024/lessons-from-running-ai-voice-calls-at-scale-16o2", "published_at": "2026-11-20 12:10:29+00:00", "updated_at": "2026-11-20 12:12:26.850471+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "large-language-models", "ai-tools"], "entities": ["Redis"], "also_reported_by": [], "takeaway": "Treat voice agents as long-lived stateful sessions: store call state in Redis outside the audio worker, gate outbound dialing with a per-campaign active-call counter, dedupe retried webhooks with a Redis NX key, and keep per-turn latency under about a second.", "recommendation": {"advice": "For voice agents, externalize call state to Redis, enforce per-campaign concurrency limits before dialing, and dedupe telephony webhooks with idempotency keys so retries don't double-bill or corrupt conversation state.", "playbook": "production-ops", "playbook_url": "https://wpnews.pro/playbooks/production-ops"}, "alternates": {"html": "https://wpnews.pro/news/lessons-from-running-ai-voice-calls-at-scale", "markdown": "https://wpnews.pro/news/lessons-from-running-ai-voice-calls-at-scale.md", "text": "https://wpnews.pro/news/lessons-from-running-ai-voice-calls-at-scale.txt", "jsonld": "https://wpnews.pro/news/lessons-from-running-ai-voice-calls-at-scale.jsonld"}}