# Lessons from Running AI Voice Calls at Scale

> Source: <https://dev.to/deependra_rajput_360d4024/lessons-from-running-ai-voice-calls-at-scale-16o2>
> Published: 2026-11-20 12:10:29+00:00

**Recommendation:** For voice agents, externalize call state to Redis, enforce per-campaign concurrency limits before dialing, and dedupe telephony webhooks with idempotency keys so retries don't double-bill or corrupt conversation state.
More in the playbook "Running AI in production": <https://wpnews.pro/playbooks/production-ops>

A chatbot request takes a few seconds. A phone call takes minutes, and during those minutes it holds a telephony connection, a speech-to-text stream, LLM calls and text-to-speech output, all of which have their own limits and failure modes.

Here are the lessons that matter most once an AI voice system goes from a handful of test calls to thousands a day.

Web backends are built around short requests. Voice calls are long-lived sessions with state: who's on the line, what's been said, which step of the conversation you're in. Keep that state outside the process handling the audio (Redis works well), so a worker restart or a reconnect doesn't lose the conversation.

Your telephony provider, your speech services and your LLM provider all cap how much you can run at once. For outbound campaigns, enforce a per-campaign limit before you dial:

``` php
async def acquire_call_slot(redis, campaign_id: str, limit: int) -> bool:
    key = f"active_calls:{campaign_id}"
    active = await redis.incr(key)
    if active > limit:
        await redis.decr(key)
        return False
    return True

async def release_call_slot(redis, campaign_id: str) -> None:
    await redis.decr(f"active_calls:{campaign_id}")
```

Release the slot when the call ends, whatever the reason. Then put calls that can't get a slot back in the queue instead of dropping them.

Telephony providers retry webhooks when they don't get a fast response, so the same "call answered" or "call ended" event can arrive twice. Processing it twice means double billing, duplicate records or a confused conversation state.

``` php
async def handle_call_event(event, redis) -> None:
    first_time = await redis.set(f"event:{event.id}", 1, nx=True, ex=86_400)
    if not first_time:
        return  # provider retried an event we already processed
    await process(event)
```

Respond to the webhook quickly, then do the slow work in the background.

On a call, silence feels broken after about a second. Each turn adds up: speech-to-text finishing, the LLM's first token, and text-to-speech starting to play. What helps most:

Voicemail, no answer, the caller hanging up mid-sentence, background noise, someone saying "stop calling me". Each needs a defined behaviour and a logged outcome. The happy path is maybe half the work.

For every call, record how it ended, how long it lasted and where in the flow it stopped. Those outcomes, more than error logs, show where conversations go wrong and what to fix next.

Voice AI is mostly distributed-systems engineering with a language model in the middle. Get state, concurrency, idempotency and latency right, and the AI part gets much easier to improve.

Have you built anything with voice AI? I'd love to compare notes in the comments.
