cd /news/ai-agents/why-websockets-break-telephony-voice… · home › topics › ai-agents › article
[ARTICLE · art-170202] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Why WebSockets Break Telephony Voice AI: Building a 60ms Barge-In Loop on Raw 8kHz RTP

Developer Wang Guo open-sourced a telephony voice AI lab that bypasses JSON-over-WebSocket wrappers in favor of raw 8kHz RTP through FreeSWITCH, using SOXR VHQ resampling to 16kHz for SenseVoice and VAD, and a 256-frame outbound ring buffer with a 0.16ms clearAudio() hard flush to achieve a roughly 60ms barge-in loop. The design splits the pipeline into two asymmetric loops so interruption detection relies on acoustic VAD rather than waiting 200-400ms for an LLM or full-sentence transcript.

by read1 min views3 publishedNov 16, 2026

Author: Wang Guo (@wangguode)

Source Lab: https://github.com/wangguode/ai-agent-voice/tree/main/01-voice-agent-rtp-bargein

Stack: FreeSWITCH 8kHz Raw RTP / Pipecat Full-Duplex / SenseVoice / SOXR VHQ Resampling

Most Voice AI tutorials work great over a clean laptop microphone in a quiet room using browser WebSockets. The moment you connect that same pipeline to real enterprise telephony (PSTN / SIP trunks / FreeSWITCH), three brutal physical realities hit at once:

8kHz G.711 (PCMU/ PCMA) or L16 audio, while modern acoustic models (SenseVoice, Whisper, Silero VAD) expect 16kHz PCM. Naïve linear interpolation introduces aliasing artifacts that tank ASR accuracy and cause false VAD triggers. In our open-source lab (01-voice-agent-rtp-bargein), we bypassed generic JSON-over-WebSocket wrappers and engineered a deterministic telephony loop:

[PSTN / SIP Caller (8kHz RTP, 20ms frames)]
        │
        ▼ (UDP Raw RTP)
[FreeSWITCH Media Gateway]
        │
        ├──► [SOXR VHQ Upsampler (8kHz ➔ 16kHz)] ──► [SenseVoice / Acoustic VAD (~18ms)]
        │                                                       │
        │                                                       ▼ (Barge-In Triggered!)
        └──◄ [256-Frame Outbound Ring Buffer] ◄──── [0.16ms clearAudio() Hard Flush]

SOXR_VHQ) for 8kHz <-> 16kHz conversion preserves vocal formants and prevents high-frequency aliasing from poisoning the local VAD classifier.clearAudio()`` 0.16ms—producing immediate silence on the caller's phone. A common mistake in Voice AI is waiting for the LLM or full-sentence STT transcript to confirm "the user is interrupting". Transcription requires context windows (often 200ms–400ms).

By splitting the pipeline into two asymmetric loops:

All reference code, architecture diagrams, and runnable barge-in experiments are open-sourced:

── more in #ai-agents 4 stories · sorted by recency
── more on @wang guo 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/why-websockets-break…] indexed:0 read:1min 2026-11-16 · —