# Why WebSockets Break Telephony Voice AI: Building a 60ms Barge-In Loop on Raw 8kHz RTP

> Source: <https://dev.to/wangguode/why-websockets-break-telephony-voice-ai-building-a-60ms-barge-in-loop-on-raw-8khz-rtp-27o4>
> Published: 2026-11-16 01:47:17+00:00

**Author**: Wang Guo ([@wangguode](https://dev.to/wangguode))

**Source Lab**: [https://github.com/wangguode/ai-agent-voice/tree/main/01-voice-agent-rtp-bargein](https://github.com/wangguode/ai-agent-voice/tree/main/01-voice-agent-rtp-bargein)

**Stack**: FreeSWITCH 8kHz Raw RTP / Pipecat Full-Duplex / SenseVoice / SOXR VHQ Resampling

Most Voice AI tutorials work great over a clean laptop microphone in a quiet room using browser WebSockets. The moment you connect that same pipeline to real enterprise telephony (PSTN / SIP trunks / FreeSWITCH), three brutal physical realities hit at once:

`8kHz` G.711 (`PCMU`/` PCMA`) or `L16` audio, while modern acoustic models (SenseVoice, Whisper, Silero VAD) expect `16kHz` PCM. Naïve linear interpolation introduces aliasing artifacts that tank ASR accuracy and cause false VAD triggers.
In our open-source lab ([`01-voice-agent-rtp-bargein`](https://github.com/wangguode/ai-agent-voice/tree/main/01-voice-agent-rtp-bargein)), we bypassed generic JSON-over-WebSocket wrappers and engineered a deterministic telephony loop:

```
[PSTN / SIP Caller (8kHz RTP, 20ms frames)]
        │
        ▼ (UDP Raw RTP)
[FreeSWITCH Media Gateway]
        │
        ├──► [SOXR VHQ Upsampler (8kHz ➔ 16kHz)] ──► [SenseVoice / Acoustic VAD (~18ms)]
        │                                                       │
        │                                                       ▼ (Barge-In Triggered!)
        └──◄ [256-Frame Outbound Ring Buffer] ◄──── [0.16ms clearAudio() Hard Flush]
```

`SOXR_VHQ`) for `8kHz <-> 16kHz` conversion preserves vocal formants and prevents high-frequency aliasing from poisoning the local VAD classifier.`clearAudio()`` 0.16ms`—producing immediate silence on the caller's phone.
A common mistake in Voice AI is waiting for the LLM or full-sentence STT transcript to confirm *"the user is interrupting"*. Transcription requires context windows (often 200ms–400ms).

By splitting the pipeline into **two asymmetric loops**:

All reference code, architecture diagrams, and runnable barge-in experiments are open-sourced:
