Author: Wang Guo (@wangguode)
Source Lab: https://github.com/wangguode/ai-agent-voice/tree/main/01-voice-agent-rtp-bargein
Stack: FreeSWITCH 8kHz Raw RTP / Pipecat Full-Duplex / SenseVoice / SOXR VHQ Resampling
Most Voice AI tutorials work great over a clean laptop microphone in a quiet room using browser WebSockets. The moment you connect that same pipeline to real enterprise telephony (PSTN / SIP trunks / FreeSWITCH), three brutal physical realities hit at once:
8kHz G.711 (PCMU/ PCMA) or L16 audio, while modern acoustic models (SenseVoice, Whisper, Silero VAD) expect 16kHz PCM. Naïve linear interpolation introduces aliasing artifacts that tank ASR accuracy and cause false VAD triggers.
In our open-source lab (01-voice-agent-rtp-bargein), we bypassed generic JSON-over-WebSocket wrappers and engineered a deterministic telephony loop:
[PSTN / SIP Caller (8kHz RTP, 20ms frames)]
│
▼ (UDP Raw RTP)
[FreeSWITCH Media Gateway]
│
├──► [SOXR VHQ Upsampler (8kHz ➔ 16kHz)] ──► [SenseVoice / Acoustic VAD (~18ms)]
│ │
│ ▼ (Barge-In Triggered!)
└──◄ [256-Frame Outbound Ring Buffer] ◄──── [0.16ms clearAudio() Hard Flush]
SOXR_VHQ) for 8kHz <-> 16kHz conversion preserves vocal formants and prevents high-frequency aliasing from poisoning the local VAD classifier.clearAudio()`` 0.16ms—producing immediate silence on the caller's phone.
A common mistake in Voice AI is waiting for the LLM or full-sentence STT transcript to confirm "the user is interrupting". Transcription requires context windows (often 200ms–400ms).
By splitting the pipeline into two asymmetric loops:
All reference code, architecture diagrams, and runnable barge-in experiments are open-sourced: