August 4, 2026, (Inside AI) — OpenAI has disclosed the engineering blueprint behind GPT‑Live, its third-generation voice AI system that abandons turn-based architecture for full-duplex streaming. The system processes audio continuously, enabling it to listen and speak simultaneously without a turn detector, a shift that slashes conversational latency and eliminates awkward s.
The technical deep-dive, authored by Justin Uberti and Zahan Malkani of OpenAI's technical staff, reveals how the team rebuilt model inference, context management, and media transport over six months to support real-time voice at scale. GPT‑Live now powers ChatGPT Voice, including new capabilities for computer control and agent coordination in the desktop app.
"GPT‑Live puts the voice model in control of the conversation: audio flows in and out of the model, while deeper reasoning and tool use happen asynchronously," Uberti and Malkani wrote. The core innovation is a dedicated media path that keeps audio frames streaming without interruption, while a separate asynchronous boundary handles delegation to frontier models like GPT‑5.5 for complex tasks.
This design solves a fundamental problem in voice AI: the trade-off between responsiveness and intelligence. Previous systems relied on small turn-detector models to decide when to stop listening and start generating, often cutting users off or introducing lag. GPT‑Live's voice model natively manages turn-taking, using cues like tone and pacing that cascaded speech-to-text-to-speech pipelines miss.
"For voice AI, knowing when to speak is harder than it sounds," the authors noted. "Human speakers effortlessly hand off to each other in a fraction of a second, but previous voice AI systems couldn't keep up with this rhythm."
A Stateful Architecture With Seamless Handoffs #
To sustain long-running conversations, OpenAI engineered a stateful inference system with dynamic context compaction. As session context grows beyond model limits, the system compacts it without pausing audio. A replacement model instance is warmed, prefilled with compacted context, and switched in seamlessly.
"While the original model instance keeps chatting, the system compacts the context and prepares a replacement model instance with the new context," the post explains. "Once that instance is ready, we can switch over without any media interruption." This mechanism also handles model instance spin-ups and downs based on demand, ensuring uninterrupted media flow.
The team rewrote the media frontend and inference logic in Go, replacing a previous Python implementation. This cut frame delivery latency significantly, with the new system's 95th percentile matching the old system's 50th percentile. WebRTC provides the transport layer, chosen for its ability to handle packet loss and clock drift through audio stretching and catch-up playback.
Protocol Hacks Slash Startup Latency #
OpenAI also tackled transport-layer delays. Standard WebRTC sessions require up to six network round trips to start, but the team developed the WebRTC Abridged Roundtrip Protocol (WARP), which collapses this to a single round trip. WARP piggybacks the DTLS handshake over ICE, uses DTLS 1.3, and pre-negotiates SCTP and data channels.
Complementing WARP is Instant Connect, which pre-negotiates SDP parameters outside the critical path. This allows a session to start with a single UDP packet. "Together, Instant Connect and WARP dramatically reduce the time from user intent to live media flow," the authors wrote. The specifications have been published openly and adopted by libwebrtc and Pion, with IETF standardization underway.
Before launch, OpenAI ran a silent shadow test on production traffic, routing a fraction of sessions to GPT‑Live in read-only mode while users heard the old system. This exposed real-world bottlenecks: CPU-side stream handlers saturated earlier than expected, and geographic routing added latency. The team responded by co-locating inference with users and adding granular telemetry.
"The silent test became an early launch rehearsal, not only for how much traffic the system could accept, but for how quickly we could detect, contain, and recover from failure," the post states. The architecture now underpins a growing platform for real-time interaction, with a GPT‑Live API planned to extend voice capabilities across devices and modalities. OpenAI is hiring engineers to continue this work.