# Nvidia releases an open voice AI that talks while it listens, with a catch > Source: > Published: 2026-10-12 07:30:53+00:00 *Nvidia's NemotronLabs VoiceChat 11B can hold a real conversation and fire off a tool call without ever going quiet, something ChatGPT's voice mode and ElevenLabs still can't cleanly do. But the license says research only, so nobody's shipping it to paying customers yet.* On August 3, Nvidia published NemotronLabs VoiceChat 11B on Hugging Face, an open-weight model that listens and speaks on the same channel at the same time. Most voice assistants still run on a turn-based pattern: you talk, the system waits for silence, then it replies. VoiceChat skips that wait. According to Nvidia's own model card, it hits 448 milliseconds of smooth turn-taking latency on Full-Duplex-Bench 1.0, and it's the first open full-duplex model that can call a tool, say checking a flight status or a account balance, without freezing the conversation to do it. That tool-calling trick isn't a prompt trick. Nvidia built a third output channel into the model, a dedicated 587-million-parameter projection that runs in parallel with speech generation and writes out structured The model leans on a hybrid Mamba and Transformer architecture, built from a Fast Conformer speech encoder, Nvidia's own Nemotron Nano v2 language model as the backbone, and an Nvidia TTS decoder and codec bolted on top. Independent benchmarking from Artificial Analysis, cited in coverage from AIWeekly and others, places VoiceChat as the only open-weight speech model to land in the top three on both conversational dynamics and speech reasoning at once, the two axes the firm uses to score full-duplex systems. That matters because the products actually competing for voice-agent budgets right now, OpenAI's Realtime API and ElevenLabs' Conversational AI, both charge by the minute, somewhere between $0.10 and $0.30 depending on the plan and the caching setup. Cartesia and a handful of others chase the same sub-300-millisecond turn-taking target with closed models and metered pricing. VoiceChat gives a builder a full-duplex baseline they can actually open up and inspect, run on their own GPUs, and modify, instead of calling a vendor's endpoint and hoping the latency holds under load. [Nvidia Open Sources a Voice AI That Talks and Listens at Once](https://startupfortune.com/nvidia-open-sources-a-voice-ai-that-talks-and-listens-at-once/) PersonaPlex-7B runs full-duplex, listening and speaking on the same model at once, built on Kyutai's open Moshi architecture. Nvidia also reports a time-to-first-token of roughly 170 milliseconds, on top of its interruption and turn-taking numbers. - [open source voice AI that listens simultaneously](https://startupfortune.com/nvidia-open-sources-a-voice-ai-that-talks-and-listens-at-once/) - [Nvidia PersonaPlex enables natural overlapping conversation AI](https://startupfortune.com/nvidia-open-sources-a-voice-ai-that-talks-and-listens-at-once/) Here's the catch nobody selling the "Nvidia gives it away for free" headline wants to dwell on: VoiceChat ships under Nvidia's OpenMDW License Agreement version 1.1, and that license is explicit that the model is for research purposes only. You can download the weights, run them, publish papers about them, and build a demo. You cannot put them behind a paying product. That's a real constraint on the viral framing of this release. It's open in the sense that matters to a researcher comparing architectures or a startup testing whether full-duplex is worth the engineering lift before committing to a vendor. It is not yet open in the sense that matters to a call-center company trying to cut its ElevenLabs bill. Nvidia has moved faster than that before. In January, the company released a different full-duplex model, PersonaPlex, a 7-billion-parameter system built on the Moshi architecture, under a genuinely commercial setup: MIT-licensed code and weights on Nvidia's Open Model License, which allows commercial deployment without paying Nvidia a royalty. PersonaPlex hit a 240-millisecond interruption-response latency and a 90.8% turn-taking accuracy, and it switches speakers in roughly 0.07 seconds against the 1.3 seconds Nvidia's own testing clocked for Google's Gemini Live. VoiceChat is the newer, more capable model on tool calling and benchmark coverage. PersonaPlex is the one you're actually allowed to build a business on today. Frankly, that split tells you where Nvidia's strategy is headed. The company isn't trying to win voice AI on price the way it's accustomed to winning accelerators on raw throughput. It's seeding the field with research-grade weights that out-benchmark the commercial APIs on specific axes, tool calling, latency, dual-axis conversational scoring, while keeping a separate, already-commercial model a few months old in the market for anyone who wants to ship now. Voice-agent startups building call-center bots or AI companions get a preview of where the ceiling is heading, and a clear signal that whatever full-duplex model Nvidia licenses for production next will need to clear this bar on tool accuracy before it's worth the switch from OpenAI or ElevenLabs. For now, the 33% Pass@1 score on live tool execution is the number that should temper the hype. A voice agent that picks the right tool four times out of five but gets the arguments wrong more than half the time isn't ready to handle a real customer's account lookup unsupervised. Nvidia has shown the architecture works. Getting it production-accurate, and production-licensed, is still the open question. **Also read:** [Andrej Karpathy says the AI capability gap is now a sharp funnel](https://startupfortune.com/andrej-karpathy-says-the-ai-capability-gap-is-now-a-sharp-funnel/) • [OpenAI's secret math model spent three hours of compute per answer](https://startupfortune.com/openais-secret-math-model-spent-three-hours-of-compute-per-answer/) • [Wall Street's loudest commodity bulls say the AI-driven rally has barely started](https://startupfortune.com/wall-streets-loudest-commodity-bulls-say-the-ai-driven-rally-has-barely-started/) *This article is posted in [AI News](https://startupfortune.com/category/ai/), check it out for more related stories.* [Nvidia is in talks to buy the open-source AI lab it just backed](https://startupfortune.com/nvidia-is-in-talks-to-buy-the-open-source-ai-lab-it-just-backed/) Reflection AI's valuation has tripled to $25 billion in under a year, and Nvidia, already an $800 million investor, is weighing options from an acqui-hire to a bigger equity stake, Bloomberg and the Financial Times report. - [Nvidia acquisition talks with Reflection AI lab](https://startupfortune.com/nvidia-is-in-talks-to-buy-the-open-source-ai-lab-it-just-backed/) - [how Nvidia is buying open source AI startups](https://startupfortune.com/nvidia-is-in-talks-to-buy-the-open-source-ai-lab-it-just-backed/) ## Join the discussion [Open in the community →](https://startupfortune.com/community/) Almost there. Sign in and your reply posts straight away.