Nvidia open-sources Nemotron 3 for real-time speaker tracking

Nvidia released Nemotron 3 Diarization on September 23, an open-weight speaker-diarization model with roughly 100 million parameters that identifies up to eight speakers simultaneously at latency as low as approximately 320 milliseconds, under the OpenMDW 1.1 license on Hugging Face and via the NeMo framework. On the AISHELL-4 benchmark using its low-latency profile, Nemotron 3 achieved a Diarization Error Rate of 9.8%, compared with 27.2% for its predecessor, Streaming Sortformer v2.1, released in July 2025, which supported only four speakers. Nvidia says deployment through inference providers including Baseten and DeepInfra can cost as low as $0.01 per audio hour in select configurations, targeting meeting transcription, call center analytics and podcast production.

Nvidia open-sources Nemotron 3 for real-time speaker tracking The 100-million-parameter model can identify up to eight speakers simultaneously with latency as low as 320 milliseconds Nvidia https://cryptobriefing.com/markets/nvidia/ just made it a lot easier to figure out who’s talking. The company released Nemotron 3 Diarization, an open-weight model built specifically to solve one of the most annoying problems in audio processing: accurately tracking which speaker said what, and when they said it, in real time. The model launched on September 23 under Nvidia’s OpenMDW 1.1 license, making it freely available on Hugging Face and through the company’s NeMo framework. It’s also accessible via inference providers like Baseten and DeepInfra, with deployment costs reportedly as low as $0.01 per audio hour in select configurations. What Nemotron 3 Diarization actually does Nemotron 3 tackles this with roughly 100 million parameters and what Nvidia calls its Streaming Sortformer architecture. The model handles up to eight speakers simultaneously, producing speaker-activity probabilities at 10 millisecond resolution. That’s granular enough to catch the kind of rapid-fire crosstalk that typically turns transcription software into a confused mess. The system operates in both streaming and offline modes. In streaming mode, it can run with configurable latency profiles that go as low as approximately 320 milliseconds. One of the more notable engineering decisions is the single-pass end-to-end design using what Nvidia calls an Arrival-Order Speaker Cache, or AOSC. Traditional diarization pipelines require multiple stages: first segmenting audio, then extracting speaker embeddings, then clustering those embeddings to figure out who’s who. Nemotron 3 collapses all of that into one forward pass, which simplifies deployment considerably. The model assigns speaker labels based on the order they first appear in a conversation. AI, tech, and the markets they move—in one daily briefing. Daily. Free. Join 34,000+ readers across crypto, finance, and policy. The accuracy jump is significant On the AISHELL-4 benchmark using its “low” latency profile, Nemotron 3 achieved a Diarization Error Rate of 9.8%. Its predecessor, the Streaming Sortformer v2.1 released in July 2025, scored 27.2% on the same test. Diarization Error Rate measures the proportion of time that speakers are incorrectly attributed, missed, or falsely detected. The previous model, formally known as diar streaming sortformer 4spk-v2.1, also only supported four speakers. Doubling that ceiling to eight opens the door for use cases involving panel discussions, group calls, and multi-party meetings where four-speaker limits were a real constraint. Where this fits in Nvidia’s broader strategy Nemotron 3 Diarization isn’t arriving in isolation. It’s part of a growing family of models Nvidia has been building around agentic AI and speech recognition, aimed at creating a full stack for voice-driven applications. The target use cases are meeting transcription, call center analytics, podcast production, and any scenario where multiple people are talking and software needs to keep track. By open-sourcing the weights under the OpenMDW 1.1 license, Nvidia has already seen early integrations surface through partnerships with MLX and FluidAudio. The $0.01 per audio hour price point through certain inference providers is also worth noting. At that cost, processing a full eight-hour workday of meeting audio would run roughly eight cents. Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy https://cryptobriefing.com/editorial-policy/ .