NVIDIA released Nemotron 3.5 Transcribe on 28 October 2026. The open-weight speech-augmented language model (SALM) handles transcription in 40 languages, translation to or from English in 31 of them, speaker attribution for up to eight speakers in 13 languages, language identification, word-level timestamps, and audio question answering—all from a single checkpoint you steer with a natural-language prompt. It is offline only: you feed a complete recording (up to one hour / 65,536 tokens) and get the result. It is not a streaming recognizer.
The checkpoint is named 30B-A4B but the model card counts roughly 34B total parameters once the audio encoder is included, with about 4.3B active per token. The language backbone is the Nemotron 3.5 Lightning hybrid Mamba-2 / Transformer mixture-of-experts. A parallel expert encoder sits in front: a ~0.6B Transformer ASR encoder plus a frozen Nemotron 3 diarization (Sortformer) speaker encoder. Audio features and speaker cues are projected into the LLM embedding space. Joint multi-token prediction is trained so the model can serve as its own draft for speculative decoding; an optional DFlash2 draft is also released. Precisions include BF16, FP8 and NVFP4. License is OpenMDW-1.1 (commercial use allowed).
A 30B-class model that only activates ~4B per step can run on a single NVIDIA GPU. NVIDIA reports the NVFP4 checkpoint with DFlash2 processing up to 2,238 seconds of audio per second on one RTX PRO 6000 Blackwell, and 100 hours of short clips in under three minutes under high concurrency. That is batch throughput, not single-utterance latency, but it changes the economics of offline or near-offline pipelines. You no longer need a multi-GPU cluster just to get usable multilingual transcripts with speaker labels.
On the Hugging Face Open ASR English leaderboard it ranks first among open-weight models at ~5.33% average WER. Multilingual mean WER on the evaluated set is ~4.62%. Independent benches (Artificial Analysis, Cekura) also put it at or near the top of open models. Fine-tuning on domain data (medical, earnings calls, etc.) further drops error rates; NVIDIA showed single-GPU adaptation in 0.5–2.3 GPU-hours per task in their evaluations.
Input is 16 kHz mono (WAV, FLAC, MP3, OGG, etc.). You supply a text instruction such as “Transcribe the recording and label each speaker” or “Translate this audio into English and redact PII” or “What decisions were made?”. The same weights produce verbatim, written-form, or contextual output depending on the prompt. Phrase lists or domain context can be injected in the prompt rather than requiring a separate custom model.
Response includes the transcript plus optional speaker indices and word timestamps (via CTC alignment heads exposed through NeMo Speech middleware). For a Windows tool that buffers 5–15 s of WASAPI capture, this is the classic Fast-Transcription pattern: complete utterance in, stable display-form text out, then feed the next LLM or overlay. You avoid the partial-hypothesis churn of pure streaming models while still staying interactive enough for live-adjacent UI.
Tradeoffs are clear:
The Parallel Expert Encoder injects speaker activity into the ASR representations so the LLM can emit serialized speaker-attributed output without a separate post-processing diarization stage. Frame stacking in the ASR encoder (8×) means each output frame covers 80 ms; one second of audio becomes ~12.5 LLM tokens after projection. Internal 30-second encoder chunks are an implementation detail—they do not turn the model into a streaming service.
For local Windows deployment you can serve via vLLM, Hugging Face Transformers, or the NeMo Speech stack. Community GGUF conversions appeared quickly for llama.cpp-style runtimes. Pair it with a low-latency capture path (WASAPI exclusive or shared low-period, or loopback) and a small VAD (Silero or similar) so you only send complete utterances. The promptable interface makes it easy to switch output style per call type without re weights. Compared with pure encoder-decoder ASR, the SALM approach trades a bit of raw speed for flexibility: one model, many tasks, controllable style, and the ability to ask questions about the audio after (or while) transcribing. For tools that already buffer audio and then reason over the transcript, that is often the better engineering fit than stitching three separate models together.
Granary v2 (the open dataset released alongside) gives 463k+ hours across 48 languages and three transcript styles (verbatim with fillers, written-form, filler-free). That pipeline is available if you need to adapt further.
Nemotron 3.5 Transcribe is the strongest open offline option right now if you have an NVIDIA GPU and need multilingual, promptable, speaker-aware transcription without shipping audio to a third-party API. For Windows real-time-adjacent call and interview tools it slots in cleanly as the batch/utterance backend after a local VAD + capture stage. Streaming still needs a different model, but the quality and flexibility on complete clips is a step change.
I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.