NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt

NVIDIA released Nemotron 3.5 Transcribe, an open-weight, promptable speech model on Hugging Face and ModelScope that transcribes 40 languages, translates 31 to or from English, and performs speaker diarization in 13 languages, handling recordings up to an hour long and labeling up to eight speakers. NVIDIA said the model ranks first among open-weight models on the Hugging Face Open ASR Leaderboard for English, posts the lowest error rate on David AI's 21-language diarization leaderboard, and ranks first in VoiceArena's 14-system comparison, running on a single NVIDIA GPU from DGX Spark to Blackwell data-center systems. NVIDIA also published the Granary v2 multilingual dataset and curation pipeline plus NeMo Speech fine-tuning recipes, and reported that fine-tuning in a medical case study cut word error rate on held-out speakers by 45.8% relative to the best prompt.

Automatic Speech Recognition • 34B • Updated • 338 • 11 https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt Enterprise + Article https://huggingface.co/blog One open model for multilingual transcription, speaker attribution, translation, and audio Q&A, with top leaderboard results, plus the open Granary v2 data pipeline and fine-tuning recipes to adapt it. A transcript can capture every spoken word and still leave users or applications wondering who said what, how to translate it, or how to format domain-specific jargon. Multilingual calls, fast-paced meetings, and medical dictation each need different output: a translation, speaker labels, or correctly spelled medical terms. NVIDIA Nemotron 3.5 Transcribe is an open-weight, promptable model that turns recorded speech into useful text, now available on Hugging Face https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B and ModelScope https://modelscope.ai/collections/nv-community/Nemotron-35-Transcribe . It brings transcription, speaker attribution diarization , translation, and audio question answering into one checkpoint: it transcribes 40 languages, translates 31 of them to or from English, and attributes speakers in 13. It handles recordings up to an hour long and labels up to eight speakers. Developers can guide output with natural language, choose where their audio is processed, and adapt recognition to domain vocabulary and accents. Transcribe is first among open-weight models on the Hugging Face Open ASR Leaderboard https://huggingface.co/spaces/hf-audio/open asr leaderboard for English and sits on Artificial Analysis https://artificialanalysis.ai/speech-to-text/non-streaming ’s accuracy–active-parameter frontier. For speaker attribution, it posts the lowest error rate on David AI’s 21-language diarization leaderboard https://research.withdavid.ai/blog/dai-asr-i18n leaderboard and ranks first in VoiceArena’s 14-system comparison https://voicearena.com/diarization-bench/methodology . It runs on a single NVIDIA GPU, from DGX Spark to Blackwell data-center systems. Granary v2’s multilingual data available on Hugging Face https://huggingface.co/datasets/nvidia/Granary-v2 and ModelScope https://modelscope.ai/datasets/nv-community/Granary-v2 and reusable curation pipeline, together with NeMo Speech fine-tuning tools https://github.com/NVIDIA-NeMo/Speech/blob/main/tutorials/speechlm2/finetuning/README.md , extend prompt-level control to training: developers can define the written form they need and adapt the model to the speech their applications encounter. In a medical case study https://github.com/NVIDIA-NeMo/Speech/blob/main/examples/speechlm2/finetuning/docs/medical-asr-synthetic-data.md , fine-tuning cut word error rate WER on held-out speakers by 45.8% relative to the best prompt. Guide transcription and analysis with natural language The same checkpoint can turn one meeting recording into different outputs, depending on the instruction: - “Transcribe this recording and label each speaker.” - “Translate this audio into English.” - “What decisions were made in this meeting?” Experience the model in action in the video below, where it tackles a classic scene from the 1940 screwball comedy His Girl Friday—a film famous for its rapid-fire, heavily overlapping dialogue. Nemotron 3.5 Transcribe captions a scene from His Girl Friday 1940 and labels speakers, including overlapping dialogue. For unfamiliar terms, try context before training. In a healthcare setting, for example: Transcribe this medical discussion in the original language. Possible terms: HbA1c, metformin. Use these terms only when supported by the audio. Preserve standard medical spelling and write numbers in conventional written form. The model card https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B has recommended presets, including reasoning for audio questions, plus the full prompt guide and a task-by-language matrix: language support differs by task, and audio question answering is trained in English. Word-level timestamps are also supported. As with any speech model, spot-check names, speaker changes, and answers against the recording. From live classes to lasting lessons. Agora, whose real-time voice and video platform powers live conversations in apps, used word-level timestamps in a classroom demo: Nemotron 3.5 Transcribe turns a recorded class into a transcript shown alongside the replay, so students can see what was said and when. Agora’s blog https://www.agora.io/en/blog/how-live-teaching-can-become-a-lasting-learning-resource/ walks through how they built it. Leaderboard wins from transcription to speaker attribution Among open-weight models, Transcribe leads on what was said; on who said it, it tops two speaker-attribution benchmarks outright. Hugging Face Open ASR Leaderboard. The leaderboard measures transcription accuracy as word error rate WER on public test sets, in three tracks: 1. English: first among open-weight models at 5.33% average WER Figure 1 . 2. Multilingual: 4.62% mean WER across seven languages, the lowest among evaluated open-weight models in the model card’s Open ASR comparison. 3. Long-form: 10.98% mean WER, also the lowest in that comparison. Figure 1. Nemotron 3.5 Transcribe ranks first among open-weight models on the Hugging Face Open ASR Leaderboard https://huggingface.co/spaces/hf-audio/open asr leaderboard English at 5.33% average WER, with API models excluded. Lower WER means fewer transcription errors. Artificial Analysis Speech-to-Text Leaderboard. Artificial Analysis independently ranks speech-to-text models by AA-WER, its word error rate index. Nemotron 3.5 Transcribe ties for first among open-weight models at 2.8% AA-WER, with about 4B active parameters versus 24B for the model it ties. On AA-AgentTalk, the held-out voice-agent speech set that makes up 50% of the AA-WER index, Transcribe scores 1.8% WER, versus 2.3% for the next-best open-weight model. Transcribe pairs its speech encoder with a hybrid Mamba–Transformer mixture-of-experts language model, so only about 4B of its 30B parameters are active per token. That puts it on Artificial Analysis https://artificialanalysis.ai/speech-to-text/non-streaming ’s open-weight Pareto frontier of accuracy versus active parameters. On AA’s all-model chart, it ranks ahead of several widely used proprietary speech APIs. Figure 2. Leading AA accuracy https://artificialanalysis.ai/speech-to-text/non-streaming at about one-sixth the active parameters: Nemotron 3.5 Transcribe achieves 2.80% AA-WER with approximately 4B active parameters. Speaker attribution across languages and overlap. Sortformer speaker cues reach the language model with the acoustic features, so Transcribe can tag each turn during generation instead of relying solely on a later step to attach speakers to words. David AI DAI-ASR-I18N Leaderboard. On David AI’s DAI-ASR-I18N combined transcription-and-speaker-attribution track https://research.withdavid.ai/blog/dai-asr-i18n leaderboard , Transcribe ranks first among evaluated open-weight models and second overall. On the benchmark’s diarization track, which scores who spoke when across 21 languages, it has the lowest diarization error rate of the 14 systems tested, slightly ahead of NVIDIA’s dedicated Nemotron 3 Diarization https://huggingface.co/nvidia/Nemotron-3-Diarization model. Figure 3. NVIDIA models hold the top two spots on David AI’s multilingual diarization leaderboard https://research.withdavid.ai/blog/dai-asr-i18n leaderboard : Nemotron 3.5 Transcribe posts the lowest overall diarization error rate DER , 18.26% across 21 languages, followed by Nemotron 3 Diarization https://huggingface.co/nvidia/Nemotron-3-Diarization at 18.38%. DER measures speaker-time errors; lower is better. VoiceArena Diarization-Bench. On VoiceArena’s English full-session benchmark https://voicearena.com/diarization-bench/methodology , Transcribe posts the lowest concatenated minimum-permutation WER cpWER of 14 systems and ranks first in both in-person and online settings. It also drops the fewest words: deletions are 8.32% of reference words, versus a median of about 31%. Figure 4. NVIDIA models hold the top two spots in VoiceArena’s 14-system comparison https://voicearena.com/diarization-bench/methodology : Nemotron 3.5 Transcribe leads at 32.4% cpWER, followed by NVIDIA Multitalker Parakeet with Nemotron 3 Diarization at 34.2%. Diarization-Bench v1 covers English full sessions, overlapping speech and 2–8 speakers. cpWER counts word errors after matching speaker labels; lower is better. More evaluations. Ocular AI’s Converse-STT evaluation ranks Transcribe the top open model. Josh Talks’ Voice of India https://voiceofindia.ai leaderboard reports Transcribe’s results across Indian languages. Runs from edge to data center. Transcribe runs on a single NVIDIA GPU, from Jetson AGX Thor and DGX Spark to Blackwell data-center GPUs. On one NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with NVFP4 precision, it processes 100 hours of audio in under three minutes, measured on 30-second clips at high concurrency. Serve the checkpoint with vLLM https://github.com/NVIDIA-NeMo/Speech/blob/main/docs/source/speechlm2/vllm serving.rst today or run it locally with NeMo-Speech.cpp https://github.com/NVIDIA/NeMo-Speech.cpp . Transcribe is built for recorded audio, not live streaming. Specialized recordings can still call for adaptation to domain vocabulary and transcript conventions. Granary v2 and NeMo Speech fine-tuning address that, starting with the data behind Transcribe. Granary v2: The open data curation pipeline behind Transcribe A promptable speech model requires training data with unambiguous formatting targets. Granary v2 https://huggingface.co/datasets/nvidia/Granary-v2 is part of the Nemotron speech collection https://huggingface.co/collections/nvidia/nemotron-speech on Hugging Face and ModelScope https://modelscope.ai/collections/nv-community/Nemotron-Speech . Built with NVIDIA NeMo Curator https://github.com/NVIDIA-NeMo/Curator , its reusable pipeline prepared multilingual speech used in Nemotron 3.5 Transcribe’s training, producing aligned transcript variants, task labels, and inspectable quality metadata. The public manifest covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in Granary v1 https://huggingface.co/datasets/nvidia/Granary . These figures describe the dataset release, not Transcribe’s full training mixture or its sampling weights. Granary v2 generates three transcript forms for each recording Table 1 , the same verbatim and clean styles developers can request from Transcribe with a prompt. Beyond transcription, the pipeline produces short audio captions, code-switching restoration of loanwords, speech-grounded Q&A pairs, contextual ASR biasing terms, and translations with automatic quality scores. | Training target | Illustrative transcript | |---|---| | Text-normalized, verbatim | I paid, uh, twenty-five euros. | | Inverse text normalization, with disfluencies | I paid, uh, €25. | | Inverse text normalization, without disfluencies | I paid €25. | Table 1. Granary v2 training targets for the same utterance. The pipeline is organized into four reusable stages: - 1. Audio Preparation Metadata : Segments raw or long-form audio, tracks speakers across the full recording, and tags acoustic events and language. Granary v2’s sources were already segmented, so this release skipped this stage. - 2. Transcription ASR : Routes each language to primary and recovery ASR models, falling back when the primary fails a hallucination check. Human reference transcripts help preserve trusted spellings while adding spoken forms and disfluencies. - 3. Task Label Generation Text : Derives transcript variants, then checks character rate, hallucination patterns, and model or human-reference agreement. Only accepted records receive contextual-ASR terms and translations. - 4. High Fidelity HiFi : Filters out segments that are noisy, clipped, or contain multiple speakers, keeping a smaller set of high-quality recordings for uses beyond ASR. For the VoxPopuli dataset, it reduced 354k hours to 111k hours across 22 European languages. Figure 6. Granary v2 data curation pipeline, following one sample from raw audio through the Metadata, ASR, and Text stages to a training example. Developers can run the NeMo Curator pipeline https://github.com/NVIDIA-NeMo/Curator/tree/main/tutorials/audio/granary v2 with Ray and NVIDIA Dynamo https://github.com/ai-dynamo/dynamo backends on their own recordings or authorized public corpora, or build on the Granary v2 public manifests https://huggingface.co/datasets/nvidia/Granary-v2 . The manifests include the three transcript forms as tn raw, itn raw, and itn clean, the labels described above, entity hints, derived instruction pairs, provenance, and a per-record quality flag. Download source audio separately under each corpus’s original license, and follow the release’s published terms. Adapt recognition to the vocabulary that matters Start with a development-set prompt that specifies language, script, and transcript format. In the case study below, naming English and the Latin alphabet in the prompt cut full-benchmark WER from 30.8% to 23.8%. For domain errors that remain, the NeMo Speech fine-tuning workflow pairs representative speech with explicit written targets. The NeMo Speech medical ASR case study https://github.com/NVIDIA-NeMo/Speech/blob/main/examples/speechlm2/finetuning/docs/medical-asr-synthetic-data.md shows this progression on Eka Care’s open benchmark https://huggingface.co/datasets/ekacare/eka-medical-asr-evaluation-dataset of recordings from Indian doctors and medical students, which has 3,619 clips and no designated training split. The study builds adaptation data from generated medical text and public medicine catalogs, then combines approximately 153 hours of synthetic speech, 29 hours of accented English from Common Voice, and 63 hours of medical recordings from MultiMed. Figure 7. On the full test set of Eka Care’s Indian medical speech benchmark, a better prompt cuts WER from 30.8% to 23.8%, and fine-tuning brings it to 11.8%, about 62% fewer transcription errors than the model out of the box. The full set includes the 855 tuning clips used to choose the prompt and checkpoints; the held-out results below are the cleaner test. Synthetic generation separates the written target from its spoken rendering: “HbA1c” is the label, while “H B A one C” is sent to the synthesizer. Multiple TTS voices broaden the audio; real recordings add accents and recording conditions. Training text is filtered for exact matches and shared eight-word sequences with benchmark references. As with Granary v2’s explicit targets, training examples vary digits, unit spacing, company-name expansion, and brand-name hyphens, with instructions specifying the required convention. Some prompts include term lists with near-miss distractors, so the model learns to use a spelling only when the audio supports it. The final system uses one fixed convention prompt and one model pass, without retrieval or transcript post-processing. The recipe trains the speech encoder and projection and uses LoRA to adapt the language model. Two runs use different data mixtures; after adapter merging, selected checkpoints are uniformly averaged into one final model. Checkpoint selection uses decoded development-set results, rather than training loss alone. The full adaptation, from text generation through evaluation, took about 14 GPU-hours on one NVIDIA GB300. Domain gains, measured on held-out speakers The study splits the benchmark by speaker into 855 tuning clips, used to choose the prompt and checkpoints, and 2,764 held-out clips, decoded once per final system. Using Eka Care’s KARMA scoring pipeline https://github.com/eka-care/KARMA-OpenMedEvalKit , fine-tuning cuts held-out WER from 22.44% for the best-prompt baseline to 12.17%, a 45.8% relative reduction and the clearest evidence that the gains generalize to new speakers. Keyword WER falls from 9.2% to 6.5%, and semantic WER from 10.0% to 7.6%. On the same held-out speakers and scorer, the adapted model’s 12.17% WER beats an open medical-specialist model 13.79% and a widely used general-purpose ASR model 14.92% . On general English, LibriSpeech test-clean WER rises slightly, from 1.54% to 1.69%. The medical specialist still leads on exact matches for brand-name drugs and diagnostics; the study points to real recordings of clinicians saying brand names as the next step. For your own domain, keep at least 100 real, labeled clips for checkpoint selection, and evaluate the entity types your application depends on alongside general-speech retention. The case study includes commands to reproduce each step. The fine-tuning tutorials https://github.com/NVIDIA-NeMo/Speech/blob/main/tutorials/speechlm2/finetuning/README.md cover recipes, resource requirements, and tools for training, export, and evaluation, including new-language and code-switching ASR. The accompanying agent skill https://github.com/NVIDIA-NeMo/Speech/blob/main/.claude/skills/finetune-speech-llm/SKILL.md lets a coding agent run the workflow, from baseline selection and data checks through training validation, adapter merging, and evaluation. ModelScope also enables developers to fine-tune Nemotron 3.5 Transcribe through its ms-swift framework https://github.com/modelscope/ms-swift for specialized multimodal applications. Get started with Nemotron 3.5 Transcribe and Granary v2 Try Nemotron 3.5 Transcribe in the Hugging Face demo, https://huggingface.co/spaces/nvidia/nemotron-transcribe or on ModelScope Studios, then download the checkpoint from Hugging Face https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B or ModelScope https://modelscope.ai/collections/nv-community/Nemotron-35-Transcribe and serve it with the vLLM deployment guide https://github.com/NVIDIA-NeMo/Speech/blob/main/docs/source/speechlm2/vllm serving.rst . To skip self-hosting, use a hosted endpoint from Baseten https://www.baseten.co/library/nvidia-nemotron-35-transcribe/ , DeepInfra https://deepinfra.com/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B , or Simplismart. To adapt it to your domain, start with the Granary v2 manifests on Hugging Face https://huggingface.co/datasets/nvidia/Granary-v2 or ModelScope https://modelscope.ai/datasets/nv-community/Granary-v2 and the medical ASR case study https://github.com/NVIDIA-NeMo/Speech/blob/main/examples/speechlm2/finetuning/docs/medical-asr-synthetic-data.md , or let a coding agent drive the workflow with the fine-tuning agent skill https://github.com/NVIDIA-NeMo/Speech/blob/main/.claude/skills/finetune-speech-llm/SKILL.md . Models & Datasets - Model Checkpoints: NVIDIA Nemotron 3.5 Transcribe on Hugging Face https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B , ModelScope https://modelscope.ai/collections/nv-community/Nemotron-35-Transcribe | Interactive Demo on Hugging Face https://huggingface.co/spaces/nvidia/nemotron-transcribe , ModelScope - Hosted Endpoints: Baseten https://www.baseten.co/library/nvidia-nemotron-35-transcribe/ , DeepInfra https://deepinfra.com/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B , Simplismart \ - Dataset Manifests: Granary v2 Manifests on Hugging Face, https://huggingface.co/datasets/nvidia/Granary-v2 ModelScope https://modelscope.ai/datasets/nv-community/Granary-v2 - High Fidelity Subset: Manifests on Hugging Face, https://huggingface.co/datasets/nvidia/Granary-HiFi ModelScope https://modelscope.ai/datasets/nv-community/Granary-HiFi | Space on Hugging Face https://huggingface.co/spaces/nvidia/Granary-HiFi Repositories & Tutorials: NVIDIA NeMo Speech GitHub https://github.com/NVIDIA-NeMo/Speech | NeMo Curator Granary v2 Pipeline https://github.com/NVIDIA-NeMo/Curator/tree/main/tutorials/audio/granary v2 | vLLM Deployment Guide https://github.com/NVIDIA-NeMo/Speech/blob/main/docs/source/speechlm2/vllm serving.rst | Speech Fine-Tuning Guide https://github.com/NVIDIA-NeMo/Speech/blob/main/tutorials/speechlm2/finetuning/README.md | Fine-Tuning Agent Skill https://github.com/NVIDIA-NeMo/Speech/blob/main/.claude/skills/finetune-speech-llm/SKILL.md Partner Blogs: Agora https://www.agora.io/en/blog/how-live-teaching-can-become-a-lasting-learning-resource/ | Inflection AI | VoiceArena | Simplismart Local AI: What’s new in local AI on NVIDIA RTX PCs, laptops, and DGX Spark https://blogs.nvidia.com/blog/local-ai-rtx-spark-laptops-updates-models-agents-games