NVIDIA Releases Audex (Nemotron-Labs-Audex-30B-A3B): A Unified Audio-Text LLM That Preserves the Text Intelligence of Its Backbone NVIDIA released Audex (Nemotron-Labs-Audex-30B-A3B), a unified audio-text large language model that processes both audio and speech while preserving the text intelligence of its backbone. The model uses a Mixture-of-Experts architecture with 30B total parameters and avoids the text regression common in multimodal models through multi-stage training and text-only reinforcement learning. Audex is released under a noncommercial license alongside a smaller 2B variant. NVIDIA has released Audex Nemotron-Labs-Audex-30B-A3B , a unified audio-text large language model. It understands and generates both audio and speech. It also keeps the text intelligence of its backbone. The checkpoints, along with a smaller Audex-2B, are released under a noncommercial license. Most multimodal models pay a text tax. When labs add audio or vision output, text benchmarks often drop. NVIDIA research team reports this even for speech-only output models. Audex is designed to avoid that regression. TL;DR - Audex is a single 30B-A3B MoE model that handles audio in and out. - Audio inputs enter the text embedding space; audio outputs are treated like text tokens. - Text scores match the backbone, with small gains and small losses per benchmark. - Multi-stage SFT plus text-only Cascade RL avoids the usual multimodal text regression. - It is few of the open models that generate general audio beyond speech. What is Audex? Audex is a single Mixture-of-Experts MoE Transformer decoder. It has 30B total parameters and 3B activated per token. The backbone is Nemotron-Cascade-2-30B-A3B, a text-only MoE LLM. That backbone is a hybrid Mamba-Transformer with 52 layers. It uses 128 routable experts and 6 activated experts. The design is deliberately simple. Audio inputs are encoded and projected into the text embedding space. Text tokens and quantized audio tokens are then treated uniformly during generation. There is no thinker-talker split and no stacked cascade of models. Because the design stays simple, Audex runs on standard LLM stacks. These include Megatron-LM for training and vLLM for inference. It supports both an instruct mode and a thinking mode. Context length reaches 1M tokens. How the Unified Design Works Three components sit around the LLM backbone: - An audio encoder reads sound. Audex uses AF-Whisper from Audio Flamingo 3. It shares the Whisper Large-v3 architecture and handles 16kHz input. - Two-layer MLP adapters map audio features into the model dimension. - An extended vocabulary holds discrete audio output tokens. The original 131,072 tokens grow to 205,312. Audex uses two codecs for output. Speech uses X-Codec2 at 50 tokens per second. It applies single-layer finite scalar quantization FSQ with a 65,536 codebook. Non-speech sound uses X-Codec at 200 tokens per second. It uses four flattened residual vector quantization RVQ layers. Complex sound gets a larger token budget than speech. The interactive demo below computes these token counts for any duration. Training Audex needs no audio pretraining. It starts from the text-only SFT checkpoint. Training then adds capabilities stage by stage. The multi-stage SFT curriculum runs in order: text SFT, audio warmup, audio generation, then audio understanding. During audio warmup, text token embeddings stay frozen. Unfreezing them degraded text quality in ablations. NVIDIA research team also tested a single-stage recipe that mixes all data at once. That recipe broke long-context retrieval on NIAH. Multi-stage training avoided this, so it became the default. After SFT, the research team applies text-only Cascade RL and multi-domain on-policy distillation MOPD . Audio tasks show marginal or no regression after this text-only RL. Text scores improve at the same time. The data mix is large. It combines 157.4B audio tokens and 320.5B text tokens. Tasks span ASR, AST, TTS, text-to-audio, and audio understanding. Benchmark and Performance On text, Audex tracks its backbone closely. It scores 86.4 on MMLU-Redux against the backbone’s 86.3. It even leads on IMO AnswerBench, 81.1 versus 79.3. Small drops appear on MMLU-Pro and GPQA-Diamond. Audex also tops the text-only Qwen3.5-35B-A3B on several reasoning, alignment, and instruction-following benchmarks. The comparably sized Qwen3-Omni-30B-A3B-Thinking shows large reasoning drops versus its own backbone. | Benchmark | Audex 30B-A3B | Qwen3.5-35B-A3B | Qwen3-Omni-30B-A3B-Thinking | |---|---|---|---| | HMMT Feb25 | 92.2 | 89.0 | 60.4 | | IMO AnswerBench | 81.1 | 74.8 | 59.9 | | LiveCodeBench v6 | 85.3 | 74.6 | 59.2 | | ArenaHard v2 | 81.6 | 65.4 | 55.1 | | IFBench prompt | 77.8 | 70.2 | 52.4 | On speech recognition, Audex leads these open models. It records 6.82 average word error rate on the OpenASR leaderboard. That beats Step-Audio-R1.1-33B and Qwen3-Omni-30B-A3B-Thinking. On audio understanding the picture is mixed. Audex leads open models on MMAU. It shows gaps on MMAR and MMSU versus the strongest audio LLMs. Audex also generates general audio, which the other leading open models cannot. | Audio benchmark | Audex 30B-A3B | Step-Audio-R1.1-33B | Qwen3-Omni-30B-A3B-Thinking | |---|---|---|---| | MMAU | 75.6 | 73.6 | 75.4 | | MMAR | 63.2 | 69.8 | 66.4 | | MMSU | 63.4 | 74.1 | 70.2 | | Audio Entailment | 95.0 | 61.6 | 61.6 | | OpenASR WER, lower is better | 6.82 | 7.91 | 8.00 | | BigBenchAudio | 90.0 | 97.6 | not reported | Audex leads on MMAU, Audio Entailment, and OpenASR word error rate. It trails these open baselines on MMAR, MMSU, and BigBenchAudio. Use Cases with Examples - Consider a multilingual call center. Audex can transcribe a German call and translate it to English. Its speech translation output lists the source language, transcript, then English translation. - Consider accessibility tooling. A developer can add fixed-voice text-to-speech to a reading app. The Seed-TTS-Eval English word error rate is a low 1.70. - Consider sound design or prototyping. A caption like “birds chirping in a forest” yields a 10-second clip. General audio generation uses an enhancement VAE for 48kHz output. - Consider a voice assistant. Speech-to-speech runs as a cascade, but one checkpoint serves every step. Audex scores 90.0 on BigBenchAudio. Quick Start Example Audex follows the ChatML template. The reference container is vLLM 0.20.0. Audio input decoding needs the audio extras. Audio understanding, ASR, and translation share one audio question-answering format. The