{"slug": "nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for", "title": "Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning", "summary": "NVIDIA released Nemotron 3 Ultra, a 550B-parameter hybrid Mamba-Transformer model with 55B active parameters, achieving up to 6x higher inference throughput than state-of-the-art LLMs while maintaining accuracy. The model, trained on 20 trillion tokens with a 1M token context, is optimized for autonomous agentic reasoning and is open-sourced on HuggingFace.", "body_md": "arXiv:2606.15007v1 Announce Type: new\nAbstract: We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 trillion text tokens, then extended the context length to 1M tokens, and post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD). Nemotron 3 Ultra is our most capable model yet, employing multiple key technologies - LatentMoE, Multi Token Prediction (MTP), NVFP4 pre-training, multi-environment RLVR, MOPD, and reasoning budget control. Nemotron 3 Ultra achieves up to ~6x higher inference throughput as compared to state-of-the-art publicly available LLMs while attaining on-par accuracy. The state-of-the-art accuracy, high inference throughput, and 1M token context length make Nemotron 3 Ultra ideal for long-running autonomous agentic tasks. We open-source the base, post-trained, and quantized checkpoints, along with the training data and recipe on HuggingFace.", "url": "https://wpnews.pro/news/nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for", "canonical_source": "https://arxiv.org/abs/2606.15007", "published_at": "2026-06-16 04:00:00+00:00", "updated_at": "2026-06-16 04:22:53.449879+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure", "ai-agents", "machine-learning"], "entities": ["NVIDIA", "Nemotron 3 Ultra", "HuggingFace", "LatentMoE", "Multi Token Prediction", "NVFP4", "MOPD", "RLVR"], "alternates": {"html": "https://wpnews.pro/news/nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for", "markdown": "https://wpnews.pro/news/nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for.md", "text": "https://wpnews.pro/news/nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for.txt", "jsonld": "https://wpnews.pro/news/nemotron-3-ultra-open-efficient-mixture-of-experts-hybrid-mamba-transformer-for.jsonld"}}