{"slug": "nvidia-nemotron-3-5-super-and-lightning", "title": "Nvidia Nemotron 3.5 Super and Lightning", "summary": "NVIDIA released Nemotron 3.5 Super, a 120-billion-parameter, 12-billion-active Mixture-of-Experts Hybrid Mamba-Transformer multimodal model, and Nemotron 3.5 Lightning, a 30-billion-parameter, 3-billion-active text model, both trained for context lengths up to 1 million tokens. Nemotron 3.5 Super scores 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra with roughly 4.6× fewer active parameters, and gains up to 9.82 percentage points across five Indic benchmarks after a 100-billion-token Indic continued-pretraining stage. NVIDIA released BF16 and NVFP4 checkpoints with optimized inference recipes and standalone speculative drafters.", "body_md": "# NVIDIA Nemotron 3.5 Super and Lightning\n\n **Published:** \n\n#### [Technical Report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-Lightning-Technical-Report.pdf)\n\nWe introduce **NVIDIA Nemotron 3.5 Super** and **NVIDIA Nemotron 3.5 Lightning**, open and efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, and agentic workloads.\n\nNemotron 3.5 Super is a **120-billion-parameter, 12-billion-active multimodal model**, while Nemotron 3.5 Lightning is a smaller **30-billion-parameter, 3-billion-active text model**. Both are designed for efficient deployment and combine large-scale distillation, long-context training, native speculative decoding, and efficiency-focused quantization. Super further supports reasoning across text, images, video, and interactive tool environments.\n\n### Key Features\n\n1. **Efficient hybrid architecture:** Mixture-of-Experts Hybrid Mamba-Transformer models activate only a fraction of their total parameters for each token.\n2. **Native speculative decoding:** Multi-Token Prediction (MTP) heads accelerate generation, with dedicated post-training that improves acceptance at deeper draft positions.\n3. **Long-context support:** Both models are trained for context lengths up to**1 million tokens** .\n4. **Agentic and multimodal capabilities:** Super supports text, images, video, software-engineering agents, visual tool use, and long-running interactive workloads.\n5. **Reasoning control:** Super supports reasoning-off, regular, and medium-effort modes, as well as inference-time reasoning-budget control.\n6. **Efficient deployment:** Released BF16 and NVFP4 checkpoints are accompanied by optimized inference recipes and standalone speculative drafters.\n\n### Training Nemotron 3.5\n\nBoth models begin with a dedicated **100-billion-token MTP warm-up**, followed by **1 trillion tokens of distillation** from the next-larger Nemotron 3 model. Nemotron 3.5 Super is distilled from Nemotron 3 Ultra Base, while Lightning is distilled from Nemotron 3 Super Base. Following distillation, Lightning undergoes a 21-billion-token long-context phase that extends it to a 1-million-token context length. Super proceeds through the additional continued-pretraining and mid-training stages below.\n\nNemotron 3.5 Super then undergoes additional training targeted at multilingual, long-context, and agentic use cases:\n\n- **Indic continued pretraining:** A 100-billion-token stage includes synthetic and natural Indic data spanning 22 Indian languages and 34 language-script configurations. It improves all five evaluated Indic benchmarks, with gains of up to**9.82 percentage points** .\n- **Long-context training:** A 35-billion-token continued-pretraining stage extends Super to a**1-million-token context length** .\n- **Agentic software-engineering mid-training:** A 65.4-billion-token stage at 256K context uses repositories, issue–pull-request pairs, commit histories, build and CI logs, and agent trajectories. It improves long-horizon software engineering and robustness across agent harnesses.\n- **Long-context recovery:** Because Super’s 256K mid-training stage reduced its RULER-1M score, a final 9-billion-token recovery stage at 1-million-token context restores the score from 33.5 to 81.0.\n\nSuper’s post-training pipeline combines supervised fine-tuning, Reinforcement Learning from Verifiable Rewards (RLVR), Reinforcement Learning from Human Feedback (RLHF), specialized teacher training, and Multi-Teacher On-Policy Distillation (MOPD). This process transforms the text-only base model into a multimodal agent while consolidating capabilities from reasoning, agentic, alignment, video, and vision-tool-use teachers. Lightning follows a lighter text-only pipeline using supervised fine-tuning and RLVR.\n\n### Key Highlights\n\n- Nemotron 3.5 Super achieves **51.78% on SWE-bench Pro** , surpassing Nemotron 3 Ultra while using approximately**4.6× fewer active parameters** .\n- Agentic mid-training improves DeepSWE pass@1 from **6.4% to 10.4%** and improves robustness across multiple software-engineering agent harnesses.\n- Parallel agent execution followed by self-verification improves Terminal-Bench 2.1 from **62.7% to 71.9%** , DeepSWE from**24.6% to 34.5%** , and BrowseComp from**71.5% to 79.8%** on the evaluated setups.\n- MTP boosting improves average speculative-decoding acceptance length from 4.287 to 4.718 under greedy decoding and from 3.787 to 4.612 under probabilistic draft sampling.\n- The released Nemotron 3.5 Super NVFP4 checkpoint achieves **99.62% median accuracy recovery** relative to BF16 across 36 benchmarks.\n- On a single NVIDIA GB200 GPU, Nemotron 3.5 Super reaches up to **4,856 output tokens/s/GPU** while sustaining at least 50 tokens/s/user on the evaluated 20-turn agentic workload.\n- Under the evaluated interactive serving conditions, NVFP4 delivers **2.43× the throughput of BF16** . Nemotron 3.5 Super delivers**2.34× and 1.90× the throughput** of Qwen3.5 122B-A10B and Qwen3.8 Flash-Next, respectively.\n\n### Efficient Inference\n\nNemotron 3.5 is optimized for interactive, long-context agentic workloads. Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned to sustain responsive per-user generation rates at high aggregate throughput.\n\nPost-training MTP boosting improves native speculative decoding, while standalone parallel drafters target low-concurrency, memory-bound deployments:\n\n- **LiLiCorr** for Nemotron 3.5 Super\n- **DFlash** and**DSpark** for Nemotron 3.5 Lightning\n\nNemotron 3.5 Super is validated across NVIDIA A100, H100, B200, GB200, and GB300 platforms, as well as DGX Spark, DGX Station, and RTX-class GPUs. Its quantization and speculative-decoding configurations can be tuned to the target platform and workload.\n\n### Open Source\n\nWe are releasing base, post-trained BF16, and NVFP4-quantized checkpoints, together with speculative drafters, datasets, model cards, cookbooks, evaluation recipes, and training recipes.\n\n#### Checkpoints\n\n- [**Nemotron 3.5 Super VL 120B-A12B Base BF16**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Base-BF16) — base model\n- [**Nemotron 3.5 Super VL 120B-A12B BF16**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) — post-trained model\n- [**Nemotron 3.5 Super VL 120B-A12B NVFP4**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4) — post-trained and NVFP4-quantized model\n- [**Nemotron 3.5 Super VL 120B-A12B NVFP4 LiLiCorr**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr) — standalone LiLiCorr drafter\n- [**Nemotron 3.5 Lightning 30B-A3B Base BF16**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16) — base model\n- [**Nemotron 3.5 Lightning 30B-A3B BF16**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16) — post-trained model\n- [**Nemotron 3.5 Lightning 30B-A3B NVFP4**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4) — post-trained and NVFP4-quantized model\n- [**Nemotron 3.5 Lightning 30B-A3B NVFP4 DFlash**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash) — standalone DFlash drafter\n- [**Nemotron 3.5 Lightning 30B-A3B NVFP4 DSpark**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark) — standalone DSpark drafter\n\n#### Data and Recipes\n\n- **Nemotron-Pretraining-Indic-v1:** approximately 2.83 trillion synthetic output tokens across 22 Indian languages and 34 language-script configurations.\n- **Post-training datasets and environments:** resources for supervised fine-tuning, reinforcement learning, reasoning, coding, software engineering, tool use, and multilingual capabilities.\n- [**Nemotron 3.5 Super evaluation recipes**](https://github.com/NVIDIA-NeMo/Gym/blob/main/nemotron_recipes/super-3.5/reproducibility.md)\n- [**Nemotron 3.5 Lightning evaluation recipes**](https://github.com/NVIDIA-NeMo/Gym/blob/main/nemotron_recipes/lightning-3.5/reproducibility.md)\n- [**NVIDIA Nemotron Developer Repository**](https://github.com/NVIDIA-NeMo/Nemotron)\n\n#### Technical Report\n\nThe [**Nemotron 3.5 Super and Lightning Technical Report**](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-Lightning-Technical-Report.pdf) provides full details on pretraining, post-training, quantization, inference optimization, evaluation methodology, and benchmark results.", "url": "https://wpnews.pro/news/nvidia-nemotron-3-5-super-and-lightning", "canonical_source": "https://research.nvidia.com/labs/nemotron/Nemotron-3.5-Super-and-Lightning/", "published_at": "2026-10-29 16:58:40+00:00", "updated_at": "2026-10-29 17:17:37.863291+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-agents", "ai-tools"], "entities": ["NVIDIA", "Nemotron 3.5 Super", "Nemotron 3.5 Lightning", "Nemotron 3 Ultra Base", "Nemotron 3 Super Base", "SWE-bench Pro", "RULER-1M"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/nvidia-nemotron-3-5-super-and-lightning", "markdown": "https://wpnews.pro/news/nvidia-nemotron-3-5-super-and-lightning.md", "text": "https://wpnews.pro/news/nvidia-nemotron-3-5-super-and-lightning.txt", "jsonld": "https://wpnews.pro/news/nvidia-nemotron-3-5-super-and-lightning.jsonld"}}