What Is Multi-Tier On-Policy Distillation? How NVIDIA Trained Nemotron 3 Ultra
NVIDIA used multi-tier on-policy distillation to train its Llama-3.1-Nemotron-Ultra-253B model, enabling it to outperform larger systems on reasoning tasks without additional compute. The technique ad…