TransformerEngine

NVIDIA released NeMo AutoModel, a library that integrates Expert Parallelism and DeepEP into Hugging Face's API, achieving 3.4x to 3.7x higher training throughput and 29% to 32% lower GPU memory consu…

NVIDIA released NeMo AutoModel, an open-source library that accelerates fine-tuning of Mixture-of-Experts (MoE) transformer models by 3.4-3.7x in training throughput and reduces GPU memory usage by 29…

NVIDIA has introduced the NVFP4 training recipe in TransformerEngine, enabling 4-bit mixed-precision pre-training on Blackwell GPUs with no measurable accuracy loss compared to FP8 baselines. The reci…