NVIDIA and Google researchers have developed a host offloading technique in JAX that reduces high-bandwidth memory bottlenecks during LLM training, achieving up to 57% throughput improvements on NVIDI…
Google Cloud demonstrated elastic training on TPUs, where a worker failure during multi-node LLM training was recovered in under two minutes without restarting the job. Using the JAX AI stack (MaxText…
NVIDIA has introduced the NVFP4 training recipe in TransformerEngine, enabling 4-bit mixed-precision pre-training on Blackwell GPUs with no measurable accuracy loss compared to FP8 baselines. The reci…
MaxText has introduced new post-training capabilities, specifically Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), now available on single-host TPU configurations like v5p-8 and v6e-8. …