Google's MaxText team reproduced AI2's OLMo 3 7B language model from scratch on Google Cloud TPUs, matching AI2's published loss curve over the full ~5.93T-token / 1.41M-step stage-1 pre-training budg…
A new benchmark suite called JAXBench, comprising 50 JAX workloads for AI-generated kernel optimization on Google Cloud TPUs, shows that target-specific documentation context matters more than model s…
NVIDIA and Google researchers have developed a host offloading technique in JAX that reduces high-bandwidth memory bottlenecks during LLM training, achieving up to 57% throughput improvements on NVIDI…
Google Cloud demonstrated elastic training on TPUs, where a worker failure during multi-node LLM training was recovered in under two minutes without restarting the job. Using the JAX AI stack (MaxText…
NVIDIA has introduced the NVFP4 training recipe in TransformerEngine, enabling 4-bit mixed-precision pre-training on Blackwell GPUs with no measurable accuracy loss compared to FP8 baselines. The reci…
MaxText has introduced new post-training capabilities, specifically Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), now available on single-host TPU configurations like v5p-8 and v6e-8. …