19:55
2026-08-18
ezyang.github.io
machine-learning
How to Parallelize a Transformer for Training
A new interactive adaptation of the JAX scaling book chapter on transformer training parallelism lets readers drag sliders to see how data, tensor, pipeline, and expert parallelism affect communicatio…