18:01
2026-06-30
gilesthomas.com
large-language-models
Writing an LLM from scratch, part 34a -- building a JAX training loop for an LLM training run
Giles Thomas built a JAX training loop for an LLM from scratch, using NNX and Optax, as part of a project to replicate GPT-2 small quality without referencing Sebastian Raschka's book. He trained a ba…