Giles Thomas

OpenAI's original GPT-2 small and medium weights consistently outperform custom-trained models in instruction-following evaluations, despite some custom models achieving better test loss, according to…

Giles Thomas completed building and training a GPT-2 small model from scratch using JAX, achieving a test loss of 3.418784, outperforming both his PyTorch model (3.538161) and the original GPT-2 small…

Giles Thomas built a JAX training loop for an LLM from scratch, using NNX and Optax, as part of a project to replicate GPT-2 small quality without referencing Sebastian Raschka's book. He trained a ba…