00:02
2026-08-10
martinkristiansen.com
machine-learning
Optimizing a GPT-2-Class Transformer on a GPU
A developer's optimization campaign on an RTX 3080 Ti cut a GPT-2-small-class transformer's forward pass from 78.2ms to 1.60ms, a 49Γ speedup, beating torch.compile's 1.72ms and reaching 136,000 tokenβ¦