01:12
2026-08-20
gilesthomas.com
machine-learning
Use the built-in GELU, don't roll your own!
PyTorch's built-in GELU function is 20% faster than a hand-rolled version when training GPT-2 small models, according to a developer's benchmark. The same code training the same model on the same data…