22:19
2026-06-25
lilianweng.github.io
machine-learning
Scaling Laws, Carefully
Scaling laws in deep learning describe a predictable power-law relationship between training loss and model size, dataset size, and compute, enabling extrapolation of resource requirements for larger …