04:55
2026-07-16
thegustafson.com
machine-learning
Optimizers: Momentum, Adam, and Learning Rate Schedules
Optimizers like Momentum, Adam, and learning rate schedules solve problems with vanilla SGD, such as zigzagging in steep valleys and slow progress in flat directions. Momentum adds inertia to smooth o…