Muon: The Orthogonalized Optimizer Explained Through Equations, Intuition, Code and Results
Keller Jordan published Muon, an orthogonalized optimizer short for MomentUm Orthogonalized by Newton–Schulz, as a blog post in December 2024 after it cut the CIFAR-10 training record from 3.3 to 2.6 …