{"slug": "the-active-ingredient-in-muon-s-grokking", "title": "The Active Ingredient in Muon's Grokking", "summary": "A new study from researchers at arXiv finds that the Muon optimizer's speedup in reaching the grokking threshold on modular arithmetic comes from orthogonalization via the Newton-Schulz iteration, not spectral-norm constraints. Ablations show orthogonalize-only matches full Muon, while spectral-only is no faster than AdamW and unreliable across learning rates. Reducing Newton-Schulz iterations from five to one accelerates grokking but makes the solution fragile, with the canonical five iterations being the learning-rate-robust choice.", "body_md": "arXiv:2607.20512v1 Announce Type: new\nAbstract: The Muon optimizer reaches the grokking threshold on modular arithmetic faster than AdamW. Prior work attributes this to \"spectral-norm constraints plus orthogonalized momentum\" but does not isolate which mechanism matters. To better understand Moun's behavior, we run multi-seed and multi-learning-rate sweeps to decompose and stress-test the effect. First, an ablation shows the speedup comes from orthogonalization (the Newton-Schulz iteration): orthogonalize-only matches full Muon, whereas spectral-only is no faster than AdamW and is unreliable, and this verdict holds across learning rates. Second, a mechanistic analysis finds that orthogonalizing optimizers reach generalization at roughly 3x lower spectral norm and, controlling for how much the embedding actually moves, settle into a lower-norm solution rather than simply perturbing the embedding less. Third, reducing the Newton-Schulz iteration count from five to one accelerates reaching the threshold but makes the grokked solution fragile, prone to transient collapse, with fragility that grows with learning rate; a single iteration is fast and stable only at small learning rate, while the canonical five iterations are the learning-rate-robust choice. We also show spectral scaling can be dropped at no measured cost. A methodological thread runs throughout: under a stability-aware metric, \"faster\" claims about grokking optimizers can invert, so we report both first-crossing and sustained-grok times. To support reproducibility, we release our full training and analysis code at https://github.com/louiswang524/muon-grokking-frontier", "url": "https://wpnews.pro/news/the-active-ingredient-in-muon-s-grokking", "canonical_source": "https://arxiv.org/abs/2607.20512", "published_at": "2026-07-24 04:00:00+00:00", "updated_at": "2026-07-24 04:11:43.519737+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks"], "entities": ["Muon", "AdamW", "arXiv", "Newton-Schulz iteration"], "alternates": {"html": "https://wpnews.pro/news/the-active-ingredient-in-muon-s-grokking", "markdown": "https://wpnews.pro/news/the-active-ingredient-in-muon-s-grokking.md", "text": "https://wpnews.pro/news/the-active-ingredient-in-muon-s-grokking.txt", "jsonld": "https://wpnews.pro/news/the-active-ingredient-in-muon-s-grokking.jsonld"}}