{"slug": "tutorial-understanding-and-implementing-the-muon-optimizer", "title": "[Tutorial] Understanding and Implementing the Muon Optimizer", "summary": "A tutorial on the Muon optimizer has been updated with large-scale experiments testing whether Muon and Dion3 retain their convergence advantages over Adam under practical training conditions, work conducted for a PyTorch Conference North America 2026 presentation. The author reports the effort became a debugging saga involving learning-rate scheduling, distributed optimizer state, and a silent tensor aliasing bug, with the presentation and an OLMo-core pull request collected in one place.", "body_md": "**Update: Muon → Muon/Dion3 experiments at scale!**\n\nSince writing this tutorial, I’ve been investigating whether Muon and Dion3 retain their convergence advantages over Adam under practical training conditions, as part of my work for [PyTorch Conference](https://events.linuxfoundation.org/pytorch-conference-north-america/program/schedule/?topic=training) NA 2026.\n\nThe work grew into a debugging saga involving learning-rate scheduling, distributed optimizer state, and a silent tensor aliasing bug.\n\nI’ve collected the presentation and my OLMo-core PR in one place:\n\n **PyTorch Con NA 2026 project materials:**\n\nIf you’re interested in how orthogonalized optimizers behave beyond the algorithm itself, come take a look!", "url": "https://wpnews.pro/news/tutorial-understanding-and-implementing-the-muon-optimizer", "canonical_source": "https://discuss.huggingface.co/t/tutorial-understanding-and-implementing-the-muon-optimizer/167717#post_4", "published_at": "2026-10-25 21:54:21+00:00", "updated_at": "2026-10-25 22:13:50.499131+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "artificial-intelligence"], "entities": ["Muon", "Dion3", "Adam", "PyTorch Conference", "OLMo-core"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tutorial-understanding-and-implementing-the-muon-optimizer", "markdown": "https://wpnews.pro/news/tutorial-understanding-and-implementing-the-muon-optimizer.md", "text": "https://wpnews.pro/news/tutorial-understanding-and-implementing-the-muon-optimizer.txt", "jsonld": "https://wpnews.pro/news/tutorial-understanding-and-implementing-the-muon-optimizer.jsonld"}}