{"slug": "i-gave-a-talk-at-pytorch-conference-na-2026", "title": "🎤 I gave a talk at PyTorch Conference NA 2026!", "summary": "Independent researcher Jen Wei presented findings at PyTorch Conference NA 2026 on whether the Muon and Dion3 optimizers retain their convergence advantages over Adam under practical training conditions, covering mixed-precision training, checkpointing and distributed recovery. The work included hands-on integration of Dion3 into OLMo-core, training experiments, and documentation of two debugging episodes: a learning-rate scheduling failure in distributed optimizer state and a silent PyTorch tensor aliasing bug. Presentation materials and an OLMo-core pull request were published on Hugging Face.", "body_md": "Still letting this sink in a little. I went to PyTorch Conference NA last week and presented my investigation into a question that had been driving my independent research:\n\n**Do Muon and Dion3 retain their convergence advantages over Adam under practical training conditions?**\n\nThe work took me into mixed-precision training, checkpointing, distributed recovery, and a debugging saga involving learning-rate scheduling, distributed optimizer state, and a silent PyTorch tensor aliasing bug.\n\nAlong the way, I worked hands-on in OLMo-core, integrated Dion3, ran training experiments, and documented the journey.\n\nThis project now includes:\n\n **The research and experiments:** investigating optimizer behavior under practical training conditions.\n\n **The debugging saga:** [Part 1 — The LR Death Spiral](https://medium.com/@jenwei0312/the-lr-death-spiral-escaping-pytorchs-memory-traps-in-distributed-optimizers-da1d724a557f)\n\n **The season finale:** [Part 2 — The Anatomy of a Silent Tensor Aliasing Bug](https://medium.com/@jenwei0312/taming-orthogonal-optimizers-at-scale-part-2-the-anatomy-of-a-silent-tensor-aliasing-bug-147431a9c7b0)\n\n **The project materials:** [Presentation + OLMo-core PR on Hugging Face](https://huggingface.co/datasets/bird-of-paradise/muon-dion3-pytorchcon-2026)\n\nI’m proud that I took this investigation from independent experiments to a conference presentation, and grateful for the opportunity to share it with the PyTorch community.\n\nHere’s to more questions about how models learn, more experiments, and more things to debug.", "url": "https://wpnews.pro/news/i-gave-a-talk-at-pytorch-conference-na-2026", "canonical_source": "https://discuss.huggingface.co/t/i-gave-a-talk-at-pytorch-conference-na-2026/191007#post_1", "published_at": "2026-10-25 22:02:57+00:00", "updated_at": "2026-10-25 22:13:47.768260+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "developer-tools"], "entities": ["PyTorch Conference NA 2026", "Muon", "Dion3", "Adam", "OLMo-core", "PyTorch", "Hugging Face", "Jen Wei"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-gave-a-talk-at-pytorch-conference-na-2026", "markdown": "https://wpnews.pro/news/i-gave-a-talk-at-pytorch-conference-na-2026.md", "text": "https://wpnews.pro/news/i-gave-a-talk-at-pytorch-conference-na-2026.txt", "jsonld": "https://wpnews.pro/news/i-gave-a-talk-at-pytorch-conference-na-2026.jsonld"}}