{"slug": "the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by", "title": "The Ultimate MoE Training Handbook: training Mixture-of-Experts models with Megatron-LM, setting by setting [P]", "summary": "Hugging Face released a Mixture-of-Experts training handbook aimed at engineers who have already trained a dense model and want to train an MoE next, according to author Nouamane, who worked on the Ultra-Scale and Smol Training Playbooks. The handbook covers training MoE models with Megatron-LM, setting by setting, and is hosted on a Hugging Face Space.", "body_md": "Handbook: https://huggingface.co/spaces/huggingface/moe-blogpost Hello, I'm Nouamane, one of the authors. I'm at Hugging Face and also worked on the Ultra-Scale and Smol Training Playbooks, if you've read those. This handbook is for people who have trained a dense model and want to train an MoE next", "url": "https://wpnews.pro/news/the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by", "canonical_source": "https://aiflash.com/news/146431/", "published_at": "2026-11-05 21:01:07+00:00", "updated_at": "2026-11-05 21:22:28.977303+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "ai-tools"], "entities": ["Hugging Face", "Nouamane", "Megatron-LM", "Ultra-Scale Playbook", "Smol Training Playbook"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by", "markdown": "https://wpnews.pro/news/the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by.md", "text": "https://wpnews.pro/news/the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by.txt", "jsonld": "https://wpnews.pro/news/the-ultimate-moe-training-handbook-training-mixture-of-experts-models-with-lm-by.jsonld"}}