09:37
2026-07-11
machinebrief.com
machine-learning
MoE Memory Breakthrough: Training Trillion-Parameter Models with Ease
Researchers have developed a memory-efficient training stack for Mixture-of-Experts models that achieves 4.7x to 8.2x higher per-GPU throughput compared to a finely-tuned FSDP2 baseline, using fewer t…