Cursor Research has open-sourced Mixture-of-Kittens (MoK), a deterministic mixture-of-experts training megakernel that reports up to 2.37x higher throughput than the strongest public baseline and powe…
NVIDIA released NeMo AutoModel, a library that integrates Expert Parallelism and DeepEP into Hugging Face's API, achieving 3.4x to 3.7x higher training throughput and 29% to 32% lower GPU memory consu…
NVIDIA released NeMo AutoModel, an open-source library that accelerates fine-tuning of Mixture-of-Experts (MoE) transformer models by 3.4-3.7x in training throughput and reduces GPU memory usage by 29…
AMD introduces DP Attention and Two-Batch Overlap (TBO) optimizations for DeepSeek-V4 inference on MI355X GPUs, using a coordinated prefill scheduler called PrefillDelayer to reduce padding waste and …
Researchers at Doubleword developed UCCL-EP, an expert parallel communications kernel that enables GPU-initiated communication patterns like DeepEP to work across arbitrary NIC-accelerator pairs, incl…
Researchers from UC Berkeley and other institutions released CommBench, a benchmark of over 100 GPU communication problems, to evaluate whether large language models can generate correct and efficient…