Cursor Open-Sources Mixture-of-Kittens (MoK): A Deterministic MoE Training Megakernel for GB300 NVL72 Racks

Cursor Research has open-sourced Mixture-of-Kittens (MoK), a deterministic mixture-of-experts training megakernel that reports up to 2.37x higher throughput than the strongest public baseline and powers Composer training across tens of thousands of GPUs. The kernel, available on GitHub under Apache-2.0, requires NVIDIA Blackwell SM100 or SM103 GPUs, Python 3.12+, PyTorch 2.10+, and CUDA toolkit 13.0+, limiting adoption to organizations with NVL72 capacity. MoK uses pull-based dispatch and push-based combine, a ring token buffer to minimize CPU involvement, and supports BF16 and MXFP8 precision modes.

Cursor Research has open-sourced Mixture-of-Kittens MoK https://cursor.com/blog/mixture-of-kittens , the mixture-of-experts training megakernel behind its Composer https://cursor.com/blog/composer-2 models. MoK fuses every MoE communication and computation step into a single deterministic kernel. Cursor team reports up to 2.37x higher throughput than the strongest public baseline. It already powers Composer training across tens of thousands of GPUs. Is it deployable Yes, but the hardware floor is high. MoK is on GitHub https://github.com/cursor/mixture-of-kittens under Apache-2.0. It requires NVIDIA Blackwell SM100 or SM103 GPUs, which means GB200 NVL72 or GB300 NVL72 racks. It also needs Python 3.12+, PyTorch 2.10+, and CUDA toolkit 13.0+. Inter-GPU buffers rely on PyTorch symmetric memory. That limits realistic adopters to organizations that own or rent NVL72 capacity. Frontier labs, funded model startups, GPU neoclouds, and national computing centers fit. Single-node teams and 8-GPU shops do not. Applications are narrow but high-value. They include pretraining and post-training of DeepSeek-V3-style MoE models. Determinism also makes it useful for on-policy RL post-training and internal ablations. Relevant industries are AI model development, cloud GPU infrastructure, code-generation tooling, and quantitative research. MoE layer as the bottleneck Cursor’s earlier work covered the compute side. The research team wrote its own MXFP8 and NVFP4 training kernels https://cursor.com/blog/kernels and a ‘warp decode’ https://cursor.com/blog/warp-decode path for MoE inference. Those assumed inter-GPU communication was handled separately. In production, communication became the limiting factor. The MoE layer can consume more than half of end-to-end training time. Moving to GB300 NVL72s changed the problem again. A rack is 72 GPUs inside one NVLink domain, which allows fine-grained overlap. But the integrated Grace CPUs are slow relative to the GPUs. CPU-GPU synchronization therefore has to be minimized aggressively. Three design decisions that matter Communication direction is chosen per operation : Existing approaches such as DeepEP https://github.com/deepseek-ai/DeepEP lean on push-based transfers. Cursor’s microbenchmarks show push moves fewer total bytes in one direction. That leaves the reverse NVLink lane mostly idle. Pull-based dispatch delivers up to 29% higher NVLink bandwidth utilization under expert imbalance. It also eliminates cross-GPU completion signals. Push dispatch signalling measured 103 µs against 18 µs for pull, roughly 5.8x. MoK therefore uses pull-based forward dispatch and push-based forward combine. The backward pass mirrors this with pull reverse-combine and push reverse-dispatch. One schedule table serves all four, costing under 3% of MoE runtime. Overlap granularity sits between the extremes : Comet https://arxiv.org/abs/2502.19811 is fine-grained; DeepEP is coarse-grained. Cursor team argues the optimum is in the middle and workload-dependent. The heuristic targets at least two full SM waves per expert-grouped GEMM. For Kimi 2.5 shapes, the base model for Composer 2.5, the floor is 2,368 tokens. Measured latency matches that estimate closely. A ring token buffer removes the CPU from the loop : The alternatives are dropping tokens or asking the CPU to size buffers. MoK instead cycles a fixed ring buffer of a few hundred megabytes. It does so at minibatch granularity, interleaving dispatch and combine at macrobatch boundaries. The ring is walked in reverse to minimize forward activation replay during backward. MoK is built as a megakernel https://hazyresearch.stanford.edu/blog/2025-05-27-no-bubbles and is fully deterministic. It supports BF16 and MXFP8 precision modes. Scheduling runs through Blackwell’s Cluster Launch Control, so inter-rack RDMA does not serialize behind it. Router weight gradients use a SonicMoE https://arxiv.org/abs/2512.14080 -style calculation fused into the SwiGLU backward. Results Layer benchmarks ran in a single NVL72 rack at EP degree 64. Each GPU held 2,048 tokens before routing. Baselines were NCCL+PyTorch, DeepEP+PyTorch, DeepEP+TransformerEngine, and HybridEP+Megatron. Shapes covered Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. Against the fastest baseline, MoK is up to 2.37x faster for MXFP8 forward. The other figures are 1.78x MXFP8 backward, 1.92x BF16 forward, and 1.58x BF16 backward. End-to-end testing used 512 GPUs across several GB300 NVL72 racks. Tokens per second per GPU rose from 760.9 to 1,070.2, a 1.41x gain. Key Takeaways - MoK fuses all MoE communication and computation into one deterministic megakernel for NVL72 racks. - Pull dispatch plus push combine cuts signalling from 103 µs to 18 µs. - A ring token buffer drops zero tokens and removes CPU-GPU synchronization entirely. - Up to 2.37x over the fastest public baseline; 1.41x end-to-end on 512 GPUs. - Apache-2.0, but it demands Blackwell SM100/SM103, CUDA 13.0+, and PyTorch 2.10+. Check out the GitHub Repo and Also, feel free to follow us on Technical details https://cursor.com/blog/mixture-of-kittens . and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.