Moonshot AI released Kimi Linear, a hybrid attention architecture that pairs its new Kimi Delta Attention (KDA) module with full attention in a 3:1 ratio, achieving up to 75% KV cache reduction and 6.…
Moonshot AI released Kimi K3, a 2.8-trillion-parameter open-weight model, the largest open-source model as of mid-2026 and the first to reach the 3-trillion-parameter class. The model uses Kimi Delta …
Moonshot AI's Kimi Linear attention architecture, introduced in October, now powers the company's 2.8-trillion-parameter Kimi K3 flagship model released in mid-July, marking the first production deplo…
Kimi K3, a 2.8-trillion-parameter open-weight model from Moonshot AI, is the largest open-weight model to date, scaling up their earlier Kimi Linear architecture from 48B parameters. The architecture …
Researchers at Moonshot AI introduced Kimi Linear, a hybrid linear attention architecture that outperforms full attention across short-context, long-context, and reinforcement learning scaling regimes…
Moonshot's Kimi K3, a 2.8-trillion-parameter mixture-of-experts model released July 16, achieves a 1M-token context window with decoding up to 6.3x faster than its predecessor by using Kimi Delta Atte…
The Kimi Team published a new attention architecture, Kimi Linear, on arXiv in 2025. The architecture uses Kimi Delta Attention interleaved 3:1 with full attention, becoming the first linear hybrid to…