Moonshot AI's Kimi Linear attention architecture, introduced in October, now powers the company's 2.8-trillion-parameter Kimi K3 flagship model released in mid-July, marking the first production deplo…
Kimi K3, a 2.8-trillion-parameter open-weight model from Moonshot AI, is the largest open-weight model to date, scaling up their earlier Kimi Linear architecture from 48B parameters. The architecture …
Researchers at Moonshot AI introduced Kimi Linear, a hybrid linear attention architecture that outperforms full attention across short-context, long-context, and reinforcement learning scaling regimes…
Moonshot's Kimi K3, a 2.8-trillion-parameter mixture-of-experts model released July 16, achieves a 1M-token context window with decoding up to 6.3x faster than its predecessor by using Kimi Delta Atte…
The Kimi Team published a new attention architecture, Kimi Linear, on arXiv in 2025. The architecture uses Kimi Delta Attention interleaved 3:1 with full attention, becoming the first linear hybrid to…