Moonshot released Kimi K3 model weights and technical report Moonshot AI released the model weights and technical report for Kimi K3, a 2.8-trillion-parameter Mixture-of-Experts model with native visual understanding and a 1-million-token context window, along with open-sourced infrastructure technologies MoonEP, FlashKDA, and AgentEnv. The company claims a 2.5× improvement in scaling efficiency through innovations including Kimi Delta Attention and Attention Residuals, enabling each unit of compute to produce roughly 2.5 times as much intelligence as before. Moonshot released Kimi K3 model weights and technical report Source:https://mp.weixin.qq.com/s/tryHe81IyM6nr0fBPDz72g Kimi K3 Open Day Thank you for your patience. Today is Kimi K3 Open Day. We are releasing the Kimi K3 model weights https://huggingface.co/moonshotai , the technical report https://github.com/MoonshotAI/Kimi-K3 , and the key infrastructure technologies that support Kimi K3 model training: MoonEP https://github.com/moonshotAI/moonep , FlashKDA https://github.com/MoonshotAI/FlashKDA/ , and AgentEnv https://github.com/kvcache-ai/AgentEnv . We hope this will accelerate the deployment and adoption of frontier intelligence and advance AGI research. Kimi K3 Weights Released Kimi K3 https://mp.weixin.qq.com/s? biz=Mzk0NDU1MDkyNg==&mid=2247488658&idx=1&sn=a4fd7ead31c1b49357b29535c0dea4fe&scene=21 wechat redirect is our most capable model. It is a 2.8-trillion-parameter Mixture-of-Experts model with native visual understanding and support for a 1-million-token context window. Kimi K3 has roughly three times as many parameters as Kimi K2.5. Scaling, however, is not merely the accumulation of parameters. Under constrained compute conditions, we achieved a 2.5× improvement in scaling efficiency through a set of technical innovations, including Kimi Delta Attention https://github.com/MoonshotAI/Kimi-K3 , Attention Residuals https://github.com/MoonshotAI/Kimi-K3 , and MoonEP https://github.com/MoonshotAI/MoonEP . In compute-optimal terms, each unit of compute now produces roughly 2.5 times as much intelligence as before. Everyone can now download and deploy the Kimi K3 model, whether for internal research and development or for integration into end-user products. For other use cases, please see the Kimi K3 License https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE . Download Kimi K3 model weights https://huggingface.co/moonshotai Kimi K3 Technical Report Published Alongside the Kimi K3 model weights, we are also making public our model training methods. The Kimi K3 technical report https://github.com/MoonshotAI/Kimi-K3 is now available. The technical report covers the following details: KDA + AttnRes: KDA and Gated MLA are mixed at a 3:1 ratio to enable efficient long-context modeling, while block-level attention residuals improve cross-layer information flow. Stable LatentMoE: Each token activates 16 out of 896 routed experts. SiTU-GLU and Quantile Balancing maintain training stability under extremely high sparsity. MoonViT-V2: The visual encoder is trained from scratch using next-token prediction, without contrastive pretraining. It reaches the baseline performance of SigLIP initialization while providing a more stable optimization process. Post-training and evaluation: The report covers large-scale task synthesis across general reasoning, general agents, and coding agents; reinforcement learning infrastructure for million-token contexts; and full evaluation results from nearly 20 internal benchmarks. These points are only an overview. Detailed discussions and ablation experiments are included in the technical report. Download the Kimi K3 technical report https://github.com/MoonshotAI/Kimi-K3 Key Kimi K3 Infrastructure Technologies Open-Sourced Model capability depends on stable training systems at the infrastructure layer. Today, we introduce three infrastructure technologies that support Kimi K3 training: MoonEP, FlashKDA, and AgentEnv. They cover key links from high-performance communication and high-performance kernels to distributed RL environments. FlashKDA had already been open-sourced; MoonEP and AgentEnv are being officially open-sourced with this release. MoonEP: MoonEP is a high-performance communication library designed for extremely large, fine-grained MoE models. It enables expert-parallel communication to maintain high efficiency even under imbalance. FlashKDA: FlashKDA is our high-performance kernel implementation of Kimi Delta Attention. On NVIDIA H20, compared with the flash-linear-attention baseline, its prefill speed is 1.72–2.22× faster, and it can be used directly as a replacement backend for flash-linear-attention. AgentEnv: AgentEnv is a sandbox system developed jointly with KVCache.ai for running agent environments at scale. It provides high-fidelity, strongly isolated sandboxes for Kimi K3 post-training, with flexible support for fast snapshotting, restoration, and forking. It is designed for massively parallel agent workflows and training tasks. These three technologies are key to Kimi K3’s training efficiency and stability. We are opening them up in the hope that they can also help you train next-generation models. Why We Choose Openness We firmly believe in the value of open-weight models. They lower the barrier to accessing intelligence, promote innovation, and give users greater control over their data, privacy protection, and ownership. Over the past few weeks, we have been pleased to hear support from many members of the AI community and from technology leaders who share our vision. We believe that for a technology with far-reaching impact like AGI, a broad and open ecosystem is the most suitable foundation. We will continue to contribute to that ecosystem. Quick Start GitHub Repository Content Repository Overview MoonshotAI/Kimi-K3 is titled “Open Frontier Intelligence.” The repository is public and includes an assets directory, LICENSE , README.md , and k3 tech report.pdf . The repository page links to the Kimi chat site, Moonshot AI homepage, Hugging Face organization, Twitter/X account, Discord, ModelScope organization, the Kimi K3 license, the technical blog, and the full report. 1. Model Introduction Kimi K3 is an open-weight, native multimodal agentic model and Moonshot AI’s most capable model to date. It is a 2.8T-parameter model built on Kimi Delta Attention KDA and Attention Residuals AttnRes , with native vision capabilities and a 1-million-token context window. It is described by the repository as the world’s first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning. Key Features New Architecture: Kimi K3 is built on Kimi Delta Attention KDA and Attention Residuals AttnRes , and scales up MoE sparsity with a Stable LatentMoE framework that activates 16 out of 896 experts, yielding an approximate 2.5× improvement in overall scaling efficiency over Kimi K2. Long-Horizon Coding: Operating with minimal human oversight, Kimi K3 sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools, from GPU kernel optimization and compiler development to vision-in-the-loop game development, CAD, and chip design. Agentic Knowledge Work: Kimi K3 advances end-to-end knowledge work, producing deep research with interactive visualizations, widgets and dashboards, motion design, and video editing, powered by its native multimodal architecture. Native Multimodality & Long Context: Kimi K3 understands text, images, and video within the same model, and supports a 1-million-token context window. Open Frontier Weights: Moonshot AI releases the full Kimi K3 model weights under the Kimi K3 License, making frontier intelligence openly available for research, deployment, and further innovation. 2. Model Summary 3. Evaluation Results Benchmark | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol | Claude Opus 4.8 | GPT-5.5 | GLM-5.2