# Kimi K3: Is It Really Open Source? And What Would It Take to Recreate It? > Source: > Published: 2026-07-17 16:51:48+00:00 # Kimi K3: Is It Really Open Source? And What Would It Take to Recreate It? ## 1. What Is Kimi K3? Kimi K3 is the latest flagship model from **Moonshot AI** (Beijing), released **July 16, 2026**. It is a **Mixture-of-Experts** model with **2.8 trillion total parameters** and about **32 billion active per token** (16 out of 896 experts). It supports **1 million tokens of context** and handles text, images, and video natively. On launch, K3 ranked **#3 on the Artificial Analysis leaderboard** behind Claude Fable 5 (Anthropic) and GPT 5.6 Sol (OpenAI), but **#1 on Arena.ai for front-end coding**. ## 2. Key Specs | Spec | Value | |---|---| | Total params | 2.8T | | Active params | ~32B | | Architecture | MoE | | Experts | 896 total, 16 active per token | | Context | 1M tokens | | Multimodal | Text + Images + Video (native) | | Attention | Kimi Delta Attention (KDA) + Attention Residuals (AttnRes) | | MoE framework | Stable LatentMoE | | Activation | Sigmoid Tanh Unit (SiTU) | | Quantization | MXFP4 weights, MXFP8 activations (QAT from SFT onward) | | Thinking | Always on (max effort by default) | | Input price | $3.00/MTok (cache miss), $0.30/MTok (cache hit) | | Output price | $15.00/MTok | ## 3. Is It Actually Open Source? **Status as of July 17, 2026** | Element | Public? | Details | |---|---|---| | Weights | Pending release by July 27, 2026 | Announced by Moonshot | | Technical report | Not yet published | "Coming soon" | | API | Yes | OpenAI-compatible | | Consumer apps | Yes | Kimi.com, Kimi Work, Kimi Code | | Architecture code | No | KDA, AttnRes not released | | Training code | No | No RL scripts | | Training data | No | Neither pretraining nor post-training | **The Pattern** Moonshot follows a consistent playbook: **K2 (July 2025)**: Weights on HuggingFace with Modified MIT License. MAI training code released.** K2.5 (Jan 2026)**: Same terms. Community accused them of not being truly open source.** K3 (July 2026)**: Weights promised, but no training code or datasets. **Verdict**: K3 is **open-weight**, not open source by OSI standards. You will be able to download and use the weights freely, but the entire training system stays proprietary. This is the same business model DeepSeek and other Chinese labs use: release weights for adoption, keep training details secret for competitive advantage. ## 4. Moonshot Timeline ``` Oct 2023: Kimi chatbot (128K context) Mar 2024: Kimi 2M character context Jan 2025: Kimi k1.5 (RL scaling, matched o1) Apr 2025: Kimi-VL (16B MoE, open source) Jun 2025: Kimi-Dev (72B coding), Kimi-Researcher Jul 2025: Kimi K2 (1T params, 32B active, open-weight) Sep 2025: K2-Instruct-0905 (256K context) Oct 2025: Kimi Linear (48B, KDA preview) Jan 2026: Kimi K2.5 (multimodal, Agent Swarm) Apr 2026: Kimi K2.6 (1000+ parallel agents) Jun 2026: Kimi K2.7 Code Jul 16, 2026: Kimi K3 (2.8T params) Jul 27, 2026: K3 weights release (promised) ``` ## 5. What Would It Take to Recreate K3? Assume you have the hardware (tens of thousands of H200/B200 GPUs with supernode interconnects of 64+ accelerators). ### 5.1 Architecture (Not Documented) K3's architectural innovations are **not publicly detailed**: **Kimi Delta Attention (KDA)**: A hybrid linear attention variant. Previewed in Kimi Linear (Oct 2025), but precise details are not public.** Attention Residuals (AttnRes)**: Selectively recovering representations across network depth instead of accumulating them uniformly. Implementation unknown.**Stable LatentMoE**: Routing framework with 896 experts (16 active). Includes** Quantile Balancing**that eliminates balancing hyperparameters.** Per-Head Muon**: An extension of MuonClip that optimizes attention heads independently.** Sigmoid Tanh Unit (SiTU)**: New activation function, not documented.** Gated MLA**: Evolved version of Multi-Head Latent Attention. **Estimated effort**: 6-12 months of reverse engineering just to reconstruct the architecture. ### 5.2 Pretraining Data - K3 was trained on a corpus of unknown size (estimated 20T+ tokens vs 15.5T for K2) - Proprietary **rephrasing** techniques **Training curriculum**(LR scheduling, data proportions for multilingual/code/math, long-context phases) is entirely not public ### 5.3 Post-Training Data and Pipeline **Agentic data synthesis**: Unknown scale, likely much larger than K2 (3000+ real tools, 20,000+ synthetic)** Preference data**for initializing the critic model: internal only** Trajectory generation pipeline**: not public ### 5.4 RL Algorithms and Hyperparameters K3's technical report has not been published yet. Based on K2 and Kimi-Researcher, the unknowns include: | Parameter | Known? | Detail | |---|---|---| | RL algorithm | No | REINFORCE? PPO? GRPO? | | Reward model | No | Not specified for K3 | | Gamma-decay factor | No | Unknown thresholds | | Temperature decay schedule | No | Not published | | Token budget limits | No | Not specified | | Negative sample control ratio | No | Not published | | Turn-level partial rollout | No | Precise mechanism unknown | | Context management strategy | No | Needed for 1M tokens | ### 5.5 Training Infrastructure **QAT with MXFP4/MXFP8**: proprietary implementation** Supernode config with 64+ accelerators**: not documented** Fully balanced expert-parallel training**: code not released** Asynchronous rollout system**for agent RL: not public** Sandbox Kubernetes + MCP**: configuration not documented ### 5.6 Estimated Cost **Pretraining only**: ~$15-25M (estimate based on 2.8T params with 20T+ tokens)** R&D reverse engineering**: $5-10M** Post-training RL**: $3-8M for experiments** Infrastructure**: $2-5M for cluster setup** Total**:**$25-50M+, with 12-18 months of work from a team of 50+ researchers** ## 6. Why K3 Matters **The US-China Gap Is Shrinking** K3 shows Chinese labs can produce frontier models months earlier than US analysts expected. Bank of America: "Despite persistent hardware/compute capacity constraints in China, K3 demonstrates that pre-training scaling, paired with architectural innovation, can still deliver step-change gains." **Price Pressure** | Model | Output price (per MTok) | |---|---| | Claude Fable 5 | $50.00 | | Kimi K3 | $15.00 | | GLM-5.2 (z.ai) | $4.40 | | DeepSeek V4 | $0.87 | K3 costs **one-third of Fable 5** for comparable performance. **Distillation Allegations** Anthropic accused Moonshot (along with z.ai, MiniMax, Alibaba, DeepSeek) of "illicit distillation attacks" -- using US model outputs to train their own. The US Congress is considering legislation to stop this. **Regulatory Context** - July 2026: US temporarily blocked Mythos/Fable 5 exports - June 2026: Executive order on GPT-5.6 Sol licenses - K3 launched on the eve of the **World AI Conference 2026** in Shanghai ## 7. Final Verdict | Question | Answer | |---|---| | Is K3 open source? | No, it is open-weight (weights yes, training code no) | | Can you download and use it? | Yes, from July 27, 2026 | | Can you modify it? | Yes (Modified MIT License) | | Can you recreate it from scratch? | No, not without proprietary knowledge | | Is it reproducible with $50M? | Maybe, with 12-18 months of reverse engineering | | Does it compete with top US models? | Yes, #1 in front-end coding, #3 overall | **Bottom line**: Kimi K3 is a remarkable engineering achievement, but reproducing it is a pipe dream for anyone outside Moonshot AI. The weights will be open, but the science behind them stays mostly secret. This is the new industry standard: models keep getting more powerful, and they keep getting harder to reproduce. ## Sources [Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3?ref=grigio.org)[Kimi API Platform - K3 Quickstart](https://platform.kimi.ai/docs/guide/kimi-k3-quickstart?ref=grigio.org)[Fortune: Moonshot's K3 pushes Chinese AI into Fable-level territory](https://fortune.com/2026/07/16/moonshots-kimi-k3-pushes-chinese-ai-into-fable-level-territory/?ref=grigio.org)[CNBC: Moonshot AI unveils Kimi K3](https://www.cnbc.com/2026/07/17/moonshot-ai-kimi-k3-model-openai-anthropic-china.html?ref=grigio.org)[Axios: China's open-weight Kimi model stuns AI world](https://www.axios.com/2026/07/16/moonshot-kimi-ai-china-model-openai-anthropic?ref=grigio.org)[Wikipedia: Kimi (chatbot)](https://en.wikipedia.org/wiki/Kimi_(chatbot)?ref=grigio.org)[arXiv: Kimi K2 Technical Report (2507.20534)](https://arxiv.org/abs/2507.20534?ref=grigio.org)[arXiv: Kimi k1.5 (2501.12599)](https://arxiv.org/abs/2501.12599?ref=grigio.org)[Kimi-Researcher: End-to-End RL Training](https://moonshotai.github.io/Kimi-Researcher/?ref=grigio.org)[IntuitionLabs: Kimi K2 Technical Deep Dive](https://intuitionlabs.ai/articles/kimi-k2-technical-deep-dive?ref=grigio.org)[dbreunig.com: How Kimi K2 RL'ed Qualitative Data](https://www.dbreunig.com/2025/07/31/how-kimi-rl-ed-qualitative-data-to-write-better.html?ref=grigio.org)[OpenRouter: Kimi K3 Pricing](https://openrouter.ai/moonshotai/kimi-k3?ref=grigio.org)[Gizmodo: China Just Dropped Another Bomb](https://gizmodo.com/china-just-dropped-another-bomb-on-americas-frontier-ai-companies-2000786670?ref=grigio.org)