# How to Download and Run Kimi K3 Open Weights > Source: > Published: 2026-07-28 09:47:51+00:00 Moonshot AI shipped the full Kimi K3 weights on **July 27, 2026**: 2.8 trillion parameters, a one million token context window, and a 1.4 TB download. This guide covers what matters before you start that transfer. Where to get the weights, how much disk and VRAM you actually need, working vLLM and SGLang commands, what the license permits, and an honest answer to whether you should self-host at all. For most people that answer is no, and there's a better option further down. ## TL;DR | Question | Answer | |---|---| Official repo | `huggingface.co/moonshotai/Kimi-K3` | **Release date****Download size****~1.4 TB**(native MXFP4 safetensors)** License****Parameters****Context****Minimum realistic hardware****Runs on Ollama / LM Studio?****No.** llama.cpp cannot load the architecture.**Tech report**[→](https://github.com/MoonshotAI/Kimi-K3)`github.com/MoonshotAI/Kimi-K3` `k3_tech_report.pdf` ## What Kimi K3 actually is Worth understanding before you commit a terabyte of bandwidth, because the architecture is *why* the deployment is awkward. | Spec | Value | |---|---| | Architecture | Mixture-of-Experts (Stable LatentMoE) | | Total / activated params | 2.8T / 104B | | Layers | 93 (1 dense) | | Attention composition | 69 KDA + 24 Gated MLA | | Experts | 896 total, 16 selected per token, 2 shared | | Attention heads | 96 | | Vocabulary | 160K | | Context length | 1,048,576 | | Activation | SiTU-GLU | | Vision encoder | MoonViT-V2 (401M params) | | Quantization | MXFP4 weights / MXFP8 activations, quantization-aware from the SFT stage onward | | Modality | Text + image | Three things here are unusual enough to break tooling that works fine on other models. **Kimi Delta Attention (KDA).** Most layers are linear-attention with a fixed-size recurrent state instead of a growing KV cache, interleaved with periodic full-attention layers that preserve exact recall. That's what makes a 1M-token window affordable. It's also why prefix caching had to be redesigned from scratch in vLLM, since you can't cache a recurrent state the way you cache paged KV blocks. **Attention Residuals (AttnRes).** Instead of ordinary residual accumulation, each sublayer uses a learned pseudo-query to weight normalized residual states from preceding blocks. The residual stream becomes persistent cross-layer state with its own memory footprint. **Native MXFP4.** The model was *trained* to be 4-bit, not squeezed into it afterward. Self-hosted quality should match the hosted API, since it's the same quantization. The flip side: you can't quantize your way down to consumer hardware, because you're already there. ## Where to download Kimi K3 [https://huggingface.co/moonshotai/Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3) That's the canonical path. The MXFP4 quantization is native to this repo rather than living in a separate quantized fork, so there's nothing else to hunt down. Moonshot also maintains a [ModelScope organization](https://modelscope.cn/organization/moonshotai), typically the faster mirror if you're in mainland China. Check there for a K3 entry rather than assuming the path. A word on mirrors, because a release this big attracts them. Within hours of any major weight drop, dozens of reuploads appear with plausible names. Some are honest re-hosts. Some are stale. At least a few will be neither, and a 1.4 TB blob is not something you want to eyeball for tampering. Pull from `moonshotai` or a mirror you have an actual reason to trust, and **pin a commit hash**. Day-0 repos get amended, and "it worked yesterday" is not a reproducible build. ## Downloading it, step by step ### 1. Check you have the disk ``` df -h /data ``` You need ~1.4 TB for the weights themselves. Give yourself **2 TB** of real headroom, because the Hugging Face cache stages files before linking them into place, and you'll want room for logs, configs, and a second revision when Moonshot pushes a fix. ### 2. Install the CLI and turn on fast transfers ``` pip install -U "huggingface_hub[cli,hf_transfer]" export HF_HUB_ENABLE_HF_TRANSFER=1 hf auth login ``` `hf_transfer` is a Rust-backed downloader that will saturate a fast link in a way the default Python path won't. On a terabyte-scale pull, this is the difference between hours and most of a day. Authenticating isn't strictly required for public repos, but it raises your rate limits and saves you from mysterious throttling halfway through. ### 3. Pull the weights ``` hf download moonshotai/Kimi-K3 \ --revision main \ --local-dir /data/models/kimi-k3 ``` Real talk on timing: at a genuine 1 Gbps you're looking at roughly **3.5 hours** of continuous transfer, and that assumes you get full line rate. On a 10 Gbps link, closer to 20 or 25 minutes. Anything slower than gigabit and you should start this before you go to bed. Modern `huggingface_hub` resumes interrupted downloads automatically, so a dropped connection isn't fatal. Just re-run the same command. If you're following an older guide telling you to pass `--resume-download` , that flag is deprecated and resumption is now the default. Once it lands, **replace main with the actual commit hash** you got, and record it. Pin it in your deployment config. You will thank yourself the first time a silent repo update changes behavior in production. ### 4. Sanity-check the download ``` du -sh /data/models/kimi-k3 ls /data/models/kimi-k3/*.safetensors | wc -l ``` If `du` reports meaningfully less than ~1.3 TiB, your download didn't finish, regardless of what the progress bar claimed. ## The hardware reality check This is where the enthusiasm usually meets the spec sheet. Per vLLM's own engineering write-up, the full model **barely fits inside a single NVIDIA DGX B300**, and needs a minimum of **16 B200/GB200 GPUs** to serve on that hardware generation. Moonshot's own production guidance points at supernode configurations of 64+ accelerators. | Setup | Verdict | |---|---| | 8× B300 (or GB300 NVL72) | ✅ Works, this is the reference config | | 8× AMD MI355X / MI350X | ✅ Supported at launch via ROCm | | 16× B200 / GB200 | ✅ Supported | | 8× H100 80GB (640 GB total) | ❌ Weights alone are ~2× your VRAM | | Any single GPU | ❌ Not remotely | | 512 GB Mac Studio | ❌ Not even at 2-bit | That 8× H100 row deserves emphasis, because "8× H100 minimum" is a figure making the rounds. 640 GB of VRAM cannot hold 1.4 TB of weights, and there is no clever flag that changes this. Community quantization won't rescue a workstation either. K3 already ships at 4 bits, so there's no fat left to trim. A hypothetical 2-bit build still lands north of 700 GB, past a maxed-out Mac Studio, before you consider what that would do to a model this sparse. **Rough cost of entry:** an 8× B300 node runs somewhere in the neighborhood of $25 to $40 per hour on demand at current cloud pricing. Before you spin one up "just to try it," compare against the hosted API at $3 per million input tokens. You'd need to be pushing serious sustained volume before self-hosting pencils out on cost alone. The honest reasons to self-host are data residency, audit, and fine-tuning rights, not saving money. ## Serving it with vLLM vLLM shipped day-0 support. The reference command: ``` vllm serve moonshotai/Kimi-K3 \ --tensor-parallel-size 8 \ --trust-remote-code \ --load-format fastsafetensors \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser kimi_k3 \ --reasoning-parser kimi_k3 ``` **Important:** because of pre-release dependencies (notably FlashInfer), only the **Docker images work right now**. A plain `pip install vllm` won't get you there. Grab the images from the [Kimi K3 recipes page](https://recipes.vllm.ai/moonshotai/Kimi-K3). ### Roughly tripling your token rate Inferact trained and open-sourced a DSpark speculative decoder for K3. Add: ``` --speculative-config '{"model":"Inferact/Kimi-K3-DSpark","method":"dspark","num_speculative_tokens":7,"attention_backend":"FLASHINFER_MLA","draft_sample_method":"probabilistic","rejection_sample_method":"block"}' ``` Measured single-user decode on GB300 NVL72: **111 → 331 tok/s** at TP8, and **118 → 370 tok/s** at TP16. Acceptance rates run around 4.73 tokens per step on low-entropy work like coding, dropping to ~2.61 on creative writing. If you're serving interactive workloads, this is the single highest-leverage flag on the list. ### Backend selection | Situation | Flag | |---|---| | Tensor parallel (TP > 1) | `--moe-backend flashinfer_trtllm` | | Disaggregated / expert-parallel | `--moe-backend deep_gemm_mega_moe` | | NVLink interconnect | `--all2all-backend flashinfer_nvlink_one_sided` | | RDMA interconnect | `--all2all-backend deepep_v2` | ## Serving it with SGLang SGLang also has day-0 support with its own [K3 cookbook](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3). Pull the right image for your platform: ``` docker pull lmsysorg/sglang:kimi-k3 # CUDA 13 docker pull lmsysorg/sglang:kimi-k3-cu12 # CUDA 12 docker pull lmsysorg/sglang-rocm:rocm720-mi35x-k3-20260727 # ROCm ``` On Blackwell, the FlashInfer MXFP4 runner serves the weights natively. Elsewhere it falls back to Marlin (W4A16), with MegaMoE for short-context batch throughput. ## Five gotchas that will bite you **1. Preserved thinking history is mandatory.** K3 was trained in a mode where the *complete* prior assistant message, including `reasoning_content` and `tool_calls` and not just `content` , must be passed back into `messages` verbatim on every turn. Drop it and output degrades unpredictably. This also means you can't cleanly hand a running session over from another model mid-task. It's one line in the model card and it's the single most consequential detail in the whole release. **2. Prefix caching is off by default.** Unusual for vLLM, but the hybrid KDA cache design is still evolving, so it's disabled specifically for K3. Pass `--enable-prefix-caching` explicitly or you'll silently pay full prefill on every request, which is brutal for agentic workloads replaying long prompts. **3. Tool calling is occasionally flaky.** vLLM's own team flags this. K3 sometimes emits a tool-call format its own parser doesn't expect, yielding an empty `tool_calls` array, while clean probes on the identical setup parse fine. It's prompt- and run-dependent. Validate against your schema, retry or fall back on empty results, and consider constrained or structured tool calling. **4. There is no tokenizer.json, and no Jinja chat template.** K3 uses a tiktoken-style `encoding_k3.py` , and the chat template is a **Python render program** that builds the token sequence directly rather than rendering text first. This is precisely why llama.cpp can't load it, and why GGUF conversions floating around are weights-only curiosities rather than runnable models. Anything in your stack that assumes a Jinja template will need work. **5. Low scores are usually truncation, not failure.** K3 thinks a *lot* before answering, and `reasoning_effort` defaults to `max` . If your eval numbers look bad, check for cut-off generations and raise `max_tokens` before you start debugging anything else. ## The license: read this before you productize The repo carries a custom **Kimi K3 License**, not Apache 2.0 and not the "modified MIT" that shipped with earlier Kimi releases. Don't pattern-match from K2.x. For most of its length it reads like MIT: free to use, copy, modify, distribute, sublicense, sell, and to run, deploy, fine-tune, or build derivatives from the weights. Two conditions sit on top: **Model-as-a-Service threshold.** If you give third parties inference or fine-tuning access with meaningful control over inputs, parameters, or training data, and revenue across you and your affiliates passes**$20M over any consecutive 12 months**, you need a separate agreement with Moonshot.** Attribution threshold.**Any commercial product or service exceeding** 100M monthly active users**or**$20M in monthly revenue** must display "Kimi K3" prominently in its interface. Neither applies to purely internal use, or to access through Moonshot's own products and certified inference partners. Practical read: research, internal deployment, and most startup-scale commercial embedding are wide open. If you plan to resell K3 inference at scale, budget a license conversation alongside your GPU budget. *This is a summary, not legal advice. Read the LICENSE file in the repo, and have counsel look at it if real money depends on the answer.* ## Should you actually download this? Honestly? Most people reading this shouldn't. **Download the weights if** you need offline or VPC deployment, you have a hard data-residency requirement, you're fine-tuning, you're doing interpretability or architecture research, or you need to audit what you're running. Those are real reasons and the weights are the only way to get them. **Use the hosted API instead if** your actual pain was the API bill or rate limits. Moving a 2.8T model in-house doesn't remove the capacity problem, it relocates it onto hardware you now have to rent, operate, and debug. Hit `platform.kimi.ai` with model ID `kimi-k3` at $3.00 per million input tokens ($0.30 cached) and $15.00 per million output. Third-party routers like OpenRouter and Together AI carry it too. **Run something else locally if** you wanted a model on your own machine. K3 was never that. Depending on your hardware, look at GLM-5.2, Kimi K2.7 Code, or the smaller Kimi-VL and Moonlight models, all of which will actually load. A pattern worth stealing: route regulated or sensitive traffic to a self-hosted deployment, everything else to the hosted API. You pay the self-hosting premium only where it buys you something. ## FAQ **Is Kimi K3 free to download?** Yes. The weights are publicly available at no cost under the Kimi K3 License. Your costs are bandwidth, storage, and the compute to run it. **How big is the Kimi K3 download?** About **1.4 TB** for the native MXFP4 safetensors. Plan for 2 TB of free disk once you account for cache staging and headroom. **Can I run Kimi K3 on Ollama or LM Studio?** No. llama.cpp doesn't support the KDA/AttnRes architecture, and K3 ships no `tokenizer.json` . Use vLLM or SGLang. You *can* point an OpenAI-compatible client at a vLLM endpoint you serve yourself, which gets you a similar workflow. **What's the minimum hardware to run Kimi K3?** 8× NVIDIA B300 or 8× AMD MI355X in a single node, or 16× B200/GB200. An 8× H100 node cannot hold the weights. **Is Kimi K3 open source?** It's **open weights**, which isn't the same thing. Moonshot released trained parameters and a technical report, not training code, data, or a reproduction pipeline. The license also carries commercial conditions that would disqualify it from an OSI definition. **Is it better than Claude Fable 5 or GPT-5.6 Sol?** Mixed, and Moonshot is refreshingly upfront that K3 trails both on overall intelligence metrics. It leads on specific axes: BrowseComp (91.2), MCPMark-Verified (94.5), SWE-Marathon (42.0), and it took the #1 spot in Frontend Code Arena. It trails on others like HLE-Full and FrontierSWE. Treat any leaderboard as a reason to run your own eval, not a substitute for one. **Can I fine-tune Kimi K3?** The license permits it. Whether you can *afford* to is a different question. Fine-tuning a 2.8T MoE is a multi-node undertaking. vLLM has added rollout support with broader RL ecosystem integration on the roadmap. ## Quick links **Weights:**[huggingface.co/moonshotai/Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3)** Tech report:**[github.com/MoonshotAI/Kimi-K3](https://github.com/MoonshotAI/Kimi-K3)** vLLM recipes:**[recipes.vllm.ai/moonshotai/Kimi-K3](https://recipes.vllm.ai/moonshotai/Kimi-K3)** SGLang cookbook:**[docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3)**DSpark speculator:**[huggingface.co/Inferact/Kimi-K3-DSpark](https://huggingface.co/Inferact/Kimi-K3-DSpark)** API:**[platform.kimi.ai](https://platform.kimi.ai)(model ID`kimi-k3` ) *Specs and deployment details verified against the official model card, Moonshot's technical report, and the vLLM and SGLang day-0 documentation as of July 28, 2026. Day-0 repos move fast, so pin your revisions and check the model card before anything reaches production.* Share this article ## Ready to share files securely? Experience password protection, auto-expiry, and download limits with Comfyfile [Start Sharing Free](/)