Kimi K3 Open Weights Live: Self-Host or Use the API?

Moonshot AI released the open weights of its 2.8-trillion-parameter Kimi K3 sparse Mixture-of-Experts model on Hugging Face under an Apache 2.0 license, but the 594 GB MXFP4-quantized model requires approximately 1.4 TB of aggregate GPU memory, making self-hosting feasible only for teams with data-center-grade infrastructure. The release includes Moonshot's Kimi Delta Attention vLLM prefill cache implementation, which is necessary to serve the full 1M-token context window, and the API remains the practical option for most users at $3 per million input tokens and $15 per million output tokens.

Kimi K3’s open weights landed on Hugging Face at 00:00 UTC today — 11 days after the API launched and a week after the prep guide https://byteiota.com/kimi-k3-open-weights-july-27-developer-prep/ told you to clear disk space. The 594 GB MXFP4 release is live at moonshotai/Kimi-K3-MXFP4 https://huggingface.co/moonshotai/Kimi-K3-MXFP4 under an Apache 2.0 license. Whether downloading them makes sense for your team is a harder question. What Actually Dropped Today The release is the MXFP4-quantized version of Kimi K3 — Moonshot’s 2.8-trillion-parameter sparse Mixture-of-Experts model that topped SWE Marathon and Program Bench https://byteiota.com/kimi-k3-open-model-developer-guide/ on launch. At 4-bit precision with per-block scaling, the download clocks in at 594 GB. Full FP16 would be roughly 5.6 TB; MXFP4 gets you near-lossless quality at a fraction of the weight. Apache 2.0 means commercial use, modification, and redistribution are all permitted. What it does not mean is open-source: training data, training code, and methodology remain Moonshot’s. You are getting the weights, not the recipe. Critically, Moonshot ships its Kimi Delta Attention KDA vLLM prefill cache implementation alongside the weights. This matters because standard vLLM prefix caching does not map to KDA’s hybrid linear attention architecture. Without Moonshot’s implementation, you cannot efficiently serve the full 1M-token context window. The Hardware Gate Here is the number that ends most self-hosting conversations: at runtime, K3 requires approximately 1.4 TB of aggregate GPU memory. That is not storage — that is VRAM. The minimum viable configuration is 8x H100 80GB GPUs 640 GB aggregate . That barely fits the model for inference; leave room for KV caches and you need more. Moonshot’s own recommendation is 64 or more accelerators — eight nodes of eight H100s — for production workloads. A single H200 node is the realistic sweet spot if you want to run this without distributing across multiple machines. There is no consumer path. No gaming GPU gets you there. A MacBook does not help. If your team does not already operate data-center-grade GPU infrastructure, you are using the API, and that is the correct decision for now. Community GGUF quantizations will arrive within weeks and will dramatically lower the floor — but production-stable local inference is realistically a Q4 2026 story for most teams. Setting Up vLLM For those with qualifying hardware, the download and serve flow is straightforward. Accept the license on the Hugging Face model page https://huggingface.co/moonshotai/Kimi-K3-MXFP4 first — the repository is gated. huggingface-cli download moonshotai/Kimi-K3-MXFP4 \ --local-dir /data/models/kimi-k3-mxfp4 \ --local-dir-use-symlinks False \ --resume-download On a 10 Gbps connection, expect 8 to 10 minutes. On 1 Gbps, roughly 80 to 90 minutes. Once downloaded: pip install "vllm =0.7.0" python -m vllm.entrypoints.openai.api server \ --model /data/models/kimi-k3-mxfp4 \ --trust-remote-code \ --quantization mxfp4 \ --max-model-len 131072 \ --tensor-parallel-size 8 \ --gpu-memory-utilization 0.92 \ --port 8000 The --trust-remote-code flag is not optional — KDA runs custom CUDA kernels. Set --max-model-len to 131072 for a 128K context window; the full 1M requires more VRAM than most single-node setups can provide. The resulting server exposes a standard OpenAI-compatible /v1/chat/completions endpoint, compatible with the OpenAI Python SDK https://platform.openai.com/docs/api-reference , LangChain, and LlamaIndex. API vs Self-Host: The Economics Moonshot’s API prices K3 at $3 per million input tokens and $15 per million output tokens. Cached input drops to $0.30 per million — and on coding workloads with stable prompt prefixes, cache hit rates can reach 90% or higher. At that cache rate, effective input cost collapses to roughly $0.30/M. The output price is fixed regardless. Self-hosting break-even sits at approximately 10 billion tokens per month, assuming you already have GPU capacity sitting idle. A purpose-built 8x H100 node runs roughly $370,000 in hardware plus $1,400 per month in electricity. That capital cost alone requires enormous token volume to justify. One more factor: third-party inference providers can now host K3 on the public weights. Expect prices to fall below Moonshot’s list rates as competition arrives — exactly what happened with K2 and DeepSeek V4. The API economics improve without you doing anything. The Argument That Overrides the Math For most developers, the API wins on pure economics. There is one case where the calculation changes entirely: regulated data. Moonshot AI is Beijing-based. Its API servers sit under Chinese jurisdiction, subject to the Personal Information Protection Law PIPL , the Data Security Law DSL , and — since January 2026 amendments — a Cybersecurity Law that now includes AI-specific provisions https://techjacksolutions.com/ai-governance-china/data-triad/ . Sending customer data through a Chinese-jurisdiction API is a legal and compliance question, not just a technical one. Legal, healthcare, and finance teams handling sensitive client or patient data face a different decision. Self-hosting the weights on your own infrastructure eliminates the jurisdictional question. Your data stays under your governance framework. That is not a performance argument; it is a compliance argument, and it holds regardless of where the break-even math lands. Who Should Self-Host Self-host if you operate existing GPU infrastructure with excess capacity, work in a regulated industry with data residency requirements, exceed 10 billion tokens per month, or have ML engineering resources to manage distributed inference. Use the API if your token volume is below that threshold, you have no existing GPU infrastructure, you work in an unregulated domain, or you need production reliability without infrastructure overhead. The $0.30 cached input rate already makes the API competitive for coding workloads with repeated context. Watch the third-party provider market over the next two to four weeks. When inference providers deploy K3 on these weights, prices will drop and the API case strengthens further. And when the community GGUF quantizations arrive, the self-hosting floor will fall to something a smaller cluster can handle. For now, the weights are live — the question was never whether Moonshot would ship them. It is whether your infrastructure justifies the detour.