Moonshot AI released the weights for Kimi K3 on July 27 — 2.8 trillion parameters, a 1-million-token context window, and benchmark scores that put it third in the world behind only closed frontier models. The open-source AI community declared it a watershed moment. The breathless headlines wrote themselves. What the headlines skipped: you need at least eight H100 GPUs before you can load the model. For most developers, “open weights” and “open access” are not the same thing.
What Kimi K3 Is #
Kimi K3 is a Mixture-of-Experts model with 896 experts and 16 active per token — meaning 1.8% of the network fires for any given inference. That extreme sparsity is how Moonshot keeps serving costs tractable despite the 2.8-trillion-parameter count. Two architectural innovations power it: Kimi Delta Attention (KDA), a hybrid linear attention mechanism that delivers 6.3x faster decoding at long context, and Attention Residuals, which add 25% training efficiency at roughly 2% extra compute cost.
The weights ship in native MXFP4 — not as a post-hoc quantization but as the training format. The model is commercially usable under a Modified MIT license and available on Hugging Face under the moonshotai organization. The full weight repository runs to about 1.56 TB; the native MXFP4 checkpoint alone is roughly 594 GB.
The Hardware Reality #
Here is where the conversation shifts. The minimum viable setup to load Kimi K3 is eight H100 80GB GPUs, giving you 540 GB or more of combined VRAM. That is before you think about production serving. Moonshot recommends 18 to 24 H100s for stable production with Q4 quantization, and their reference architecture targets 64-accelerator supernodes for high-throughput deployments.
The KDA attention kernels depend on a pre-release version of FlashInfer, which means a standard pip install of vLLM will not work. The practical day-one path is the official Docker image:
docker pull vllm/vllm-openai:kimi-k3
Moonshot shipped vLLM day-0 support simultaneously with the weight release. Once you have the image, serving looks like this:
vllm serve moonshotai/Kimi-K3 \
--tensor-parallel-size 8 \
--trust-remote-code \
--load-format fastsafetensors \
--enable-prefix-caching \
--enable-auto-tool-choice \
--tool-call-parser kimi_k3 \
--reasoning
vLLM provides two MoE backends: TRT-LLM-Gen for tensor-parallel configurations and MegaMoE for disaggregated expert-parallel setups. For most teams starting out, tensor-parallel with --tensor-parallel-size 8
is the right starting point. An AWS deployment guide covers cloud infrastructure options, and RunPod, Baseten, and DigitalOcean have each published their own recipes.
The Benchmark Picture #
The performance numbers justify the infrastructure investment — if you have the infrastructure. On GDPval-AA v2, Kimi K3 earned an Elo of 1668, placing it third globally behind Claude Fable 5 Max (1815) and GPT-5.6 Sol Max (1747). On Terminal-Bench 2.1 it scores 88.3 and on FrontierSWE 81.2. One notable regression: SWE-bench Verified came in at 76.8%, down from K2.6’s 80.2%. Worth tracking, but it does not undercut the overall picture — this is a legitimate frontier model in open weights.
API vs Self-Hosting: Run the Numbers First #
The Kimi API charges $3 per million input tokens (cache miss), $0.30 per million cache hits, and $15 per million output tokens. Self-hosting the economics are sharper than most teams realize. An on-demand 8-GPU B200 node runs about $32,000 a month in cloud compute. The break-even point against the $15-per-million output API sits above 2 billion output tokens per month. For 16 H200s with operational overhead, that threshold climbs to roughly 38.8 billion tokens monthly.
For almost every product team, the API wins on cost. Self-hosting makes concrete sense in three scenarios:
Data residency requirements. Healthcare, finance, and public-sector organizations that cannot send data to infrastructure operating under Chinese law have a genuine compliance reason to self-host. The April 2026 cross-user data incident that Moonshot never publicly addressed makes this concern more concrete.Existing GPU capacity. If your organization already has idle H100 or H200 clusters, the marginal cost of running K3 on owned hardware can undercut the API significantly.Inference resale. If you are building a managed AI platform and serving K3 to downstream customers at volume, the economics shift once you clear the break-even threshold. Review the Modified MIT license reseller clause before committing.
The Verdict #
Kimi K3 is a genuine achievement. Moonshot has put frontier-class capability under an open license, and the simultaneous vLLM support is exactly what the open-source ecosystem needed. For researchers and organizations with GPU clusters, this is the most capable open-weight model available and worth serious evaluation. For the rest of the developer community, the right move is the Kimi API or a managed inference provider now offering K3 endpoints. Read Moonshot’s technical breakdown for the full architecture details, and check the hardware breakdown at Northflank before committing to self-hosting. The weights are open. The access is not.