Member-only story
Two weeks ago the largest open-weight model in existence had one trillion parameters.
Then Moonshot AI shipped Kimi K3.
2.8 trillion parameters. Released July 16, 2026. Open weights. 1 million token context window.
#1 on LMArena’s Frontend Code Arena, beating Claude Fable 5 on the benchmark developers actually care about.
And it runs inference with the computational cost of a 50 billion parameter model.
That last number is the one worth understanding. Not because it is impressive — though it is — but because the mathematics behind it is a specific set of decisions, each one solving a specific problem, and each one documented in Moonshot’s technical report.
This is not “China’s ChatGPT moment” coverage. This is the four engineering decisions that made K3 possible, derived step by step.
The Problem With Being Large #
Scale works. Every lab agrees on this. More parameters, more capability, smoother loss curves, better benchmarks.
The problem: a 2.8 trillion parameter dense model is not deployable. It is not even runnable.
A dense transformer with 2.8T parameters in float16: 2.8 × 1⁰¹² × 2 bytes = 5.6 terabytes.