Colibri: Run a 744B MoE Model Locally on 25GB RAM

Colibri is a new inference architecture that runs GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, locally on 25GB of RAM by streaming expert weights from disk, since the model activates only about 40 billion parameters per token. On a typical CPU-only machine the setup generates roughly 0.05 tokens per second, a speed the architecture's backers frame as acceptable for the right workloads rather than a general-purpose replacement for GPU inference.

The headline writes itself: run a 744-billion-parameter model on a laptop. The fine print is less spectacular — on a typical CPU-only machine, you’ll clock roughly 0.05 tokens per second. But that framing misses the point. Colibri isn’t a trick. It’s a genuinely novel inference architecture that changes the economics of running frontier-scale models locally, and for the right workloads, it works today. Why Disk Streaming Works for MoE Models Colibri’s core insight is deceptively simple: Mixture-of-Experts models are sparse. GLM-5.2 has 744 billion parameters on paper, but only activates roughly 40 billion per token. For each layer, just 8 … The post