Colibri: Run a 744B MoE Model Locally on 25GB RAM

The headline writes itself: run a 744-billion-parameter model on a laptop. The fine print is less spectacular — on a typical CPU-only machine, you’ll clock roughly 0.05 tokens per second. But that framing misses the point. Colibri isn’t a trick. It’s a genuinely novel inference architecture that changes the economics of running frontier-scale models locally, and for the right workloads, it works today. Why Disk Streaming Works for MoE Models Colibri’s core insight is deceptively simple: Mixture-of-Experts models are sparse. GLM-5.2 has 744 billion parameters on paper, but only activates roughly 40 billion per token. For each layer, just 8 […]

The post