{"slug": "colibri-run-a-744b-moe-model-locally-on-25gb-ram", "title": "Colibri: Run a 744B MoE Model Locally on 25GB RAM", "summary": "Colibri is a new inference architecture that runs GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, locally on 25GB of RAM by streaming expert weights from disk, since the model activates only about 40 billion parameters per token. On a typical CPU-only machine the setup generates roughly 0.05 tokens per second, a speed the architecture's backers frame as acceptable for the right workloads rather than a general-purpose replacement for GPU inference.", "body_md": "The headline writes itself: run a 744-billion-parameter model on a laptop. The fine print is less spectacular — on a typical CPU-only machine, you’ll clock roughly 0.05 tokens per second. But that framing misses the point. Colibri isn’t a trick. It’s a genuinely novel inference architecture that changes the economics of running frontier-scale models locally, and for the right workloads, it works today. Why Disk Streaming Works for MoE Models Colibri’s core insight is deceptively simple: Mixture-of-Experts models are sparse. GLM-5.2 has 744 billion parameters on paper, but only activates roughly 40 billion per token. For each layer, just 8 […]\n\nThe post", "url": "https://wpnews.pro/news/colibri-run-a-744b-moe-model-locally-on-25gb-ram", "canonical_source": "https://byteiota.com/colibri-744b-moe-local-inference/", "published_at": "2026-09-18 07:09:14+00:00", "updated_at": "2026-09-18 07:23:52.234204+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-research"], "entities": ["Colibri", "GLM-5.2"], "alternates": {"html": "https://wpnews.pro/news/colibri-run-a-744b-moe-model-locally-on-25gb-ram", "markdown": "https://wpnews.pro/news/colibri-run-a-744b-moe-model-locally-on-25gb-ram.md", "text": "https://wpnews.pro/news/colibri-run-a-744b-moe-model-locally-on-25gb-ram.txt", "jsonld": "https://wpnews.pro/news/colibri-run-a-744b-moe-model-locally-on-25gb-ram.jsonld"}}