MoE workstation744B total, ~40B active per token (MoE: 256 routed experts, 8 active + 1 shared). Uses MLA + DeepSeek Sparse Attention (IndexShare) for a solid 1M context. Q4_K_M (~410GB) fits a 512GB Mac Studio M4 Ultra or 4x DGX Spark; Q2_K (~240GB) fits 2-3 DGX Sparks. Strongest open-source coding/reasoning model as of June 2026.
- 744.0B
- 1000k
- mit
- Jun 2026
1 person run this as their daily driver.
Scores #
Run it locally #
Per-quant memory needs and a static "can you run it?" reference - no rig entry required
Can you run it? - reference rigs
| Rig | Q2_K | Q3_K_M | Q4_K_M | Q5_K_M | BF16 |
|---|---|---|---|---|---|
| NVIDIA Jetson Orin NX 16GB | |||||
no -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudno -> cloudFit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.
How can a 24GB GPU run a 744B model? It does not load the model into VRAM. The quantized weights (e.g. ~410GB at Q4) sit in system RAM; the GPU only holds the small shared attention and router tensors and accelerates prompt processing. Because GLM 5.2 is a Mixture-of-Experts model, each token activates only ~40B of its 744B params, so llama.cpp streams just those active experts from system RAM to the GPU each token (the -cmoe
offload path).
That makes decode speed bound by system-RAM bandwidth, not GPU bandwidth - single digits on DDR4, which is why these rigs show 3-8 t/s even though they “fit.” A bigger GPU (e.g. 2x 3090) keeps more experts resident on-card and raises tok/s; a smaller GPU still runs it but pays the bandwidth tax. A 744B dense model could not run this way - only MoE’s small-active-params trick makes it possible.
Aggressive quants (1-2 bit) trade accuracy for size - roughly 17% accuracy loss at 2-bit vs full precision, and real long-context work often needs Q5 or Q6 even when lower quants “fit.”
Formula estimates here are conservative; real tuned setups can exceed them (one HN user reports ~6 tok/s on a 512GB DDR4 + 2x 3090 rig).
Download options #
Or run it in the cloud #
Live per-provider pricing, throughput and uptime - refreshed about 10 hours ago via OpenRouter. Click a column to sort.
| Provider | Type | Input $/M | Output $/M | Cache $/M | Tok/s | Latency | Uptime | Value |
|---|---|---|---|---|---|---|---|---|
| WandB | ||||||||
| API | 1.39 | 4.40 | 0.260 | - | - | 100.00% | best uptime | |
| API | 1.40 | 4.40 | - | - | - | - | ||
| API | 3.00 | 10.25 | 0.500 | - | - | 100.00% | ||
| Alibaba | ||||||||
| API | 1.32 | 4.14 | 0.264 | - | - | 99.97% | ||
| Novita | ||||||||
| API | 0.98 | 3.08 | 0.182 | - | - | 99.95% | ||
| Fireworks | ||||||||
| API | 2.10 | 6.60 | 0.210 | - | - | 99.94% | ||
| Together | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 99.88% | ||
| StreamLake | ||||||||
| API | 1.12 | 3.52 | 0.208 | - | - | 99.87% | ||
| AtlasCloud | ||||||||
| API | 1.26 | 3.96 | 0.234 | - | - | 99.84% | ||
| API | 1.40 | 4.40 | 0.260 | - | - | 99.78% | ||
| Baidu | ||||||||
| API | 0.97 | 3.07 | 0.181 | - | - | 99.73% | ||
| AkashML | ||||||||
| API | 1.30 | 4.40 | 0.180 | - | - | 99.72% | ||
| Decart | ||||||||
| API | 1.20 | 4.20 | 0.200 | - | - | 99.71% | ||
| SiliconFlow | ||||||||
| API | 1.30 | 4.09 | 0.260 | - | - | 99.64% | ||
| Ambient | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 99.51% | ||
| Z.AI | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 99.47% | ||
| Io Net | ||||||||
| API | 1.60 | 4.99 | 0.799 | - | - | 99.25% | ||
| DeepInfra | ||||||||
| API | 0.93 | 3.00 | 0.180 | - | - | 98.84% | ||
| Parasail | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 98.81% | ||
| Venice | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 98.73% | ||
| Cloudflare | ||||||||
| API | 1.40 | 4.40 | 0.260 | - | - | 98.47% | ||
| Inceptron | ||||||||
| API | 0.90 | 3.08 | 0.180 | - | - | 98.04% | cheapest | |
| DigitalOcean | ||||||||
| API | 1.05 | 4.40 | 0.210 | - | - | 97.56% | ||
| API | 0.98 | 3.08 | 0.182 | - | - | 97.15% | ||
| API | 1.40 | 4.40 | 0.700 | - | - | 95.48% | ||
| Morph | ||||||||
| API | 1.10 | 4.10 | 0.180 | - | - | 95.32% | ||
| DekaLLM | ||||||||
| risky | ||||||||
| API | 0.94 | 3.00 | 0.180 | - | - | 92.01% |
| Sub | - | - | - | - | - | - | $10.00/mo Coding Plan Lite | |
| Sub | - | - | - | - | - | - | $20.00/mo Pro | |
| Sub | - | - | - | - | - | - | $30.00/mo Coding Plan Pro | |
| Sub | - | - | - | - | - | - | $80.00/mo Coding Plan Max | |
| Sub | - | - | - | - | - | - | $100.00/mo Max |
Default order: throughput among 95%+ uptime providers, then latency; subscriptions last. Sort by any column. Subscription rows show $/mo in the Value column - per-token columns are "-". Affiliate links are marked sponsored / nofollow. Confirm current pricing on the provider's site before committing.
Detailed API pricing page + JSON endpoint →
Inference cost over time #
Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.