How a 20B Model Hits 120 tok/s on an iPhone

DeepGrove's Maple-Preview, a 20B-parameter mixture-of-experts reasoning model with ternary weights, achieves 120 tokens per second on an iPhone and 218 tok/s on a base M4 Mac mini, according to the company. The model, which has 1B active parameters per token and a 5.31 GB checkpoint under an MIT license, is claimed to be 5–16× faster than efficient competitors like Gemma 4, Qwen3.5, and gpt-oss. Its architecture combines native ternary training with MoE sparsity and interleaved sliding-window attention, making the performance claims physically plausible.

AI https://sourcefeed.dev/c/ai Article How a 20B Model Hits 120 tok/s on an iPhone DeepGrove's Maple-Preview stacks native ternary training on MoE sparsity, and the arithmetic largely holds up. Priya Nair https://sourcefeed.dev/u/priya nair A 20-billion-parameter model generating 120 tokens per second on a phone would have sounded like a typo two years ago. That's the headline claim behind Maple-Preview https://huggingface.co/deepgrove/maple-preview , a new open-weights reasoning model from DeepGrove https://deepgrove.ai that hit the Hacker News front page this week. And unlike most "frontier model on a potato" demos, this one mostly survives scrutiny — because it's not one trick, it's two tricks that multiply. Maple-Preview is a 20B-parameter mixture-of-experts model with roughly 1B parameters active per token 20B-A1B , 24 layers, 256 experts with 8 active, and a 131k context window. The weights are ternary — every weight is -1, 0, or +1 — and the whole checkpoint is 5.31 GB under an MIT license. DeepGrove claims 218 tok/s on a base M4 Mac mini and the 120 tok/s iPhone figure from the announcement, and says the model is 5–16× faster than efficient competitors like Gemma 4, Qwen3.5, and gpt-oss. Why the number is plausible On-device decoding is memory-bandwidth-bound: every generated token requires streaming the active weights through the memory bus. That's where the two tricks compound. MoE sparsity means only ~1B of the 20B parameters are touched per token. Ternary packing means those parameters cost about two bits each instead of sixteen. Multiply them and each decode step reads roughly 300 MB of weights — against the 50–120 GB/s of memory bandwidth in a modern iPhone or base M4. That puts the theoretical ceiling in the low hundreds of tokens per second, which makes both the 218 tok/s Mac number and the 120 tok/s phone number physically consistent rather than magical. The checkpoint size passes the same smell test. A 20B model shipped in BF16 would be ~40 GB; 5.31 GB is exactly what you'd expect from genuinely packed ternary MoE weights plus higher-precision attention and embeddings. This isn't a "ternary" model that secretly ships full-precision tensors. The architecture shows the same discipline on the other bandwidth hog, the KV cache. Maple interleaves sliding-window attention 512-token window with global attention at a 3:1 ratio, Gemma-style, so three-quarters of the layers have a KV cache that stops growing after 512 tokens. Without that, a 131k context on a phone would be dead on arrival regardless of how small the weights are. Native ternary finally cashes a real check The interesting part isn't quantization — it's when the quantization happens. Microsoft's BitNet https://github.com/microsoft/BitNet line argued back in 2024 that ternary models should be trained that way from scratch, and proved it at 2B parameters with b1.58-2B4T. DeepGrove's own earlier Bonsai https://github.com/deepgrove-ai/Bonsai was a 500M-parameter native-ternary proof of concept trained on under 5B tokens. What nobody had shipped was native ternary at a scale where the model is actually useful. Maple-Preview, if its claims hold, is that model — and bolting ternary onto a fine-grained MoE the DeepSeek-style many-small-experts recipe is the move that gets capacity back without paying for it at decode time. The contrast with post-hoc compression is instructive. PrismML's Ternary-Bonsai-27B — confusingly named, but an unrelated post-training ternary quantization of Qwen3.6-27B — manages about 11 tok/s on an iPhone 17 Pro, and independent analysis found its tool-calling accuracy collapsed from 80 to 66 at 1-bit while math barely degraded. Squeezing a trained full-precision model into two bits destroys exactly the fine-grained discrimination that post-training taught it. Training in ternary from day one sidesteps that failure mode, at least in principle. The gap between 11 tok/s and a claimed 120 tok/s for models in the same weight class is the strongest single argument that native-ternary-plus-MoE is a different category, not a better compressor. Where the claims get soft Now the caveats, because there are real ones. The benchmark story — Pareto-frontier results on LiveCodeBench v6, AIME 2026, HMMT 2026, and GPQA Diamond — is published as an image on the model card with no eval harness, and HN commenters noted the comparisons target Qwen3.5 rather than the newer 3.6. The iPhone figure comes from the team's own announcement, doesn't specify which iPhone, and hasn't been independently reproduced yet. Coverage so far, including Latent.Space's AINews, relays DeepGrove's numbers rather than confirming them. The capability critiques are more predictable but worth taking seriously. Early users caught it confidently inventing etymology and hallucinating a nonexistent Seattle observation deck. Of course it did: 5 GB of weights can only memorize so much world knowledge, whatever the reasoning benchmarks say. And DeepGrove is upfront that this preview received minimal agentic post-training and only small-scale RL. This is a reasoning engine, not an encyclopedia and not yet an agent. What to actually do with it If you're building on-device or local-first, the adoption path is concrete. The checkpoint runs today through Transformers, vLLM, and SGLang, and there's already a community WebGPU demo — a weekend on an M-series Mac tells you whether the reasoning quality is real for your workload. Treat the model as a compute core, not a knowledge store: pair it with retrieval for facts and route tool-calling to something proven until the non-preview release ships with real agentic training. Phone deployment is the frontier, not the default — 5.31 GB of weights plus KV cache is a tight fit inside iOS app memory limits, and the mobile inference stack that produced the 120 tok/s demo isn't in your hands yet. The strategic read: this squeezes dense small models hardest. The default local stack — a dense 4B–9B model at 4-bit — now competes with something that has 4× the capacity, comparable memory, and multiples of the decode speed. Apple already ships its on-device foundation model at 2 bits per weight; the direction of travel is unanimous, and cheap-tier cloud inference APIs should be nervous about every workload that fits in 6 GB. My verdict: genuine shift, preview-grade execution. The physics checks out, the lineage from BitNet through Bonsai is real research rather than launch-week theater, and the MoE-times-ternary compounding is an insight others will copy within months. But hold the confetti until third parties reproduce the throughput and the benchmarks escape the PNG. Watch for independent evals and a GGUF-style mobile runtime — those two things turn this from an impressive demo into the new baseline for local inference. Sources & further reading - Show HN: Maple-Preview - Ternary 20B MoE running at 120 tok/s on a iPhone https://news.ycombinator.com/item?id=49173984 — news.ycombinator.com - deepgrove/maple-preview model card https://huggingface.co/deepgrove/maple-preview — huggingface.co - AINews: Megakernels are so dead and so back https://www.latent.space/p/ainews-megakernels-are-so-dead-and — latent.space - Bonsai: a 500M native-ternary language model https://github.com/deepgrove-ai/Bonsai — github.com - Bonsai 27B: A 27B Model on Your Phone - and the One Benchmark That Collapses https://rohitraj.tech/en/notes/bonsai-27b-ternary-quantization-guide-2026 — rohitraj.tech Priya Nair https://sourcefeed.dev/u/priya nair · AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. Discussion 0 No comments yet Be the first to weigh in.