{"slug": "maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations", "title": "Maple-Preview: 120 tok/s 20B MoE on iPhone Defies Expectations", "summary": "Maple-Preview, a 20B parameter Mixture-of-Experts transformer quantized to ternary weights, achieves 120 tokens per second on an iPhone, according to a developer preview. The model, which runs on Apple's Neural Engine via a custom backend, trades accuracy for speed and is a proof-of-concept with limited context length. The full Maple stack aims to ship local LLMs that feel fast, potentially enabling on-device coding assistants and knowledge bases.", "body_md": "# Maple-Preview: 120 tok/s 20B MoE on iPhone Defies Expectations\n\n**ternary** 20B parameter Mixture-of-Experts model — most of those weights are literally baked into 2-bit values, which is why an iPhone can chew through 120 tokens per second without melting.\n\n## What's Real Here\n\nMaple-Preview isn't some benchmark stunt. It's a trimmed-down demo of what the full Maple stack aims to ship: a 20B MoE transformer quantized down to ternary activations and weights, then compiled with a custom backend that leans hard on Apple's Neural Engine. The trick is that only a fraction of experts fire per token, and the ternary representation cuts memory bandwidth demands so drastically that the ANE doesn't become the bottleneck.\n\n## The Caveats (Because I'm Skeptical Too)\n\nThe model is, unsurprisingly, **dumb as a rock** compared to its dense counterparts. You're trading accuracy for sheer speed — think basic instruction following, not nuanced reasoning. The demo also caps sequence length aggressively, so it's fine for quick prompts but falls apart on anything requiring long context. This is very much a **proof-of-concept**: here's the raw performance, now go build something useful on top.\n\n## Setup Notes for Hackers\n\nThe repo ships with a Swift package wrapper around a quantized Core ML model. Running it locally requires:\n\n```\n# iOS 17+ required for full ANE support\nxcodebuild -project MaplePreview.xcodeproj -scheme MaplePreview -destination 'platform=iOS Simulator,name=iPhone 15'\n```\n\nMemory footprint sits around 400MB once loaded, which is honestly impressive given the parameter count. The tokenizer is a stripped BPE variant, and the MoE routing is handled by a lightweight learned gate that runs entirely on-device.\n\n## Why This Matters\n\nMost \"LLM on mobile\" demos either punt to the cloud or squeeze a tiny 1B model onto device. Maple-Preview flips that script — it keeps the parameter count high but slashes the computational cost through quantization and expert sparsity. If the full release holds up, we might finally get local LLMs that feel **fast** instead of tolerating sluggish inference.\n\nThe real test will be whether the accuracy cliff is survivable for niche use cases like coding assistants or local knowledge bases. For now, the 120 tok/s number is legit, and that alone makes this worth a look.\n\n[OpenAI Just Dragged Apple Into the Dirt Over Those Trade Secret 7h ago](/en/news/5048/)\n\n[Maple-Preview: 20B MoE Hits 120 tok/s on iPhone 10h ago](/en/news/5019/)\n\n[Apple's latest legal filing reveals a concerning pattern 18h ago](/en/news/4966/)\n\n[OpenAI Fires Back at Apple's Trade Secret Lawsuit with Public 18h ago](/en/news/4958/)\n\n[TurboFieldfare: Running Gemma 4 26B on 8GB Macs 6d ago](/en/news/4258/)\n\n[Next AI Search Is Weaponizing Reddit Comments Into Stealth Ads →](/en/news/5081/)", "url": "https://wpnews.pro/news/maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations", "canonical_source": "https://promptcube3.com/en/news/5085/", "published_at": "2026-08-05 09:17:11+00:00", "updated_at": "2026-08-05 10:54:27.811330+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-infrastructure"], "entities": ["Apple", "Maple-Preview", "Maple", "Apple Neural Engine", "Core ML", "Swift"], "alternates": {"html": "https://wpnews.pro/news/maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations", "markdown": "https://wpnews.pro/news/maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations.md", "text": "https://wpnews.pro/news/maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations.txt", "jsonld": "https://wpnews.pro/news/maple-preview-120-tok-s-20b-moe-on-iphone-defies-expectations.jsonld"}}