cd /news/large-language-models/qwen3-8-27b-on-the-iphone-18-neural-… · home › topics › large-language-models › article
[ARTICLE · art-152035] src=twitter.com ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)

Anemll reported running the Qwen 3.8-27B model on the iPhone 18's Apple A20 Neural Engine at 15–60 tokens per second, peaking at 70 tok/s, using a base model requantized from PrismML's Bonsai-2 at 1.585 bpw down to 1.5 bpw (6-bit vector quantization per 4 weights). Anemll confirmed a 24K context window on the iPhone with both K/V caches in int8 while the delta net stays FP16, and added a dflash-2 draft model at roughly 2.8 bpw with mixed 2–4 vector quantization and a small drop in acceptance rate. Anemll wrote that "Apple Neural Engine on A20 is fast.

read1 min views3 publishedOct 17, 2026
Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)
Image: source

Anemll on X: "Confirmed 24K context on iPhone with both K/V on int8. Delta net stays FP16. Decoding also peaks at 70 tok/s with easy to predict context. Apple Neural Engine on A20 is fast 🚀"

Confirmed 24K context on iPhone with both K/V on int8. Delta net stays FP16. Decoding also peaks at 70 tok/s with easy to predict context. Apple Neural Engine on A20 is fast 🚀

Qwen 3.8- 27B on iPhone 18 Apple Neural Engine. Added draft model , dflash-2 at ~ 2.8 bpw ( mixed 2-4 vector quantization with small drop in acceptance rate) Base is @PrismML Bonsai-2, 1.585 bpw requantized to 1.5 bpw ( VQ 6 bits per 4 weights) Decoding speed is 15-60 tps.

Confirmed 24K context on iPhone with both K/V on int8. Delta net stays FP16. Decoding also peaks at 70 tok/s with easy to predict context. Apple Neural Engine on A20 is fast 🚀

Qwen 3.8- 27B on iPhone 18 Apple Neural Engine. Added draft model , dflash-2 at ~ 2.8 bpw ( mixed 2-4 vector quantization with small drop in acceptance rate) Base is @PrismML Bonsai-2, 1.585 bpw requantized to 1.5 bpw ( VQ 6 bits per 4 weights) Decoding speed is 15-60 tps.

source & further reading
twitter.com — original article
── more in #large-language-models 4 stories · sorted by recency
── more on @anemll 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/qwen3-8-27b-on-the-i…] indexed:0 read:1min 2026-10-17 · —