# Bonsai 27B Runs on iPhone: The On-Device AI Tradeoffs > Source: > Published: 2026-07-15 01:10:49+00:00 PrismML dropped [Bonsai 27B](https://prismml.com/news/prismml-releases-bonsai-27b) yesterday — a 27-billion parameter AI model squeezed into 3.9GB that runs natively on an iPhone 17 Pro. A standard FP16 27B model requires roughly 54GB of memory. That is a 14.2x compression ratio, and it is not a trick. Whether the resulting model is actually useful for what you are building is a more complicated answer. ## How You Fit 27 Billion Parameters into 3.9GB Standard large language models store weights as 16-bit floating point numbers. Bonsai 27B uses 1-bit binary quantization: each weight becomes either -1 or +1. To recover the precision lost by that reduction, a single shared FP16 scale factor covers every group of 128 weights — enough to guide the model without blowing up the file size. The result is 1.125 effective bits per weight instead of 16. The critical difference from most quantized models is that PrismML trained Bonsai 27B natively in 1-bit from the start, applying the constraint across embeddings, attention layers, and MLPs end-to-end. Conventional post-training quantization tends to collapse on hard reasoning tasks — math, live code benchmarks, agentic workflows. PrismML’s argument is that native training degrades more gracefully, and the benchmarks mostly support that claim. Math benchmarks (AIME) drop from 95.3 on the full-precision Qwen3.6-27B base to 91.7 on 1-bit Bonsai — a 4.8% reduction. Coding falls from 88.7 to 81.9. However, the ternary variant (5.9GB, weights at -1/0/+1) retains 94.6% of baseline performance. Both models support a 262K token context window and multimodal image input. ## What You Can Access Today Bonsai 27B is available under Apache 2.0 — free, with commercial use permitted. The model runs via MLX on Apple Silicon and iOS devices, via llama.cpp in GGUF format for cross-platform use, and through a free developer preview API hosted by Together AI for teams not ready to run inference locally. Model weights are on [Hugging Face under Apache 2.0](https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit). On the iPhone 17 Pro, the 1-bit variant runs at 11 tokens per second. On an M5 Max, that rises to 87 tok/s. On an RTX 5090, 163 tok/s. The phone speed is real but modest — below the 15-20 tok/s threshold that feels conversationally natural. Background processing, document analysis, and offline batch tasks fit well here. Waiting for a live chat response at 11 tok/s gets tedious. ## The Caveats That Benchmarks Do Not Show The [Hacker News thread landed 408 points](https://news.ycombinator.com/item?id=48910545) and surfaced things the marketing copy omits. Tool-calling performance is reportedly worse than the base model — which matters considerably if you are building agentic workflows where the model must reliably invoke tools. Multiple developers flagged this, and one was direct: being good at coding benchmarks is beside the point if tool-calling regresses. There are also reports of reasoning loops on specific tasks, particularly SQL queries. Initial toolchain support was rough — LM Studio and Unsloth did not support the format at launch, requiring custom llama.cpp forks. PrismML acknowledged the issues and directed users to Discord. Moreover, one independent evaluation found benchmark irregularities on GSM8K, suggesting the degradation on certain mathematical reasoning tasks may exceed what the headline numbers indicate. Vision performance also came in below expectations given the multimodal claims. None of this disqualifies the model. These are the normal friction points of a fresh release. But they mean you should validate your specific use case before committing Bonsai 27B to a production agentic pipeline. The [technical breakdown at MarkTechPost](https://www.marktechpost.com/2026/07/14/prismml-releases-bonsai-27b-1-bit-and-ternary-builds-of-qwen3-6-27b-that-run-on-laptops-and-phones/) covers the benchmark methodology in detail. ## When On-Device 27B Actually Makes Sense The strong case for Bonsai 27B is not speed — it is data residency. For healthcare applications, legal tooling, or any context where code and documents cannot leave the device, on-device inference is not a feature. It is the compliance path. Bonsai 27B gives you 27B-class reasoning for those workloads without a cloud dependency or a third-party processor in the inference path. This is worth more than any benchmark number in GDPR and HIPAA-regulated contexts. The 262K token context is also significant. Most on-device models top out at 8K-32K context, which limits what you can process in a single pass. At 262K, you can feed an entire codebase or document set into a single prompt — on the device, offline, with no egress cost. For developers already using [local AI inference for large models](https://byteiota.com/mesh-llm-lets-you-run-235b-ai-models-without-the-cloud/) or [privacy-first local AI in editors](https://byteiota.com/zed-1-10-local-ai-llama-cpp/), Bonsai 27B slots naturally into the same workflow. Apple is reportedly in early discussions evaluating PrismML’s models — nothing confirmed, and the conversations are described as preliminary. Even so, the direction is clear: the industry is pushing toward 27B-class capability at phone-native sizes. Bonsai 27B is the first model to actually deliver that, with known trade-offs that are worth testing before you ship. The [9to5Mac coverage](https://9to5mac.com/2026/07/14/prismml-releases-bonsai-27b-claiming-first-major-ai-model-of-its-size-fit-for-iphone/) covers the iPhone memory budget constraints in detail.