Did the Device Watershed Just Arrive?

PrismML released a ternary-weight version of Qwen3.8-27B under its Bonsai family on Tuesday, cutting the model from more than 50 GB of weights at full precision to just under 6 GB, which lets it run on an 8 GB PC graphics card from 2019 at 17 tokens per second. PrismML claims the ternary model retains 98.2% of the original's benchmark score, though the author notes agentic work and tool use give up the most, and PrismML measures energy use at roughly half a milliwatt-hour per token. The compression puts a high-quality reasoning model within reach of 16 GB Android phones such as the Pixel 9 Pro and Galaxy S26 Ultra, while the 12 GB in the latest iPhones is not quite enough room.

Did the Device Watershed Just Arrive? It's now clear exactly how much Qwen3.8-27B https://thewatershed.markpesce.com/ai-comes-home/ changed things. Once people got the sense that a small model could be as powerful as a frontier model, they threw an enormous amount of effort into fitting that model into anything . I now have various versions of Qwen running on both of my Macintoshes. My PCs, with their fixed and modest allotments of VRAM, had been left out. Until today. For the last three years I've been deeply interested in work out of Microsoft Research in China on ternary weights https://www.microsoft.com/en-us/research/publication/bitnet-scaling-1-bit-transformers-for-large-language-models/?ref=thewatershed.markpesce.com . Ternary weights break every parameter down to one of three values: -1, 0 or 1. In theory, three values squeeze into 1.58 bits. In practice it never works out quite that tightly, but the result is the same: most of the power of a model, in a fraction of the space. A group named PrismML has been releasing a family of ternary models under the name Bonsai. I tried one a couple of months ago. It wasn't great. Yesterday they released https://prismml.com/news/bonsai-2-27b?ref=thewatershed.markpesce.com a ternary version of Qwen3.8-27B. Instead of more than fifty gigabytes of weights at full precision, it comes in at just under six. Which means it fits on a common PC graphics card. Which was the very first thing I did with it. A PC that could not run Qwen at all is now running a version of Qwen as fast as my MacBook Pro does - 17 tokens a second, on a 2019 card with 8 GB VRAM that the market discarded two generations ago. What remains to be tested is whether the quality is still "good enough". PrismML says it retains 98.2% of the original's benchmark score. That's their number, on their suite. The row that gives up the most is the one I care about: agentic work and tool use, where small errors compound across a long task. Bonsai doesn't need to be as good as Qwen, only close, because Qwen is very good. But close on a benchmark and close across eight hours of unattended work are different. It's the second one that really counts. If it holds, a high-quality reasoning model now fits in 8 GB. And that changes everything. There's a range of smartphones that can run this model today. All of them Android. The Pixel 9 Pro has 16 GB; so does the Galaxy S26 Ultra, if you buy the terabyte version. That's enough RAM to run Bonsai with room to think, and the silicon to inference at a usable rate. The other number that matters on a phone is energy, and here the ternary case gets stronger: PrismML measures the model at around half a milliwatt-hour per token. Thinking at length on a battery is suddenly plausible. The latest iPhones have 12 GB. Not quite enough room. It might fit, but you'd have to kick everything else out, and that's not what you want. It's here that Apple made an unforced error https://thewatershed.markpesce.com/apples-difficult-calculus/ , when they didn't push their phones to 16 GB. They reckon their own models will be fine in a smaller footprint, and perhaps they will be. Yet the thing we want from a phone now is the kind of thinking that takes memory: at length, with context, over a task rather than a query. Apple built the chip for it and shorted the RAM. The few tests I've been able to run show Bonsai is capable. But I'll need to give it a long-horizon task to know for sure how capable it is: a night's work, unattended, with the errors counted in the morning. It's only by watching things fall over that we learn where the limits really are. It only has to be good enough. If it meets that bar, the Device Watershed https://thewatershed.markpesce.com/four-watersheds/ is here. Five weeks after the Home Watershed. If not... it doesn't feel very far away.