Member-only story
It beat my expectations on some tasks and fell short on others — and where the line falls is the actually useful part. #
I’ve been writing a lot lately about cutting LLM costs the “proper” engineering way — model routing, caching, trimming context. All real techniques. But there’s a much blunter version of the same idea sitting right on my own laptop, and I wanted to actually try it before recommending it to anyone: what if the cheapest tier in your routing table isn’t a smaller paid model, but a model that costs exactly nothing, because it’s running locally?
So I installed one and started using it for real work. Here’s the honest report — not a hype piece, because the results didn’t come out as a clean win.
What I picked, and why #
I’m on a 16GB Apple Silicon Mac, and after checking what actually fits that spec well, I landed on Qwen 3 8B. This wasn’t a random pick — multiple independent sources converge on it specifically for 16GB machines. At Q4 quantization it’s about a 5GB download, and it’s repeatedly recommended as the model to start with for general chat, summarization, and writing on this exact RAM tier.