You Got mlx-serve'd!

Mark Murphy reports that mlx-serve, an Ollama-like inference server with a GUI, delivers a 4x speed increase for running Qwen 3.8 on his 64GB M2 Ultra Mac Studio compared to his previous Ollama setup, thanks to better MLX support and Multi-Token Prediction (MTP). The tool enables local agentic coding but consumes 99% of the GPU during such tasks, limiting it to lightweight workloads.

Earlier, I wrote about Qwen 3.8 being good but slow https://pac.commonsware.com/archive/qwen-38-quality-sans-speed/ . Since then, I have been trying to figure out how to boost the speed of running Qwen 3.8 on my 64GB M2 Ultra Mac Studio. What is working for me is mlx-serve https://github.com/ddalcu/mlx-serve . This is kinda Ollama-ish, in that it has an inference server with an associated GUI. Overall, it does quite a bit more than Ollama does. But the biggest thing is that it runs Qwen 3.8 On the one hand, that means that Sonnet is much faster and gives higher-quality results. On the other hand, Qwen 3.8 is a local model : Right now, I am having Sonnet review Qwen's work, but so far, Qwen has reliably delivered what it was supposed to do. And this is about a 4x speed increase from what I had with my Ollama-based Qwen 3.8 setup. Part of that speed increase may be from better MLX support in mlx-serve than in Ollama. The bigger thing might be the support that mlx-serve has for MTP Multi-Token Prediction . MTP is a form of "speculative decoding", where lesser models do the heavy lifting and the top-tier model chooses among their outputs. Qwen 3.6 and 3.8 offer MTP, and mlx-serve lets me toggle that on. There are clear limits. Having Qwen 3.8 do agentic coding consumes 99% of the GPU of my Mac Studio, and that is with just a single conversation. Agent swarms require a fleet of hardware, not just a single Mac. Still, this is more than good enough for lightweight tasks and lets me offload some of my development from frontier models and their data centers.