{"slug": "you-got-mlx-serve-d", "title": "You Got mlx-serve'd!", "summary": "Mark Murphy reports that mlx-serve, an Ollama-like inference server with a GUI, delivers a 4x speed increase for running Qwen 3.8 on his 64GB M2 Ultra Mac Studio compared to his previous Ollama setup, thanks to better MLX support and Multi-Token Prediction (MTP). The tool enables local agentic coding but consumes 99% of the GPU during such tasks, limiting it to lightweight workloads.", "body_md": "Earlier, I wrote about [Qwen 3.8 being good but slow](https://pac.commonsware.com/archive/qwen-38-quality-sans-speed/). Since then, I have been trying to figure out how to boost the speed of running Qwen 3.8 on my 64GB M2 Ultra Mac Studio.\n\nWhat is working for me [is mlx-serve](https://github.com/ddalcu/mlx-serve). This is kinda Ollama-ish, in that it has an inference server with an associated GUI. Overall, it does quite a bit more than Ollama does. But the biggest thing is that it runs Qwen 3.8\n\nOn the one hand, that means that Sonnet is much faster and gives higher-quality results. On the other hand, *Qwen 3.8 is a local model*:\n\nRight now, I am having Sonnet review Qwen's work, but so far, Qwen has reliably delivered what it was supposed to do. And this is about a 4x speed increase from what I had with my Ollama-based Qwen 3.8 setup.\n\nPart of that speed increase may be from better MLX support in `mlx-serve`\n\nthan in Ollama. The bigger thing might be the support that `mlx-serve`\n\nhas for MTP (Multi-Token Prediction). MTP is a form of \"speculative decoding\", where lesser models do the heavy lifting and the top-tier model chooses among their outputs. Qwen 3.6 and 3.8 offer MTP, and `mlx-serve`\n\nlets me toggle that on.\n\nThere are clear limits. Having Qwen 3.8 do agentic coding consumes 99% of the GPU of my Mac Studio, and that is with just a single conversation. Agent swarms require a fleet of hardware, not just a single Mac.\n\nStill, this is more than good enough for lightweight tasks and lets me offload some of my development from frontier models and their data centers.", "url": "https://wpnews.pro/news/you-got-mlx-serve-d", "canonical_source": "https://pac.commonsware.com/archive/you-got-mlx-served/", "published_at": "2026-08-31 16:33:13+00:00", "updated_at": "2026-08-31 16:54:44.721422+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-infrastructure"], "entities": ["mlx-serve", "Qwen 3.8", "Ollama", "Mark Murphy", "M2 Ultra Mac Studio", "MLX", "Multi-Token Prediction"], "alternates": {"html": "https://wpnews.pro/news/you-got-mlx-serve-d", "markdown": "https://wpnews.pro/news/you-got-mlx-serve-d.md", "text": "https://wpnews.pro/news/you-got-mlx-serve-d.txt", "jsonld": "https://wpnews.pro/news/you-got-mlx-serve-d.jsonld"}}