Running Qwen3.6 Locally on a MacBook: oMLX + opencode, Faster Than I Expected

A developer ran Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with roughly 3B active parameters, locally on a 14-inch MacBook Pro (M5 Pro, 48GB) using the MLX-based oMLX v0.7.2 inference server, and connected it to the opencode terminal coding assistant through oMLX's OpenAI-compatible API. Estimating from the chat UI's token counter, the setup generated roughly 100 tokens per second, though the developer cautioned this was an order-of-magnitude figure rather than a rigorous benchmark. The omlx-server process alone used about 18GB of memory, and the developer concluded that simple and text-heavy tasks can be handed to a local model while only deep-reasoning work needs a top cloud model.

One 14-inch MacBook Pro. No internet, no API bill. Chat and coding both work, and token output is fast. Load Qwen3.6-35B-A3B MLX 4bit locally with oMLX . It is good enough for everyday chat and coding. | Item | Spec | |---|---| | Machine | MacBook Pro 14" M5 Pro | | CPU | 15 cores 10 performance + 5 super cores | | GPU | 16 cores | | Memory | 48 GB | | OS | macOS Tahoe 26.6.1 | | Inference server | oMLX v0.7.2 | | Model | Qwen3.6-35B-A3B-MLX-4bit | The 35B-A3B in the name is the key. It is a MoE Mixture of Experts model with 35B total parameters, but only about 3B are active per token. MLX is Apple's own machine learning framework. It uses Metal and unified memory directly, and is usually easier and faster on a Mac than generic solutions. oMLX is a local inference server built on MLX, with a built-in web Chat. Once oMLX is running, it comes with a web chat UI. Model selection, conversation history and search all work out of the box. A few details from the screenshot: Qwen3.6-35B-A3B-MLX-4bit . The recording below about 34 seconds is one complete conversation: The flow: This detour is normal: oMLX is a new tool and not in the model's training data. A local model is good for general Q&A, rewriting and explaining code. For very new tools, you still need to read the docs yourself. For coding I used opencode an AI coding assistant for the terminal . After pointing its model at the local oMLX endpoint, the bottom of the UI shows the current mode and model: Build · Qwen3.6-35B-A3B-MLX-4bit · oMLX . Here is a recording about 23 seconds : How to connect it: oMLX exposes an OpenAI-compatible API, so you only need to add a custom provider in the opencode config. The config below is an example. Use the port and model ID that your own oMLX shows. { "$schema": "https://opencode.ai/config.json", "provider": { "omlx": { "npm": "@ai-sdk/openai-compatible", "name": "oMLX", "options": { "baseURL": "http://localhost:<oMLX port /v1" }, "models": { "Qwen3.6-35B-A3B-MLX-4bit": { "name": "Qwen3.6-35B-A3B-MLX-4bit" } } } } } After that, pick this model in opencode, and in Build mode it can read code, edit files and run commands. All inference happens on your machine and the code never leaves it, which is especially useful for company projects. For speed, I made a rough estimate from the token counter in the chat recording above. The counter at the bottom right of the input box reads 1,617 tokens at 25s and 2,224 tokens at 31s. That is over 600 new tokens in about 6 seconds, or roughly 100 tokens/s . This is only an order-of-magnitude estimate from the UI counter. The counting method and refresh timing can introduce error, so it is not a rigorous benchmark. Add your precise measurements here, e.g. time to first token, generation tokens/s, coding task duration Subjectively the output feels smooth. The gap with cloud models is mainly in reasoning depth on complex tasks, not in waiting time. For memory, here is Activity Monitor: Beyond memory, one more thing to know up front: while a task is running, GPU usage hits 100%, the fans start spinning, and there is some noise. This is the unavoidable price of running large models locally. If you work in a quiet office or at night, be prepared. For long batch jobs like the translation mentioned below , plug in the power and keep the laptop somewhere ventilated. The screenshot above was taken with a browser, an IDE and other everyday apps open. The omlx-server process alone uses about 18GB. Memory pressure shows Normal, but 84% is used and there is a little swap. So 48GB is a comfortable minimum. With 32GB, close other big apps or switch to a smaller model. My conclusion after using it: simple tasks and text-heavy tasks can be fully handed to a local model, and only complex tasks that need deep reasoning go to a top cloud model. Not everything needs the most expensive model. Good for local: Still better in the cloud: I once used Claude to translate about 5,000 articles, and the API cost was over 100 USD . Looking back, this kind of task never needed a flagship model: the source is clear, the goal is clear, and the quality bar is "fluent and accurate", not "deep reasoning". If I had run it on a local Qwen3.6-35B-A3B, that money would mostly have been saved. The cost would be electricity and machine time. There is also no quota locally, so a batch job can just run in the background without worrying about the bill. My approach now would be: That way most tokens are spent locally, and cloud money goes only where it is worth it. oMLX makes running models locally painless: one service provides both web Chat and a compatible API, and together with opencode, both chat and coding work on a single MacBook. A MoE model like Qwen3.6-35B-A3B gives 35B-level knowledge with 3B-level generation speed, and the experience on Apple Silicon is very good. If you have a Mac with around 48GB of memory, it is worth a try. Handing translation, summarization and other simple but token-hungry work to the local model, and saving cloud quota for truly hard problems, is the most cost-effective way I have found so far. Just be ready for the GPU at full load and some fan noise while it runs. Have you run large models locally? What machine, which model, how fast? Which tasks would you hand to a local model, and which would you keep in the cloud? Feedback and comments are very welcome. Please also point out anything inaccurate in this post, and I will keep updating it. I built a personal product called GoodAI.Life https://goodai.life https://goodai.life . I’m currently building Good AI Life opens in a new tab https://goodai.life/ , a next-generation AI directory designed to showcase your product in action. If you are interested, please take a look, give it a try, and send me feedback.