# Running Qwen3 Through the ExecuTorch MLX Delegate: Up to 4.52x Faster on M1 Max > Source: > Published: 2026-07-22 03:57:08+00:00 Hello, everyone. There are now many ways to run an LLM on a Mac, but exporting a PyTorch model for Apple Silicon and executing it in a lightweight runtime is still an evolving path. How much faster is it, and does 4-bit quantization change the output? Today, I am looking at ExecuTorch's experimental MLX delegate, released in May 2026. It enables PyTorch models to run on Apple Silicon GPUs. I use ExecuTorch 1.3.1 to run Qwen3-0.6B and compare it with PyTorch MPS. The short result is that decode throughput was **41.8 tokens/s** with PyTorch MPS BF16, **134.8 tokens/s** with MLX BF16, and **188.9 tokens/s** with MLX INT4. MLX INT4 was 4.52x faster, and its file was 71.8% smaller than BF16. However, INT4 changed the generated output in two of three simple prompts. ExecuTorch is a runtime for running trained PyTorch models on desktops, phones, and embedded devices. Inference means feeding input into a trained model to obtain an output. The [MLX delegate](https://pytorch.org/blog/running-pytorch-models-on-apple-silicon-gpus-with-the-executorch-mlx-delegate/) was released on **May 18, 2026**. It sends a PyTorch computation graph to Apple's MLX framework for execution on an Apple Silicon GPU. It supports BF16, FP16, FP32, and 2/4/8-bit quantization, among other formats. It is currently experimental, so its APIs and supported scope may change. Quantization stores weights with fewer bits to reduce size and computation. I compared ordinary BF16 with INT4, which quantizes the linear layers and embeddings to four bits. [Qwen3](https://qwenlm.github.io/blog/qwen3/) is an LLM family announced by the Qwen Team on **April 29, 2025**. A single model can switch between a thinking mode for step-by-step reasoning and a non-thinking mode for shorter answers. The family supports more than 100 languages and dialects. The [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) used here is one of the smaller dense models in the family. “Dense” means that it generally uses the full model for each input, unlike a mixture-of-experts model that activates only selected parts. It has a nominal 0.6 billion parameters, 28 layers, and a context length of 32,768 tokens. A token is a small unit of text processed by the model. I disabled thinking mode for this test and pinned the model revision to `c1899de289a04d12100db370d81485cdf75e47ca` . | Component | License | |---|---| | Qwen3-0.6B weights | | These permissive open-source licenses allow broad use, including commercial use, but their conditions—such as retaining copyright notices when redistributing—still apply. Check the linked license text for your use case. Only one trained model, Qwen3-0.6B, was used. The same weights were tested through three execution paths. ``` php same prompt -> tokenizer: convert text to token IDs -> Qwen3-0.6B ├─ ExecuTorch MLX BF16 -> .pte -> MLX / Metal GPU ├─ ExecuTorch MLX INT4 -> 4-bit .pte -> MLX / Metal GPU └─ PyTorch MPS BF16 ----------------> MPS / Metal GPU -> greedy generation: select the most likely next token each time -> tokenizer: convert token IDs back to text ``` A `.pte` file is a model program exported for ExecuTorch. The complete computation graph could be lowered into a single MLX subgraph. PyTorch MPS BF16 ran the same BF16 weights through the ordinary Transformers/PyTorch path and served as the reference. The complete code and JSON report are available in the [executorch-mlx-qwen3 lab in kiarina/labs](https://github.com/kiarina/labs/tree/main/2026/07/22/executorch-mlx-qwen3). You need an Apple Silicon Mac, Xcode Command Line Tools, `mise` , `uv` , and an internet connection. The first run downloads the Qwen3 weights and creates about 1.53 GB of PTE files in total. ``` git clone --depth 1 --filter=blob:none --sparse \ https://github.com/kiarina/labs.git cd labs git sparse-checkout set .gitignore .mise/tasks Makefile mise.toml \ 2026/07/22/executorch-mlx-qwen3 mise -C 2026/07/22/executorch-mlx-qwen3 run ``` The export and benchmark steps can also be run separately. ``` mise -C 2026/07/22/executorch-mlx-qwen3 run export mise -C 2026/07/22/executorch-mlx-qwen3 run benchmark machine: MacBook Pro (Apple M1 Max, 32 GPU cores, 64 GB) OS: macOS 26.5.2 Python: 3.13.7 ExecuTorch: 1.3.1 PyTorch: 2.12.1 Transformers: 4.56.1 model: Qwen/Qwen3-0.6B generation: greedy, batch 1, up to 16 tokens PTE: custom MLX SDPA / KV cache, requested maximum sequence 128 ``` For performance, each backend was given the same Japanese prompt and forced to generate 16 tokens. The reported values are medians from five trials after warm-up. Prefill reads the input and produces the first token; decode generates the remaining tokens one at a time. Each backend ran in a separate process. The MLX path loaded a fresh forward method and initialized its KV cache for each trial, while the PyTorch path created a fresh cache for every trial. | PTE | Size | Relative to BF16 | Export time | SHA-256 | |---|---|---|---|---| | MLX BF16 | 1,192,264,196 bytes | 100.0% | 41.61 s | `83da47c2…bfb8c0` | | MLX INT4 | 335,662,976 bytes | 28.2% | 54.55 s | `0e30a054…71267` | INT4 was 856,601,220 bytes, or **71.8% smaller**, than BF16. Quantization itself takes work, so exporting INT4 took about 13 seconds longer. The input was the Japanese prompt “Briefly explain local inference on Apple Silicon.” | Backend | Load | Median prefill | Median decode | 16-token total | Peak RSS increase | |---|---|---|---|---|---| | ExecuTorch MLX BF16 | 0.002 s | 0.020 s | 134.8 tokens/s | 0.131 s | 1.27 GiB | | ExecuTorch MLX INT4 | 0.003 s | 0.028 s | 188.9 tokens/s | 0.108 s | 0.47 GiB | | PyTorch MPS BF16 | 0.726 s | 0.038 s | 41.8 tokens/s | 0.396 s | 0.14 GiB | MLX BF16 decoded **3.22x** as fast as PyTorch MPS BF16, while MLX INT4 was **4.52x** as fast. INT4 was also 1.40x faster than MLX BF16 and reduced the RSS increase by about 63%. There are important qualifications. The MLX load figure measures only opening the PTE program, not all work needed to materialize weights for GPU use. RSS measures the process's main memory, not GPU memory itself. PyTorch reported 1.20 GiB of MPS driver-allocated memory at the end of the run. Because GPU memory was not measured on the same basis, the table does not prove that PyTorch used the least memory. On the first invocation only, BF16 prefill took 0.434 seconds and INT4 took 1.014 seconds. Cold starts that include Metal setup and initial compilation were much slower than the warmed-up figures. I compared the generated token sequences on three short prompts. | Prompt | PyTorch MPS BF16 | MLX BF16 | MLX INT4 | |---|---|---|---| | Answer Japan's capital in one word | `日本の首都は、**大阪**です。` (Japan's capital is Osaka.) | Exact token match | `日本の首都は、**东京**です。` (Japan's capital is Tokyo.) | | Answer 1+1 with one digit | `1+1=2` | Exact token match | `1` | Copy `MLX` unchanged | `MLX` | Exact token match | Exact token match | MLX BF16 matched PyTorch MPS BF16 token for token in all three cases. Within this narrow test, changing the execution path to MLX did not introduce a difference. INT4 matched in only one case. Quantization represents numbers more coarsely, so when candidate next tokens have similar scores, their ranking can change. The capital answer was wrong in both BF16 and INT4. The small 0.6B model, prompt, and greedy decoding could all contribute, but three questions are not enough to identify the cause. This probe checks differences between backends; it does not certify the model's knowledge or answer quality. Even BF16 decoded 3.22x as fast as PyTorch MPS. Exporting a PyTorch model to a lightweight Mac runtime looks promising. INT4 reduced the file from about 1.19 GB to 336 MB and produced the highest decode rate. Yet it changed two of only three token sequences. It should be evaluated on a task-specific quality set before adoption. This comparison covers an ExecuTorch MLX pipeline versus a Transformers/PyTorch MPS pipeline. Their runtimes, cache handling, and execution paths differ, so the numbers describe the complete pipelines. The dependency metadata for `executorch==1.3.1` allowed PyTorch 2.13.0, but importing the published ExecuTorch extension failed because the `materialize_cow_storage` symbol was missing. Pinning PyTorch to 2.12.1 made the same wheel work. The failure can be reproduced with this optional task: ``` mise -C 2026/07/22/executorch-mlx-qwen3 run probe-torch-2-13 ``` The bundled PTE inspector also failed because its included `flatc` did not recognize the `--json` option. I verified full-graph delegation from the partitioner log emitted during export instead. Lowering the entire Qwen3-0.6B graph to MLX and increasing decode throughput by more than 3x without changing the BF16 tokens was a good result. INT4 reduced the file to less than one-third of its BF16 size and reached about 189 tokens/s, which is attractive when embedding a small model on a Mac. The output changes from 4-bit quantization appeared immediately, even in this tiny probe. Quantization is not a free speedup. Paired with a quality suite for the real task, this path could work well for local helper features or small, responsive agents.