{"slug": "mps-in-pytorch-2-15", "title": "MPS in PyTorch 2.15", "summary": "Hugging Face's PyTorch team reported that PyTorch 2.15 brings FP8 support and 5–6.5× faster mixture-of-experts decoding to the MPS backend for Apple silicon Macs, with Qwen3.6-35B-A3B rising from 9.4 to 61.3 tokens/s on an M3 Ultra under torch.compile. The release adds float8_e4m3fn storage at half of BF16's memory with scaled matrix multiplication, matrix-vector kernels for single-row linear layers, and a softmax over the vocabulary roughly 8× faster. Wan2.2's tiled video VAE decode now peaks at about 8 GiB of driver memory instead of 270 GiB, and a padding fix raises Qwen-Image-2.1's encode-decode round trip to 29.95 dB PSNR from 10.06 dB; MPS now requires macOS 15 or later and PyTorch 2.15 requires Python 3.11 or later.", "body_md": "# \n\t\tMPS in PyTorch 2.15\n\t\n\n [Update on GitHub](https://github.com/huggingface/blog/blob/main/mps-pytorch-2-15.md)\n\nA quick word about who we are: we're the PyTorch team at Hugging Face. We contribute across PyTorch, and one of our focus areas is the MPS backend, where the goal is to make your Mac a first-class place to run and train PyTorch models. At the start of 2026, running even small models on MPS often meant working around unsupported operations, performance cliffs and unexpected memory growth. So we run real models from the Hugging Face Hub across language, image and video, find where MPS falls short and fix it upstream in PyTorch, more and more often by replacing MPSGraph with dedicated Metal kernels. Much of this post is the result of that work, done together with the PyTorch maintainers and community contributors.\n\n## \n\t\tTL;DR\n\t\n\n- **FP8 comes to the Mac:** store`float8_e4m3fn` values in half the space of BF16 and use scaled matrix multiplication and all other ops which are supported on CUDA/CPU!\n- **MoE models decode 5–6.5× faster than in 2.14:** Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s on an M3 Ultra, thanks to a native grouped matmul with a dedicated decoding path, fewer GPU-CPU syncs and faster top-k routing.\n- **Less time per token for every LLM:** single-row linear layers use specialized matrix-vector kernels, and softmax over the vocabulary is about 8× faster.\n- **More linear algebra on the GPU:** complex LU and triangular solves, faster small-matrix inverses, and faster QR decomposition.\n- **Audio, video, and training get attention too:** faster paths for convolutions, padding, dropout, interpolation backward and reductions. Wan2.2's video VAE decode now peaks at 8 GiB of memory instead of 270 GiB.\n- **More control from Python:** public MPS streams and more reliable event timing.\n- **Updated requirements:** MPS now requires**macOS 15 or later** ; PyTorch 2.15 requires**Python 3.11 or later** .\n- **A peek at 2.16:** attention fully on Metal, more MPSGraph-to-Metal migrations and faster LLM decoding.\n\n## \n\t\tWhat this means for you\n\t\n\nIf you run models on a Mac with transformers, diffusers or tools like ComfyUI, here's what 2.15 changes in practice:\n\n- **Much faster local LLMs.** Mixture-of-experts models decode 5–6.5× faster than in 2.14: on an M3 Ultra with`torch.compile` , Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s. Dense models benefit too, from faster linear layers and an 8× faster softmax for sampling. ([More](#faster-language-model-decoding) )\n- **Video generation that fits in memory.** Decoding a video with Wan2.2's tiled VAE used to push driver memory to 270 GiB; it now peaks at about 8 GiB. ([More](#a-much-smaller-memory-footprint-for-video-decoding) )\n- **Correct image editing with Qwen-Image-2.1.** On 2.14, the Qwen-Image-2.1 VAE encoder silently corrupted images on MPS, breaking image-edit workflows ([reported in ComfyUI](https://github.com/Comfy-Org/ComfyUI/issues/16433) ). The[same padding change](https://github.com/pytorch/pytorch/pull/195368) fixes it: an encode-decode round trip now matches CPU at 29.95 dB PSNR, up from 10.06 dB.\n- **FP8 weights on your Mac.** FP8 tensors take half the memory of BF16, and`F.scaled_mm` multiplies them directly on the GPU. ([More](#fp8-comes-to-mps) )\n\n## \n\t\tContents\n\t\n\n## \n\t\tGetting started\n\t\n\nYou'll need an Apple silicon Mac with **macOS 15 or later** and **Python 3.11 or later** (see the [2.15 compatibility matrix](https://github.com/pytorch/pytorch/blob/release/2.15/RELEASE.md#release-compatibility-matrix)). If your OS is too old or no Metal GPU is available, the MPS availability check now tells you why.\n\nTo try the release candidate in a separate environment:\n\n```\nuv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu\n```\n\n## \n\t\tFP8 comes to MPS\n\t\n\nFP8 gives you another way to save memory on your Mac. PyTorch 2.15 adds [`torch.float8_e4m3fn`](https://github.com/pytorch/pytorch/pull/194074) tensors on MPS: one byte per value, half the storage of FP16 or BF16.\n\nYou can multiply these tensors with [`F.scaled_mm`](https://github.com/pytorch/pytorch/pull/196916), which supports both tensorwise and rowwise scaling. Here's a small example with one scale per tensor:\n\n``` python\nimport torch\nimport torch.nn.functional as F\n\ndef quantize_fp8(x):\n    limit = torch.finfo(torch.float8_e4m3fn).max\n    scale = x.abs().amax().clamp_min(1e-12) / limit\n    return (x / scale).to(torch.float8_e4m3fn), scale\n\nx = torch.randn(1, 1024, device=\"mps\", dtype=torch.float32)\nweight = torch.randn(1024, 1024, device=\"mps\", dtype=torch.float32)\nx_fp8, x_scale = quantize_fp8(x)\nw_fp8, w_scale = quantize_fp8(weight)\n\ny = F.scaled_mm(\n    x_fp8,\n    w_fp8.t(),\n    scale_a=x_scale,\n    scale_recipe_a=F.ScalingType.TensorWise,\n    scale_b=w_scale,\n    scale_recipe_b=F.ScalingType.TensorWise,\n    output_dtype=torch.bfloat16,\n)\nprint(y.shape, y.dtype)\n```\n\n## \n\t\tFaster language-model decoding\n\t\n\nEvery LLM has a prefill phase and a decode phase. During decode, each new token goes through the model as a single row, so the big matrix multiplications turn into matrix-vector products. PyTorch 2.15 speeds up both kinds of layers that dominate this step: dense linear layers and mixture-of-experts layers.\n\n### \n\t\tLinear layers\n\t\n\nMPS has had GEMV kernels for `torch.mm` since 2.14, but `F.linear` still went through MPSGraph. PyTorch 2.15 [connects them to `F.linear`](https://github.com/pytorch/pytorch/pull/198721), so the common `[1, 1, hidden_size]` decoding input uses them directly, with eligible bias additions fused in. Across 96 BF16 layer shapes from real models, that's a **1.23× geometric-mean speedup**. The biggest wins land on small projections, such as the gates in MoE models, where the old path left most of the memory bandwidth unused. Large layers and LM heads were already close to bandwidth-bound, so they stay near 1×.\n\n### \n\t\tMixture-of-experts layers\n\t\n\nA mixture-of-experts layer sends different numbers of tokens to each expert. Grouped matmul handles those uneven groups in a single operation, and 2.15 gives MPS a [native implementation](https://github.com/pytorch/pytorch/pull/194933).\n\nIn 2.14, grouped matmul copied the group offsets to the CPU and looped over the groups. The new Metal kernels read the offsets directly on the GPU, using Metal Performance Primitives where available and a separate Metal path elsewhere. Small groups, the common case during decoding, get [their own kernel](https://github.com/pytorch/pytorch/pull/198725) as well.\n\nFor example, 64 tokens can be split into groups of 16, 32, and 16:\n\n``` python\nimport torch\nimport torch.nn.functional as F\n\ntokens = torch.randn(64, 64, device=\"mps\", dtype=torch.bfloat16)\nweights = torch.randn(3, 128, 64, device=\"mps\", dtype=torch.bfloat16)\nends = torch.tensor([16, 48, 64], device=\"mps\", dtype=torch.int32)\n\nout = F.grouped_mm(tokens, weights.transpose(-2, -1), offs=ends)\nprint(out.shape)  # torch.Size([64, 128])\n```\n\nThe decoding kernel shows up directly in token throughput. On an **M3 Ultra with `torch.compile`**, MoE models generate about **1.3× more tokens per second**:\n\n## \n\t\tLinear algebra gets a boost\n\t\n\n### \n\t\tComplex-valued computations\n\t\n\nWorking with complex tensors gets easier. `complex64` [LU factorization](https://github.com/pytorch/pytorch/pull/188473), [LU solve](https://github.com/pytorch/pytorch/pull/195369) and [triangular solve](https://github.com/pytorch/pytorch/pull/195861) now run on MPS, which also unlocks the operations and gradients built on them, such as complex determinants. Here's a small system solved directly on the GPU:\n\n``` python\nimport torch\n\na = torch.randn(8, 8, device=\"mps\", dtype=torch.complex64)\na = a + 8 * torch.eye(8, device=\"mps\", dtype=torch.complex64)\nb = torch.randn(8, 2, device=\"mps\", dtype=torch.complex64)\n\nlu, pivots = torch.linalg.lu_factor(a)\nx = torch.linalg.lu_solve(lu, pivots, b)\nprint((a @ x - b).abs().max().item())\n```\n\n### \n\t\tSmall matrices and larger solves\n\t\n\nSmall matrices get fast kernels too. For camera geometry and image transformations, batches of FP32 matrices up to **4 × 4** are [inverted in a single Metal kernel](https://github.com/pytorch/pytorch/pull/195944) that combines factorization and solving. Larger problems get attention as well: [QR uses blocked updates](https://github.com/pytorch/pytorch/pull/198732), and [triangular solves](https://github.com/pytorch/pytorch/pull/196241) turn much of their work into efficient matrix multiplication.\n\nGated DeltaNet workloads get a [dedicated fast path](https://github.com/pytorch/pytorch/pull/196900) for batches of **16 × 16, 32 × 32 and 64 × 64 FP32 triangular systems** with at least eight right-hand sides.\n\nSVD is also more robust at numerical extremes. It now correctly handles [very small singular values](https://github.com/pytorch/pytorch/pull/200920), [very large or very small input magnitudes](https://github.com/pytorch/pytorch/pull/201037) and [basis completion with `full_matrices=True`](https://github.com/pytorch/pytorch/pull/201198), which in turn makes pseudoinverse and least-squares results more reliable.\n\n## \n\t\tConvolutions, padding, and training\n\t\n\nAudio models, video decoders and training loops have their own expensive operations. This release gives several of them new paths through Metal:\n\n| Operation | What changes in 2.15 | \n|---|---|\n| [`conv1d` forward](https://github.com/pytorch/pytorch/pull/198878) | Specialized kernels replace MPSGraph, plus an im2col-plus-matmul path for short outputs. Useful for audio and streaming-model shapes. | \n| [Constant padding](https://github.com/pytorch/pytorch/pull/195368) | Moves to Metal kernels, fixing runaway driver-memory growth in tiled video VAE decoding (more below). | \n| [Reflection and replication padding](https://github.com/pytorch/pytorch/pull/196985) | New Metal implementations that are faster and fix backward correctness. | \n| [Out-of-place dropout](https://github.com/pytorch/pytorch/pull/195011) | `nn.Dropout` and`F.dropout` now use the existing fused kernel. | \n| [Upsampling backward](https://github.com/pytorch/pytorch/pull/199687) | A Metal gather kernel replaces the MPSGraph resize-gradient path. | \n| [Channels-last `conv2d` backward](https://github.com/pytorch/pytorch/pull/193623) | Stays on an efficient channels-last path instead of an NCHW graph with costly strided access. | \n\n## \n\t\tSoftmax and reductions\n\t\n\nToken sampling gets faster in 2.15 as well. [`softmax`](https://github.com/pytorch/pytorch/pull/196071) and [` log_softmax`](https://github.com/pytorch/pytorch/pull/195441) now run on dedicated Metal kernels, and large vocabularies benefit the most: an FP32 softmax over a `[1, 128256]` row of logits drops from **57 µs to 7 µs**, about **8× faster** than the MPSGraph version. The speedup holds across other common vocabulary sizes:\n\n## \n\t\tA much smaller memory footprint for video decoding\n\t\n\nOne of the most striking results in this release comes from the constant-padding change. The VAE decode in `Wan-AI/Wan2.2-TI2V-5B-Diffusers` makes **8,764 padding calls** over a full 28-tile decode. With padding on Metal, peak driver memory drops from about **270 GiB to 8 GiB**:\n\n| Memory metric | MPSGraph padding | Metal padding | \n|---|---|---|\n| Current tensor-allocation peak | 4.422 GiB | 4.095 GiB | \n| Allocator-reserved peak | 8.070 GiB | 8.070 GiB | \n| Total driver-memory peak | 270.135 GiB | 8.108 GiB | \n\n## \n\t\tMore control with MPS streams\n\t\n\nYou can now pick which MPS stream runs your work, directly from Python. [`torch.mps.Stream` and the `torch.mps.stream` context manager](https://github.com/pytorch/pytorch/pull/191415) provide the interface, and [`Tensor.record_stream`](https://github.com/pytorch/pytorch/pull/193464) lets the allocator track a tensor's use on another stream.\n\nHere's a small example that sends work to a separate stream and synchronizes before reading the result:\n\n``` python\nimport torch\n\nx = torch.randn(1024, device=\"mps\")\ntorch.mps.synchronize()\nworker = torch.mps.Stream()\n\nwith torch.mps.stream(worker):\n    y = x.square()\n    x.record_stream(worker)\n\ndel x\nworker.synchronize()\nprint(y.cpu().shape)\n```\n\n## \n\t\tMore operations work on MPS\n\t\n\nSeveral more operations now run on MPS: [`torch.frexp`](https://github.com/pytorch/pytorch/pull/194110), [` torch.logspace`](https://github.com/pytorch/pytorch/pull/190495), [` torch.take` and `Tensor.put_`](https://github.com/pytorch/pytorch/pull/192980), [` torch.heaviside`](https://github.com/pytorch/pytorch/pull/195265), [` torch.special.laguerre_polynomial_l`](https://github.com/pytorch/pytorch/pull/195225) and [unsigned-integer comparisons](https://github.com/pytorch/pytorch/pull/198860). `logspace` doesn't yet support integer or boolean outputs with more than one element.\n\nAttention gets a correctness fix too: the MPS math implementation of scaled dot-product attention now [handles batch and head broadcasting correctly](https://github.com/pytorch/pytorch/pull/189631).\n\n## \n\t\tA sneak peek at PyTorch 2.16\n\t\n\nWork on 2.16 is already well underway. Some of the changes below have landed on `main` and are available in the nightlies; others are still in review, so details may change before the release.\n\n### \n\t\tFaster LLM decoding\n\t\n\nOnce a model's decode path no longer contains any MPSGraph ops, the GPU only starts on a token after the CPU has finished encoding all of its kernels. PyTorch 2.16 [commits the command buffer every 16 kernels](https://github.com/pytorch/pytorch/pull/200181), so the GPU gets to work while the CPU is still encoding the rest. For compiled BF16 decoding at a 2048-token context:\n\n| Model | Before | After | Change | \n|---|---|---|---|\n| LFM2.5-8B-A1B | 108.8 tokens/s | 126.0 tokens/s | +16% | \n| Qwen3.6-35B-A3B | 52.9 tokens/s | 62.8 tokens/s | +19% | \n| Qwen3.8-27B | 12.1 tokens/s | 12.6 tokens/s | +4% | \n\n### \n\t\tAttention runs entirely on Metal\n\t\n\nScaled dot-product attention is [dropping MPSGraph completely](https://github.com/pytorch/pytorch/pull/201526): every shape will run on Metal kernels, with dedicated vector kernels for decoding and Metal Performance Primitives for half-precision prefill on newer macOS versions. The shapes that used to fall back to MPSGraph benefit the most. On an M5 Pro, decoding with grouped-query attention (32 query heads, 8 KV heads, head dim 80, 4K context) drops from 465 µs to 65 µs, and a 4K × 4K prefill with head dim 40 goes from 42 ms to 6 ms.\n\n### \n\t\tMore MPSGraph to Metal migrations\n\t\n\nThe move away from MPSGraph continues. Batch norm [forward](https://github.com/pytorch/pytorch/pull/201361) and [backward](https://github.com/pytorch/pytorch/pull/201411) are in progress, while [MSE loss](https://github.com/pytorch/pytorch/pull/201351) (up to 12× faster on small inputs), [`std_mean` and `var_mean`](https://github.com/pytorch/pytorch/pull/201310) (up to 4× faster) and [`bincount`](https://github.com/pytorch/pytorch/pull/201045) have already landed. `bincount` is the most dramatic: when most values fall into a single bin, counting 4 million integers used to take 6.8 seconds and now takes half a millisecond.\n\n### \n\t\tA smaller install\n\t\n\nBundled shaders will [build for Metal 3.1 by default](https://github.com/pytorch/pytorch/pull/197529), with Metal 4 versions compiled only for the kernels that need them. That shrinks `libtorch_cpu.dylib` by about 25%, from 384 MiB to 287 MiB, and lays the groundwork for Metal 4.1 shaders with faster matmuls and native FP8 matrix multiplication.\n\n## \n\t\tWrapping up\n\t\n\nPyTorch 2.15 brings FP8 to the Mac, faster LLM decoding for dense and MoE models, more linear algebra on the GPU and a long list of operations moved from MPSGraph to dedicated Metal kernels. Give the release candidate a try!\n\n### \n\t\tWhat's next for us\n\t\n\n- **More MPS:** 2.16 will run attention entirely on Metal, and we'll keep moving operations off MPSGraph.\n- **`torch.compile`:** we're working on getting rid of graph breaks, so more models compile end to end.\n- **Your ideas:** what else should we work on in PyTorch? If something is slow, missing or broken, on MPS or anywhere else,[open an issue on GitHub](https://github.com/pytorch/pytorch/issues) and let us know.\n\nA big thank you to [Nikita Shulga](https://github.com/malfet), [Kurt Mohler](https://github.com/kurtamohler) and [Aaron Gokaslan](https://github.com/Skylion007) for reviewing much of this work, and to everyone who contributed to MPS in this release.", "url": "https://wpnews.pro/news/mps-in-pytorch-2-15", "canonical_source": "https://huggingface.co/blog/mps-pytorch-2-15", "published_at": "2026-10-20 00:00:00+00:00", "updated_at": "2026-10-20 11:19:26.833873+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure", "ai-tools", "large-language-models"], "entities": ["Hugging Face", "PyTorch", "PyTorch 2.15", "MPS", "Apple", "Qwen3.6-35B-A3B", "Wan2.2", "Qwen-Image-2.1"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/mps-in-pytorch-2-15", "markdown": "https://wpnews.pro/news/mps-in-pytorch-2-15.md", "text": "https://wpnews.pro/news/mps-in-pytorch-2-15.txt", "jsonld": "https://wpnews.pro/news/mps-in-pytorch-2-15.jsonld"}}