{"slug": "automatic1111-for-apple-metal-40-speed-up-sd1-5", "title": "Automatic1111 for Apple metal, 40% speed up sd1.5", "summary": "Derek Anderson reports that teaching Automatic1111 to use Apple's Metal API on an M3 Pro cut Stable Diffusion 1.5 generation times from 8–10 seconds to 3–7 seconds, a roughly 40% speedup, without replacing the WebUI or converting to Core ML. The optimization involved adding a selective Metal Flash Attention path for specific SD 1.x shapes and reducing MPS command buffer commits, with the caveat that results are observed ranges, not a controlled benchmark.", "body_md": "# From 8–10 Seconds to 3–7: Teaching Automatic1111 to Speak Metal on an M3 Pro\n\n### By Derek Anderson\n\nI use Draw Things a lot on Apple hardware, and one thing has always bothered me about Automatic1111: *it feels slower than it should.*\n\nNot unusably slow. Just slow enough that you notice it.\n\nOn my M3 Pro, a short five-step DPM++ SDE generation in Automatic1111 was typically landing somewhere around 8–10 seconds. Draw Things had already shown me that Stable Diffusion on Apple Silicon could feel much more immediate than that.\n\nSo I wanted to see how much of that gap was actually necessary.\n\nThere was one important constraint: *I did not want to replace Automatic1111.*\n\nI wanted the same WebUI, checkpoints, LoRAs, samplers, extensions, API, prompt syntax, and general workflow. I wasn't interested in converting everything to Core ML and building another inference engine around it. The goal was much narrower:\n\n**How fast can Automatic1111 get if we make the parts that matter behave more like native Apple software?**\n\nThe answer, at least for the workloads I'm running, is quite a bit faster.\n\nThe same class of generation that was taking roughly 8–10 seconds on my M3 Pro is now generally landing between 3 and 7 seconds.\n\nThose are observed ranges across my current workloads, not a controlled benchmark claiming a universal 2x improvement. There is also an important distinction between the runtime improvements and NGMS, which actually reduces the amount of guidance work being performed.\n\nStill, the difference in actual use is substantial.\n\nMore interesting than the final number, though, was what it took to get there.\n\nIt wasn't one optimization.\n\n## Start with the workload, not the benchmark\n\nThe workload I cared about was pretty specific:\n\n- Stable Diffusion 1.x\n- DPM++ SDE\n- Karras\n- 5 steps\n- CFG around 1.15\n- 384×640 and 512×512\n- FP16 UNet on MPS\n- FP32 VAE by default\n\nThat specificity matters.\n\nEarly on, DPM++ 2M looked like an easy way to shave off time. It was faster, but it didn't produce the result I wanted from the short schedule.\n\nThat isn't an optimization. It's a different workload.\n\nThis became the rule for basically everything that followed: if an optimization looks great in isolation but doesn't make the actual generation faster while preserving the result I'm trying to produce, it doesn't count.\n\n## Metal Flash Attention, selectively\n\nAttention was the obvious place to start.\n\nPyTorch's MPS backend has gotten substantially better, but there are still Stable Diffusion attention shapes where going directly to Metal makes sense.\n\nThe mistake would have been treating a custom Metal implementation as universally faster.\n\nIt isn't.\n\nInstead, I added a Metal Flash Attention path specifically for the SD 1.x shapes where it actually won in testing.\n\nThe router looks conceptually like this:\n\n```\nif inference and fp16_mps and query_tokens >= 192 and head_dim in (40, 80, 160):\n    return metal_flash_attention(q, k, v)\n\nreturn pytorch_sdpa(q, k, v)\n```\n\nThere are additional checks around masks, training, dropout, tensor layout, grouped-query attention, and supported types, but that's the basic idea.\n\nMetal is not the default because Metal sounds faster. It gets the operation when we've measured that shape and it deserves it.\n\nEverything else goes back through PyTorch.\n\nThat fallback is important. Automatic1111 supports far more configurations than my five-step SD 1.x workflow. I didn't want a faster fork that only worked if nobody touched anything.\n\n## The kernel wasn't the whole problem\n\nGetting attention into Metal helped, but it exposed something more interesting.\n\nThe native extension was committing the MPS command buffer after every attention call.\n\nStable Diffusion calls attention over and over inside every UNet evaluation. With a short five-step generation, repeatedly submitting tiny chunks of work starts becoming a meaningful part of the total runtime.\n\nSo instead of treating the Metal kernel like its own little application, I integrated it into PyTorch's current MPS stream.\n\nThe extension ends PyTorch's current kernel coalescing, encodes the Metal Flash Attention operation into the current command buffer, and then lets the rest of the PyTorch MPS work continue from there.\n\nThe explicit commit after every attention call went away.\n\nThis ended up being one of the more important lessons from the entire project.\n\n**The fastest kernel still loses if you submit the command buffer after every call.**\n\nAt these generation times, overhead matters. You're no longer just optimizing how quickly the GPU can multiply matrices. You're optimizing how often Python, PyTorch, MPSGraph, and Metal have to coordinate with each other.\n\nThere was also a wonderfully obvious reminder not to trust the timer: one of the early versions produced a green image.\n\nIt was fast.\n\nIt was also green.\n\nThe Metal path now runs an isolated attention-plus-projection correctness test before the WebUI enables it.\n\n## Unified memory changes the rules\n\nThe next problem was memory.\n\nApple Silicon doesn't have a discrete pile of VRAM sitting next to system RAM. The GPU and the rest of the machine are competing for the same physical memory.\n\nThat makes some traditional GPU assumptions fairly bad ones.\n\nAn attention matrix can technically fit in memory and still be a terrible idea if macOS is under pressure, the allocator starts thrashing, or the machine begins swapping.\n\nSo instead of using a fixed VRAM threshold, the fork estimates the cost of native attention against both total and currently available memory.\n\nConceptually:\n\n```\nattention_bytes =\n    batch × heads × query_tokens × key_tokens × element_size\n\nestimated_peak = attention_bytes × 2.5\n\nbudget =\n    min(\n        10% of total memory,\n        20% of currently available memory,\n        1.5 GiB\n    )\n```\n\nIf the estimated peak fits inside that budget, native SDPA can run.\n\nIf it doesn't, the request goes through the memory-bounded sub-quadratic path instead.\n\nThe chunk size for that fallback is dynamic too. An 8 GB Mac shouldn't make the same decision as a 32 GB Mac, and neither should behave as though Chrome, Xcode, or whatever else is running doesn't exist.\n\nI don't count this as a blanket speed improvement. It's mostly about keeping performance predictable and avoiding the cases where an ostensibly fast operation causes enough memory pressure to make the whole generation slower.\n\n## Stop keeping every attention chunk around\n\nI also changed how the sub-quadratic fallback handles K/V chunks.\n\nThe existing approach computes partial attention results, keeps the numerator, normalization weight, and maximum for each chunk, then stacks everything together at the end.\n\nThat's unnecessary.\n\nInstead, the fork maintains a running maximum, normalization sum, and weighted output. Each new K/V chunk gets merged into that running state and can then be discarded.\n\nThe recurrence is basically:\n\n```\nnew_max = max(running_max, chunk_max)\n\nrunning_scale = exp(running_max - new_max)\nchunk_scale = exp(chunk_max - new_max)\n\nrunning_values =\n    running_values × running_scale +\n    chunk_values × chunk_scale\n\nrunning_weights =\n    running_weights × running_scale +\n    chunk_weights × chunk_scale\n```\n\nThis is the same general online-softmax idea that makes Flash Attention memory efficient.\n\nMemory now scales around the current chunk instead of accumulating every partial result until the end.\n\nI tested the forward results against PyTorch SDPA and also tested gradients in float64. Again, the goal wasn't just to make something clever. It had to be a safe fallback.\n\n## Some MPS workarounds have outlived the bugs\n\nThere was another category of optimization that was much less glamorous: deleting old workarounds.\n\nApple's PyTorch backend has changed a lot.\n\nAutomatic1111 accumulated defensive behavior for older MPS implementations, including cloning `torch.narrow()`\n\nresults and pushing LayerNorm through FP32.\n\nThose fixes made sense when the underlying MPS bugs existed. On newer versions of PyTorch, they can just become copies, allocations, conversions, and memory traffic.\n\nSo those behaviors are now gated by runtime version rather than applied indiscriminately.\n\nThere's still an `A1111_MPS_FORCE_LEGACY_OPS=1`\n\nescape hatch if somebody needs the old behavior.\n\nI also enabled `PYTORCH_MPS_PREFER_METAL=1`\n\nbecause direct Metal matrix multiplication tested better for the SD 1.x projection sizes I was targeting, and removed the default sampling upcast so more of the short sampling path stays in FP16.\n\nThat last change is a real tradeoff. FP16 reduction order and removing the upcast can affect same-seed output.\n\nI'm fine with that for this workflow, but it shouldn't be presented as free performance.\n\n## Fusing GroupNorm and SiLU\n\nOnce the unnecessary work was reduced, I went looking for operations that were both necessary and repeated constantly.\n\nGroupNorm followed by SiLU is everywhere in the SD 1.x UNet.\n\nNormally those are separate PyTorch operations. That means separate dispatches and an intermediate activation that gets written out and then immediately read back.\n\nSo I wrote a fused Metal kernel.\n\nFor compatible FP16 inference tensors, one 256-thread Metal threadgroup handles each batch/group pair. The kernel accumulates the sum and squared sum in FP32, reduces those into mean and variance, applies normalization and the affine parameters, applies SiLU, and writes the FP16 result.\n\nOne dispatch. No intermediate activation.\n\nIf the tensor isn't compatible, we're training, gradients are enabled, the dtype is wrong, or the native path fails, it goes straight back to:\n\n```\nF.silu(norm(input_tensor))\n```\n\nI deliberately stopped there.\n\nIt was tempting to start fusing entire residual blocks, but GroupNorm plus SiLU was a pair I could isolate, test, and prove.\n\nAs it turned out, that restraint mattered.\n\n## NGMS is different\n\nThere's one part of the final speedup that needs to be separated from the engine work.\n\nNGMS, or Negative Guidance minimum sigma, can skip unconditional guidance during eligible portions of sampling.\n\nWith classifier-free guidance, the UNet is often doing conditional and unconditional work together. At a low CFG like 1.15 on a five-step schedule, skipping eligible unconditional work can remove a meaningful amount of computation.\n\nThat's obviously fast because the GPU isn't doing some of the work at all.\n\nThe fork defaults NGMS to `1.0`\n\nwith all-steps behavior enabled for this tuned workflow.\n\nBut this isn't the same category as making attention or GroupNorm faster.\n\nNGMS changes the denoising calculation. It can change composition and detail, and it's recorded in the PNG metadata when active.\n\nSo there are really two performance stories here.\n\nThe first is making the existing engine cheaper: Metal attention, fewer command-buffer submissions, better memory behavior, fewer obsolete conversions, and fused operations.\n\nThe second is asking the engine to do less work through NGMS.\n\nAny controlled benchmark of this fork needs to show both.\n\n## The most useful optimizations were the ones I deleted\n\nA lot of this project was trying things that sounded like they should work and then removing them.\n\nPacked QKV projections were one example.\n\nI implemented them. The resulting image was byte-identical in the test.\n\nPerformance went from 8.988 seconds to 9.011 seconds.\n\nThat's about 0.26% slower.\n\nGone.\n\nThe more ambitious experiment was moving entire residual blocks into MPSGraph.\n\nThe idea looked good on paper: GroupNorm, SiLU, 3×3 convolutions, timestep embedding, residual addition, and the optional skip convolution could all live inside one graph. Existing PyTorch MPS buffers could be bound directly, compiled graphs could be cached by shape, and compatible inference blocks could avoid a pile of individual dispatches.\n\nThe microbenchmarks were encouraging too.\n\nSome mid and low-resolution blocks improved by 1–5%. The smallest blocks were as much as roughly 9% faster.\n\nThen I ran the image.\n\nThe existing path had a median of 9.5556 seconds.\n\nThe MPSGraph version came in at 9.6533.\n\nIt was 1.02% slower.\n\nSo I deleted it.\n\nThat experiment also produced one of the more interesting crashes during development. My first implementation synchronously dispatched onto PyTorch's Metal queue and then called an MPSGraph function that synchronously entered the same queue again.\n\nmacOS killed it with:\n\n```\ndispatch_sync called on queue already owned by current thread\n```\n\nAfter fixing the nested dispatch, the graph worked correctly.\n\nIt was still slower.\n\nThat distinction is important. Correct code isn't necessarily useful code.\n\nPyTorch's individual MPS convolutions are already pretty good. The graph overhead ate the dispatch savings, particularly because the largest spatial blocks, where most of the actual work happens, didn't improve enough.\n\n**Microbenchmarks nominate changes. Full generations elect them.**\n\n## What actually survived\n\nThe resulting performance diff is surprisingly small.\n\nThe implementation is four code commits beyond the Automatic1111 `dev`\n\nbase I started from and touches 20 of 329 tracked paths.\n\nThe pieces that survived were:\n\n- shape-selective Metal Flash Attention\n- deferred Metal command-buffer submission\n- unified-memory-aware attention routing\n- dynamic query chunk sizing\n- streaming online softmax\n- removal of obsolete MPS copies and FP32 detours\n- direct Metal matmul preference\n- less sampling upcasting\n- fused GroupNorm plus SiLU\n- NGMS for the tuned low-CFG workflow\n\nUnsupported inputs still fall back to PyTorch.\n\nThe native extensions also self-test at startup. They're built against the active Python/PyTorch environment, validated in a subprocess, and only enabled if the tests pass.\n\nThat subprocess is more important than it sounds. Native GPU code doesn't always politely throw a Python exception when something goes wrong. Sometimes it takes the interpreter with it.\n\nI'd rather lose the optimization than lose the WebUI.\n\n## The results\n\nOn the M3 Pro, the broad workload range that motivated this project moved from roughly **8–10 seconds to 3–7 seconds**.\n\nI'm intentionally calling that an observed range rather than a controlled benchmark. It spans workloads, and I don't have enough matched M3 Pro runs yet to pretend every second can be attributed cleanly.\n\nI do have a cleaner development comparison from an M1 Mac mini.\n\nUsing the same model hash, tensor shape, sampler, schedule, step count, CFG, dimensions, Clip skip, and NGMS configuration:\n\n| Build | Time |\n|---|---|\nAutomatic1111 `v1.10.1-96-g1937682a` | 12.8 s |\nMetal fork `v1.10.1-99-g38ac556a` | 8.7 s |\n\nThat's about 32% lower latency, or roughly 1.47x the generation throughput.\n\nThe seeds differed, so I'm treating this as a matched compute-shape throughput comparison rather than an image-parity test. The later fused GroupNorm plus SiLU work also came after this particular comparison.\n\nOne other number is worth repeating because it describes the development process better than the winning benchmark does:\n\n| Implementation | Median |\n|---|---|\n| Existing PyTorch/Metal path | 9.5556 s |\n| Experimental block MPSGraph | 9.6533 s |\n\nI spent time building the second one because the microbenchmarks said it should be faster.\n\nIt wasn't.\n\nSo it isn't in the fork.\n\n## Why this still isn't Draw Things\n\nDraw Things has a fundamental advantage here.\n\nIt can own the entire execution environment.\n\nIt can design model representation, graph execution, memory lifetime, precision, scheduling, and UI behavior around Apple hardware.\n\nAutomatic1111 can't do that without giving up much of what makes Automatic1111 useful.\n\nIt's dynamic Python software. People monkey patch it. Extensions hook into it. Models and LoRAs get swapped while it's running. ControlNet gets inserted. Users run different VAEs, different model families, different resolutions, different attention implementations, and all sorts of configurations I haven't thought about.\n\nThat's the ecosystem I wanted to keep.\n\nThis project instead targets the seams where native Apple execution can enter and leave without requiring Automatic1111 to become a different application.\n\nWe've gotten a meaningful amount of performance that way.\n\nThere is probably more available, but the next gains get harder.\n\nI want better stage-level timing around prompt encoding, the UNet, VAE decode, and postprocessing. Mixed-precision VAE decoding is interesting. Channels-last layouts across the UNet are interesting. A static whole-UNet MPSGraph is interesting too, but at that point we're getting much closer to maintaining a second execution engine.\n\nThat's a different tradeoff.\n\nFor now, this is still Automatic1111.\n\nSame checkpoints. Same LoRAs. Same UI. Same extensions. Same general workflow.\n\nIt just spends a lot less time waiting at the boundaries between Python, PyTorch, MPSGraph, and Metal.", "url": "https://wpnews.pro/news/automatic1111-for-apple-metal-40-speed-up-sd1-5", "canonical_source": "https://therad.ninja/from-8-10-seconds-to-3-7-teaching-automatic1111-to-speak-metal-on-an-m3-pro/", "published_at": "2026-08-12 13:41:12+00:00", "updated_at": "2026-08-12 13:42:08.699240+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "developer-tools"], "entities": ["Automatic1111", "Draw Things", "Apple M3 Pro", "Stable Diffusion", "Metal Flash Attention", "PyTorch", "Derek Anderson"], "alternates": {"html": "https://wpnews.pro/news/automatic1111-for-apple-metal-40-speed-up-sd1-5", "markdown": "https://wpnews.pro/news/automatic1111-for-apple-metal-40-speed-up-sd1-5.md", "text": "https://wpnews.pro/news/automatic1111-for-apple-metal-40-speed-up-sd1-5.txt", "jsonld": "https://wpnews.pro/news/automatic1111-for-apple-metal-40-speed-up-sd1-5.jsonld"}}