# The M5 Mac and Local LLMs: Apple's New Matmul Hardware Finally Attacks Prompt Processing > Source: > Published: 2026-08-27 00:23:11+00:00 **The short version:** the M5 is the first Apple chip that attacks the one thing Apple Silicon was bad at for local AI. Every GPU core now has a dedicated matrix-multiply unit (Apple calls it a Neural Accelerator), and Apple's own measurements show it makes **prompt processing 3.3 to 4.1 times faster** than M4. Token generation, the number most people watch, only improves about 20 to 27 percent, because that is still limited by memory bandwidth. The new M5 Max and M5 Ultra Mac Studio (announced August 25, shipping September 22) bring this to the high-memory tier, but no one has benchmarked those for local LLMs yet. Here is what is real, what is measured, and what to do about it. We have not tested M5 hardware first-hand. This synthesizes Apple's own MLX engineering data, the announcement specs, and owner reports, all cited. ## What is actually new in the M5 for local AI? For years, Apple Silicon had a split personality on local LLMs. It was excellent at *generating* tokens because it pairs a big unified memory pool with high bandwidth. It was frustrating at *reading* your prompt, the prefill step, because that is compute-heavy and Apple's GPUs lacked dedicated matrix hardware. Anyone who fed a long document or ran an agent hit the wall, summed up by one r/LocalLLaMA owner: "The M3 Ultra Mac Studio takes 10+ minutes on pre-fill for large models and contexts." The M5 changes exactly that piece. Apple's [M5 announcement](https://www.apple.com/newsroom/2025/10/apple-unleashes-m5-the-next-big-leap-in-ai-performance-for-apple-silicon/?ref=vettedconsumer.com) puts a Neural Accelerator, a dedicated matrix-multiplication unit, inside *every* GPU core, and claims "over 4x the peak GPU compute for AI compared to M4." That is not a bandwidth upgrade or more cores; it is the specific kind of hardware (matmul) that prefill is bottlenecked on. Our [prefill-vs-generation explainer](https://vettedconsumer.com/prompt-processing-vs-generation-why-your-box-is-fast-at-one-and-slow-at-the-other/) covers why those two phases stress a chip so differently. ## Does the M5 fix Apple's prompt-processing problem? The measured answer, from Apple's own engineers, is a clear yes for the base chip. Apple's [MLX-on-M5 writeup](https://machinelearning.apple.com/research/exploring-llms-mlx-m5?ref=vettedconsumer.com) benchmarked a MacBook Pro with M5 and 24GB against the M4, running Qwen models from 1.7B to a 30B MoE plus gpt-oss-20b. The result: **Time-to-first-token dropped 3.3x to 4.1x.** A prompt that took the M4 a minute now takes the M5 about fifteen seconds. For long-context reading, RAG, and agentic loops (which re-pay the prefill cost on every tool call, as documented in a detailed owner [disaggregation writeup](https://www.reddit.com/r/LocalLLaMA/comments/1ullaf8/team_red_and_green_union_for_disaggregated_prompt/?ref=vettedconsumer.com)), that is the difference between usable and painful. Owners saw it coming the moment Apple announced the M5 Max: "the pre fill was lacking and that is solved with Neural Accelerators aka matmul." ## Is the M5 faster at generating tokens? Only a little, and this is the caveat that matters. In the same Apple tests, token generation improved just **1.19x to 1.27x**. The reason is a rule we come back to constantly: generation is memory-bandwidth-bound, not compute-bound. Apple states it plainly: "Generating subsequent tokens is bounded by memory bandwidth, rather than by compute ability." The base M5's bandwidth rose from the M4's 120 GB/s to 153 GB/s, about 28 percent, and generation tracked it. All the new matmul horsepower sits mostly idle during decode, exactly as our [Bandwidth, Not TFLOPS](https://vettedconsumer.com/bandwidth-not-tflops-what-sets-your-local-llm-speed-and-why-the-newest-card-isnt-always-fastest/) piece predicts. So the framing to hold onto: the M5 makes your Mac feel dramatically more responsive on long prompts, and modestly faster at the words-per-second you watch scroll by. ## The new M5 Max and M5 Ultra Mac Studio: what we know On August 25, 2026, Apple [announced](https://www.apple.com/newsroom/2026/08/apple-introduces-m6-and-m5-ultra-for-a-big-leap-in-performance-and-ai-compute/?ref=vettedconsumer.com) the Mac Studio refresh that local-AI buyers had been waiting for, the machines that pair the M5's matmul GPU with serious memory. Vendor-stated specs: | Chip | Unified memory | Memory bandwidth (Apple) | GPU | Price / ship | |---|---|---|---|---| M5 Max | 36GB to 128GB | up to 614 GB/s | up to 40-core | from $2,499, Sept 22 | M5 Ultra | 96GB to 512GB | up to 1.2 TB/s | up to 80-core | from $5,499, Sept 22 (512GB late Oct) | Two things stand out for a local-LLM buyer. First, the M5 Ultra's **1.2 TB/s** is a real jump over the M3 Ultra's 819 GB/s, so it should be faster at both prefill (new matmul) and generation (more bandwidth), the first Mac that improves on both axes at once. Second, and less obvious, the **M5 Max's 614 GB/s is actually lower than the M3 Ultra's 819 GB/s**. An owner flagged this the day of the announcement: for pure token generation, a new M5 Max could decode *slower* than a two-year-old M3 Ultra, even though its prefill is far quicker. Bandwidth still rules generation, and the Max tier did not get the Ultra's bandwidth. The critical missing piece: **nobody has run local LLMs on these Studios yet**. They were announced August 25 and ship September 22. The only measured M5 local-inference data in existence is Apple's base-M5 MacBook Pro writeup above. Any "M5 Ultra vs DGX Spark" benchmark you see online right now is a projection, not a test. We will publish measured numbers when the machines are in real hands. ## M5 Mac Studio vs M3 Ultra: upgrade, buy, or wait? We literally ended our [M3 Ultra review](https://vettedconsumer.com/mac-studio-m3-ultra-the-local-ai-workhorse-buy-now-or-wait-for-m5/) on "buy now or wait for M5." The wait is over, and the answer splits by what you do. | You are... | Move | Why | |---|---|---| | Buying your first big-memory Mac for local AI | Wait for the M5 Ultra Studio | 1.2 TB/s plus matmul improves prefill and generation; the first Mac better on both | | On an M3 Ultra, mostly chatting / short prompts | Keep it | Generation is bandwidth-bound; your 819 GB/s still beats the M5 Max's 614 | | On an M3 Ultra, doing long-context / agentic work | Consider the M5 Ultra | The 3.3-4.1x prefill gain is exactly your pain point; wait for real Studio benchmarks first | | Wanting the cheapest 128GB Mac | M5 Max Studio ($2,499) | Faster prefill than any prior Mac; accept the lower decode bandwidth vs an Ultra | | Cross-shopping a Strix Halo / DGX Spark | Read the tradeoff first | Different price and bandwidth balance, see our | One more forward note: Apple also announced an M6 Mac mini (its first 2nm chip) the same day, and reporting points to an AI-focused M7 generation beyond it. The cadence is fast now, which is another reason to buy for what you need today rather than for a spec sheet. ## The bottom line The M5 is the most consequential Apple Silicon release yet for people running models at home, because it fixes the right thing. Apple Silicon never needed help generating tokens; it needed help reading prompts, and the per-core matmul units deliver a measured 3.3x to 4.1x on exactly that. The nuance to keep: generation gains are modest and bandwidth-bound, the M5 Max's bandwidth trails the older M3 Ultra, and the Studios are announced but not yet benchmarked for local LLMs. If you have an M3 Ultra and mostly chat, sit tight. If prefill latency is your daily frustration, the M5 Ultra Studio is the machine to watch, once someone actually measures it. Meanwhile, size any model against your current or prospective Mac in our [Can I run it? calculator](https://vettedconsumer.com/can-i-run-it/). ## Sources and how we researched this - Measured M5 inference data: Apple's ["Exploring LLMs with MLX on M5"](https://machinelearning.apple.com/research/exploring-llms-mlx-m5?ref=vettedconsumer.com)(MacBook Pro M5, 24GB; TTFT 3.3-4.1x, generation 1.19-1.27x vs M4; M4 120 GB/s vs M5 153 GB/s; the memory-bandwidth-bound generation quote). - Chip architecture and "4x AI compute" claim: Apple's [M5](https://www.apple.com/newsroom/2025/10/apple-unleashes-m5-the-next-big-leap-in-ai-performance-for-apple-silicon/?ref=vettedconsumer.com)and[M6 / M5 Ultra](https://www.apple.com/newsroom/2026/08/apple-introduces-m6-and-m5-ultra-for-a-big-leap-in-performance-and-ai-compute/?ref=vettedconsumer.com)newsroom releases (Neural Accelerator per GPU core). - Mac Studio specs, prices, ship dates: the August 25, 2026 Apple announcement as reported by [9to5Mac](https://9to5mac.com/2026/08/25/apple-unveils-next-generation-mac-studio-with-m5-max-and-m5-ultra/?ref=vettedconsumer.com)(M5 Max up to 614 GB/s, 128GB, $2,499; M5 Ultra up to 1.2 TB/s, 512GB, $5,499; Sept 22, 512GB late October). - Owner sentiment: the r/LocalLLaMA [M5 Pro/Max reaction thread](https://www.reddit.com/r/LocalLLaMA/comments/1rjqsv6/apple_unveils_m5_pro_and_m5_max_citing_up_to_4/?ref=vettedconsumer.com)and a detailed[disaggregated-prefill writeup](https://www.reddit.com/r/LocalLLaMA/comments/1ullaf8/team_red_and_green_union_for_disaggregated_prompt/?ref=vettedconsumer.com), quoted verbatim. - Comparison bandwidth figures (M3 Ultra 819 GB/s, M4 Max 546 GB/s) are Apple specs as documented in our prior Mac coverage. We have not tested M5 hardware first-hand, and no independent M5 Studio local-LLM benchmarks exist yet. *Related: **Mac Studio M3 Ultra: buy now or wait for M5** · **Prompt processing vs generation** · **Bandwidth, Not TFLOPS** · **Strix Halo vs the Mac, measured** · **Unified memory, explained*