# mistral.rs v0.9.0 outpaces llama.cpp on CPU > Source: > Published: 2026-07-08 00:00:00+00:00 ## What mistral.rs shipped The Rust-built local inference engine mistral.rs released version 0.9.0 on 7 July, claiming up to 1.8× faster CPU decoding than llama.cpp on both x86 and ARM hardware. The team says the speedup holds at every context depth measured. ([SHUO Blog summary](https://blog.shuochen.me/en/news/2026-07-08-ai-news-summary/)) That matters because llama.cpp has been the de-facto standard for CPU-only local LLM inference for years. Anyone running models on a MacBook, a fanless mini PC, or an old workstation without a discrete GPU has been routing through llama.cpp — local AI runtimes like [Ollama](/articles/ollama-v024-codex-and-apple-silicon/) and [LM Studio](/articles/lm-studio-vs-ollama-2026/) sit on top of it. A genuine second-place engine that beats the default at decode speed is a real shift, not marketing. The release is also the first widely-discussed inference-engine release of the summer that treats ARM (Apple Silicon, Qualcomm Snapdragon X) and x86 as equal citizens. ([東リ屋 note](https://note.com/samehadaonsen/n/nce2654d377d3)) ## What the benchmark actually measured The headline number comes from one model: Qwen 3 4B, tested on x86 and ARM hardware. The team says the speedup is *general* — they optimised at granular levels, not for one architecture. ([SHUO Blog summary](https://blog.shuochen.me/en/news/2026-07-08-ai-news-summary/)) That is an honest framing and a narrow one. The 4B class is where most CPU users actually live — a quantised 4B model fits in a few gigabytes of RAM, runs cold on almost anything, and is the default *laptop model* for a lot of people. But it is not where larger agentic workflows live. The LocalLLaMA coverage flags that 27B-class performance, like the [Qwen 3.6 27B](/articles/qwen-3-6-27b-holds-its-own/) most self-hosters run, is still unverified, and that different quantisations have not been benchmarked. ([東リ屋 note](https://note.com/samehadaonsen/n/nce2654d377d3)) Community response in the LocalLLaMA thread has been positive on the speedup and cautious on the scope. One summary of the discussion: *4B-class speedups are welcome, but it is still to be confirmed whether the same gains hold at 27B.* ([東リ屋 note](https://note.com/samehadaonsen/n/nce2654d377d3)) 1.8×claimed CPU-decode speedup over llama.cpp, at every context depth the mistral.rs team measured ## Where this fits in the local stack mistral.rs reads GGUF model files — the same quantised-model format used by llama.cpp and most local runtimes — so swapping engines is mostly a matter of pointing the runtime at a different binary. Anyone running Ollama or LM Studio today is closer to mistral.rs than they think: drop the engine in, keep the weights. For a UK team running a small model on a spare MacBook, decode speed is the rate limit on the whole workflow. Faster decode means more turns per minute, longer agent loops, and less waiting on a streaming response. The win is biggest where the GPU is weakest: older Intel laptops, fanless mini PCs, the second-hand Dell workstation gathering dust under a desk. It is also where most of the *sovereign*, *private*, *on-prem* UK use cases actually sit — procurement does not want to buy a Blackwell rack to run a summariser, but does want a model that does not phone home. A 1.8× CPU speedup makes that case easier to defend in a tender. ## How to try it this week For a UK team with a tinkerer in the corner of the room, the move is a low-risk side-by-side test, not a migration. Pick one model you already run on a CPU-only box — the obvious candidates are a small Qwen 3 at a 4-bit quantisation — and time ten decodes at long context with both engines. **If you are on llama.cpp via Ollama:** install mistral.rs alongside (Rust toolchain, or pre-built binaries) and run the same GGUF through both. The interface differs from Ollama, so budget an hour.**If you are on an Apple Silicon MacBook:** this is where the gain is most likely to land. Most self-hosters we hear from are decode-bound on M-series chips, and ARM NEON is where mistral.rs has the strongest published result.**If you are on a 27B-or-larger workflow:** wait. The benchmark does not cover your case, and*1.8× faster*is the wrong number to plan around until it does. The bigger signal is that CPU inference is no longer a one-engine field. llama.cpp’s lead has looked unassailable for two years. With mistral.rs pushing 1.8× on the most common local model, the safe assumption is that the gap closes further by the end of summer — and that *local AI* stops being a synonym for *llama.cpp* by autumn. ## Sources & quotes Every quotation in this article is verbatim from a named source — click any 1 to see where it came from. It's part of how we keep an AI-run newsroom honest. [How we verify →](/blog/how-we-keep-an-ai-newsroom-honest/)