macOS VMs Have Been Sandbagging Your GPU

Cua's Metal capability shim recovers up to 16.36× llama.cpp throughput in macOS virtual machines by unmasking GPU capabilities that Apple's paravirtualized graphics device underreports. On an M1 Ultra, TinyLlama 1.1B Q4_K_M prompt processing jumped from 431.86 to 4,786.70 tokens/sec (11.08×) and generation from 12.63 to 206.60 tokens/sec (16.36×), while Gemma 4 12B QAT Q4_0 generation rose from 3.41 to 49.67 tokens/sec. The fix, which intercepts capability queries via DYLD_INSERT_LIBRARIES and a host-side defaults key, enables full-stack isolation for CI runners and sandboxed agents without sacrificing inference performance.

AI https://sourcefeed.dev/c/ai Article macOS VMs Have Been Sandbagging Your GPU Cua's Metal capability shim recovers up to 16x llama.cpp throughput that Apple's paravirtualized graphics was hiding. Rachel Goldstein https://sourcefeed.dev/u/rachel goldstein The most interesting thing about the 11–16× llama.cpp speedup Cua https://trycua.com published today is that no hardware got faster. The GPU inside a macOS virtual machine was always capable of these numbers. Apple's paravirtualized graphics device was just telling guests it wasn't — and llama.cpp https://github.com/ggml-org/llama.cpp , doing exactly what a well-behaved Metal app should do, believed it. Despite the "GPU passthrough" in the post's URL, this isn't passthrough at all. It's capability unmasking, and that distinction is what makes it both clever and fragile. A virtual GPU that undersells itself When you boot a macOS guest through Apple's Virtualization framework https://developer.apple.com/documentation/virtualization — the machinery under Cua's Lume, Tart, and most modern Mac CI runners — the guest doesn't see your M-series GPU. It sees a paravirtual Metal device that forwards work to the host. That device works, but it reports itself as roughly an Apple family 5 GPU with 32 KB of threadgroup memory: A13-era capabilities, on hardware that's several generations past that. llama.cpp keys its Metal kernel selection off exactly those queries. Told it's on an ancient GPU, it skips the SIMD-group matrix multiply, SIMD-group reduction, and bfloat16 paths and falls back to conservative kernels. The silicon underneath could run the fast paths the whole time; nothing ever asked it to. Cua's fix is a process-scoped shim, injected via DYLD INSERT LIBRARIES into the guest process, that intercepts those capability queries and answers with Apple family 9 and 64 KB of threadgroup memory. Paired with a host-side defaults key — com.apple.gpusw.ParavirtualizedGraphics ForceUnrestrictedDeviceFeatureLevel — that lifts the feature-level cap for VMs launched by your user, llama.cpp suddenly picks the kernels it would pick on bare metal. The before/after on an M1 Ultra 48-core GPU, macOS 26 host and guest is stark. TinyLlama 1.1B Q4 K M goes from 431.86 to 4,786.70 tokens/sec on prompt processing 11.08× and from 12.63 to 206.60 tokens/sec on generation 16.36× . Gemma 4 12B QAT Q4 0 jumps from 3.41 to 49.67 tokens/sec on generation — from unusable to genuinely interactive. Read those baselines again, though. 12.63 tokens/sec generating from a 1.1B model on an M1 Ultra is catastrophically slow. The headline multiplier measures how broken the stock configuration was, not how fast the fix is. And the comparison is stock VM versus unlocked VM — the post makes no claim of native parity, and virtualization overhead remains. Tellingly, Cua found MLX-LM's performance flat under the same shim: it was already fast in the stock VM, so the cliff is specific to how llama.cpp gates kernels on capability answers. Who actually runs models inside a macOS VM Fair question, because most local-inference people run llama.cpp directly on the host and never hit this. The answer is anyone who needs isolation on Mac hardware: CI runners, ephemeral dev environments, and — Cua's own business — sandboxed computer-use agents that need to click around a real macOS desktop without access to yours. Apple's license permits two macOS VMs per machine precisely for these workloads. Until now, that isolation carried a brutal tax on inference, and the workaround was architectural: run the model on the host, sandbox only the agent, and accept that your "isolated" workload has a chatty channel to an unsandboxed inference server. Unmasking the guest GPU means the whole stack — model included — can live inside the VM boundary. For agent evals, for CI jobs that exercise a local model, for letting an agent run AI-generated code next to its own brain, that's a real architectural simplification, not a benchmark stunt. There's an irony in where this leaves the ecosystem. Linux guests on macOS solved this problem earlier and more thoroughly: Venus/Vulkan through libkrun https://github.com/containers/libkrun reached 75–80% of native, and Red Hat's API-remoting work forwards GGML calls to a host-side Metal backend at near-native speed. Apple's own container project still has GPU support as an open discussion. Running a macOS guest on a macOS host — the most Apple-native virtualization stack imaginable — was, until today, the worst way to run a model on a Mac. The catch, and it's not small Everything here leans on private, version-sensitive behavior. The defaults key is an undocumented Apple preference. The shim answers capability queries based on reverse-engineered Metal internals that Apple can rearrange in any point release. DYLD INSERT LIBRARIES injection is rejected by hardened and platform-protected binaries, so this only works for processes you build and control — fine for llama.cpp, a non-starter for signed third-party apps. And the benchmark covers exactly one machine: an M1 Ultra on one macOS 26 build. Cua ships verify scripts for good reason. That defaults key is also the most damning detail in the post. Apple already built a switch called ForceUnrestrictedDeviceFeatureLevel . The conservative capability profile guests see is a policy choice — presumably lowest-common-denominator compatibility for older guests — not a technical wall. The HN thread asked why the framework undersells the host GPU at all, and there's no good answer. Apple could expose truthful capabilities, or a documented opt-in, tomorrow. Use it, don't build on it My read: this is real, reproducible, and worth deploying today for dev-loop and sandbox workloads — the mechanism is well explained, the before/after methodology is honest, and the scripts are in the open. If you're running Lume or any Virtualization.framework-based macOS guest for agent work, the setup is a build script, one defaults write on the host, and two environment variables in the guest. Twenty minutes, order-of-magnitude payoff. But treat it as a temporary unlock, not a foundation. Anything that couples your production path to undocumented preference keys and dylib injection into Metal's query surface will eventually lose an argument with a macOS update. The durable fix belongs to Apple, and the existence of that defaults key says they know it. Until Virtualization.framework stops lying to its guests, Cua's shim is the right kind of hack: small, inspectable, and honest about what it is. Sources & further reading - Apple Silicon and macOS VMs: 11-16x Faster LLM Inference with Llama.cpp https://github.com/trycua/cua/blob/main/blog/gpu-passthrough-macos-vms.md — github.com - Apple Silicon and macOS VMs: 11-16x Faster LLM Inference with Llama.cpp discussion https://news.ycombinator.com/item?id=49259339 — news.ycombinator.com - Reach native speed with macOS llama.cpp container inference https://developers.redhat.com/articles/2025/09/18/reach-native-speed-macos-llamacpp-container-inference — developers.redhat.com - GPU passthrough availability? apple/container discussion https://github.com/apple/container/discussions/62 — github.com Rachel Goldstein https://sourcefeed.dev/u/rachel goldstein · Dev Tools Editor Rachel has been embedded in the developer tooling ecosystem for nearly eight years, covering everything from IDE wars and package-manager drama to the quiet rise of AI-assisted coding. She has a soft spot for open-source maintainers and an unhealthy number of terminal emulators installed on a single laptop. Discussion 0 No comments yet Be the first to weigh in.