{"slug": "apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp", "title": "Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with Llama.cpp", "summary": "A compatibility layer for Apple's Virtualization.framework released by Cua under a permissive license accelerates LLM inference in macOS virtual machines by 11.08× for prompt processing and 16.36× for token generation on an M1 Ultra running TinyLlama 1.1B via llama.cpp, reaching 98% of bare-metal prompt speed. With Google's Gemma 4 12B QAT Q4_0, the layer improved prompt processing 7.20× and token generation 14.54×, achieving 99.59% of bare-metal prompt speed and 94.82% of bare-metal generation speed. The layer intercepts Metal capability queries to enable newer GPU fast paths inside macOS guests.", "body_md": "*Published on August 11, 2026 by Francesco Bonacci and Johnny Franks*\n\nIf you've been following Cua from the start, you may remember that it began with a [Show HN](https://news.ycombinator.com/item?id=42908061) launch for Lume, our macOS virtualization stack.\n\nToday, we're sharing the first result from a broader effort to connect that `Virtualization.framework`\n\nfoundation to the local computer-use environments behind [Cua Driver](https://cua.ai/docs/tutorials/drive-your-first-app) and the infrastructure behind [Cua Cloud and Fleets](https://cua.ai/signup?redirect_url=%2Fwaitlist): a small, process-scoped compatibility layer that unlocks newer Metal fast paths inside a macOS guest.\n\nWe're releasing this work today as a research release under the same permissive license as Lume and Cua, so others can reproduce the results and help map which Apple Silicon chips, macOS releases, and Metal workloads benefit.\n\nApple Vz users have been running into these limitations elsewhere too. Tart, another notable CLI built on Apple's `Virtualization.framework`\n\n, has an open [“No GPU passthrough in macOS guest?”](https://github.com/openai/tart/issues/1032) issue asking whether the framework can provide usable graphics and decent LLM performance in a macOS VM guest. The VM continues to use the virtual GPU that Apple provides. Our work exposes newer Metal paths on that device and closes part of the practical gap.\n\nOn an M1 Ultra, TinyLlama 1.1B running through llama.cpp processed prompts **11.08× faster** and generated tokens **16.36× faster** than the same workload in the same stock VM. Prompt processing reached 98% of our bare-metal result. The source, build scripts, capability probe, and raw benchmark logs are included so you can inspect and reproduce the result.\n\nWe repeated the experiment with Google's [Gemma 4 12B QAT Q4_0](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf), a 6.98 GB model released this year. The same layer improved prompt processing **7.20×** and token generation **14.54×**. The unlocked VM reached 99.59% of bare-metal prompt speed and 94.82% of bare-metal generation speed.\n\nApple's `Virtualization.framework`\n\npresents a macOS guest with a [virtual graphics device](https://developer.apple.com/documentation/virtualization/vzmacgraphicsdeviceconfiguration). The guest submits Metal work through a purpose-built GPU driver, and Apple's host stack executes it on the physical GPU. This arrangement is paravirtualization, where the host keeps control of the hardware and the guest uses a virtualization-aware device.\n\nThis differs from other virtualization stacks built on QEMU and KVM, which can use a different architecture. On x86 Linux hosts, [VFIO](https://www.kernel.org/doc/html/latest/driver-api/vfio.html) can assign a compatible physical PCI device or hardware function to a VM through an IOMMU, giving the guest direct access to that device. This is the model usually meant by GPU passthrough.\n\nIn our stock Tahoe VM, the paravirtualized device reported roughly an Apple 5-era family, 32 KB of maximum threadgroup memory, and SIMD-group matrix support as unavailable. Modern Metal software uses those answers to select kernels, so llama.cpp took a slower path even though the device could execute newer kernels.\n\nApple documents GPU capability through [GPU families and feature tables](https://developer.apple.com/metal/capabilities/) and recommends [querying the device at runtime](https://developer.apple.com/documentation/metal/detecting-gpu-features-and-metal-software-versions). That makes the reported capability boundary consequential: applications are doing exactly what the platform tells them to do.\n\nWe built a small Metal capability shim (a compatibility layer inserted between an application and an API) that runs inside one guest process. It intercepts selected Metal capability queries and changes the answers returned to that process. Metal applications use those answers to select kernels, so returning the tested Apple-family and threadgroup-memory values lets llama.cpp choose its newer GPU paths. For our tested profile, the shim:\n\n- answers\n`supportsFamily:`\n\nthrough Apple family 9 (`1009`\n\n); and - raises the reported maximum threadgroup memory from 32 KB to 64 KB.\n\nThat was enough for the tested llama.cpp build to select newer SIMD-group reduction, SIMD-group matrix, and bfloat16 paths:\n\n| Capability | Stock guest | Tested profile |\n|---|---|---|\n`supportsFamily:1009` |\nfalse | true |\n| SIMD-group matrix | off | on |\n| SIMD-group reduction | off | on |\n| bfloat16 | off | on |\n| Maximum threadgroup memory | 32 KB | 64 KB |\n\nThe tested profile changes two reported values: Apple-family answers and the threadgroup-memory limit. Common, Mac, Metal, and working-set-size values keep their stock settings during the benchmark. We removed the original research hook's private feature-profile hook, clock and timing interposition, mesh substitution, ray-tracing override, argument-layout guard, and pipeline-compilation fallback. Its source is small enough to audit, and malformed or missing configuration keeps the process on its stock capability path.\n\nThe workload stays on Apple's `Virtualization.framework`\n\ngraphics path and executes on the host's Apple GPU. The capability changes are scoped to the injected guest process.\n\nPhysical GPU assignment, raw PCI or VFIO passthrough, and kernel changes sit outside this mechanism. A reported family describes the paths covered by our tests; each additional Metal API requires separate validation.\n\nThe shim unlocks Metal capabilities on Apple's existing virtual GPU path. VM users often encounter the broader limitation under the name “GPU passthrough.”\n\nWe tested on one Apple M1 Ultra with a 48-core GPU and macOS 26.6.1. The guest was the current public Tahoe Cua image (macOS 26.5.2, 8 vCPU, and 16 GiB) running in Lume 0.5.1. All three runs used the official llama.cpp `b10167`\n\nrelease and the same TinyLlama 1.1B Chat Q4_K_M model.\n\nThe command was:\n\n```\nllama-bench -m tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf \\\n  -p 512 -n 128 -r 10 -t 8 -ngl -1 -o json\n```\n\nValues below are medians of the ten samples emitted for each benchmark row:\n\n| Workload | Bare-metal host | Stock guest | Unlocked guest |\nGuest speedup | Unlocked / host |\n|---|---|---|---|---|---|\n| Prompt processing, 512 tokens | 4,871.99 tok/s | 431.86 tok/s | 4,786.70 tok/s |\n11.08× |\n98.25% |\n| Token generation, 128 tokens | 286.71 tok/s | 12.63 tok/s | 206.60 tok/s |\n16.36× |\n72.06% |\n\nPrompt processing nearly reached the host result. Generation reached 72.06% of host speed, leaving a measurable VM gap. The gain depends on the host GPU, guest version, application, and workload shape.\n\nThe [TinyLlama raw results and environment record](https://github.com/trycua/cua/tree/main/evidence/lume-metal-capability-shim/2026-08-09-m1-ultra) include the exact image digest, model and binary hashes, commands, JSON output, stderr, and checksums. These release-candidate results certify the reduced shim used in this post.\n\nTinyLlama makes a useful controlled benchmark because it runs quickly and exposes the Metal path clearly. We also wanted a larger model that developers might choose today, so we ran Google's official Gemma 4 12B instruction-tuned QAT Q4_0 GGUF through the same llama.cpp binary.\n\nThe host, VM, shim, benchmark shape, and ten-sample method stayed the same. We disabled speculative decoding and left the multimodal projector unloaded, keeping the comparison on the same Metal inference path:\n\n| Workload | Bare-metal host | Stock guest | Unlocked guest |\nGuest speedup | Unlocked / host |\n|---|---|---|---|---|---|\n| Prompt processing, 512 tokens | 517.88 tok/s | 71.66 tok/s | 515.76 tok/s |\n7.20× |\n99.59% |\n| Token generation, 128 tokens | 52.38 tok/s | 3.41 tok/s | 49.67 tok/s |\n14.54× |\n94.82% |\n\nThe [Gemma 4 evidence](https://github.com/trycua/cua/tree/main/evidence/lume-metal-capability-shim/2026-08-10-m1-ultra-gemma4) pins Google's model revision and SHA-256 alongside the final raw samples. We discarded and reran a preliminary stock series after detecting another host compute workload. The retained stock, unlocked, and bare-metal files come from the same uncontended window and show tight sample ranges.\n\nWe also tested [MLX-LM](https://github.com/ml-explore/mlx-lm) 0.31.3 with `mlx-community/Llama-3.2-3B-Instruct-4bit`\n\non MLX 0.32.0. Performance stayed flat because MLX-LM was already fast in the stock VM:\n\n| Workload | Stock guest | Unlocked guest | Ratio |\n|---|---|---|---|\n| Prompt processing, 512 tokens | 1,656.55 tok/s | 1,665.47 tok/s | 1.005× |\n| Token generation, 128 tokens | 172.09 tok/s | 170.86 tok/s | 0.993× |\n\nThat flat result helped define the release profile. During ablation, advertising `MTLGPUFamilyMetal3`\n\nmade MLX request a residency set unavailable through the paravirtualized device. The release shim limits changed answers to Apple-family enums and keeps Metal 3 at its stock value. The relevant MLX branch is visible in its [Metal residency implementation](https://github.com/ml-explore/mlx/blob/v0.32.0/mlx/backend/metal/resident.cpp).\n\nThis runs entirely on Apple hardware through the paravirtualized GPU path that Apple ships with `Virtualization.framework`\n\n. The shim affects selected values read by one guest process. The host, guest kernel, other guest processes, content-protection state, and licensing state keep their existing configuration.\n\nThe technique relies on private, version-sensitive behavior in the guest's Metal implementation. Apple may change it between macOS releases, so we test each host and guest combination independently. Unsupported methods keep the process on its stock path, and each additional API needs its own virtualization test.\n\nWe would welcome clarification from Apple on the intended behavior and supportability of the unrestricted feature level for paravirtualized graphics. Apple engineers working on Metal or `Virtualization.framework`\n\ncan reach us at [vz@trycua.com](mailto:vz@trycua.com).\n\nThe source lives in [ libs/lume/metal-capability-shim](https://github.com/trycua/cua/tree/main/libs/lume/metal-capability-shim). Build and verify both architecture-specific dylibs:\n\n```\ncd libs/lume/metal-capability-shim\n./Scripts/build.sh\n./Scripts/verify.sh\n```\n\nStop the VM, enable the unrestricted feature level for VMs launched by your macOS user, and restart it:\n\n```\nlume stop my-vm\ndefaults write com.apple.gpusw.ParavirtualizedGraphics \\\n  ForceUnrestrictedDeviceFeatureLevel -bool true\nlume run my-vm\n```\n\nCopy the matching dylib and the probe or workload into the guest, then scope activation to that process:\n\n```\nlume ssh my-vm \\\n  \"DYLD_INSERT_LIBRARIES=/path/to/LumeMetalCapabilities-arm64.dylib \\\n   LUME_METAL_APPLE_FAMILY_MAX=1009 \\\n   /path/to/metal-capabilities 1009\"\n```\n\nFor a long-running inference server, renderer, or worker, use a per-workload LaunchAgent. Set `DYLD_INSERT_LIBRARIES`\n\nin that workload's environment so the login session remains stock. The [Lume guide](https://cua.ai/docs/how-to-guides/lume/gpu-passthrough) has a complete template, checksum and verification steps, and rollback instructions.\n\nRemoving the environment variables and restarting the workload returns it to stock behavior. To restore the host preference, stop the VM, delete `ForceUnrestrictedDeviceFeatureLevel`\n\n, and start the VM again.\n\n**Experimental and version-sensitive.** The shim uses private guest Metal implementation details that can change in any macOS release.**Per-process.** It affects only the injected workload and its children; hardened or platform-protected executables may reject library injection.**Configured capability profile.** It reports the Apple-family values covered by our tests. Physical-GPU capability discovery remains outside its scope.**Narrow validation.** The current evidence covers the capability probe, two llama.cpp workloads, and one MLX-LM compatibility run on the listed M1 Ultra host and Tahoe guest. Additional chips, guest releases, models, and Metal APIs need separate tests.**Still a VM.** Existing`Virtualization.framework`\n\nrendering and virtualization limits remain.\n\nThe guest's conservative answers hid a surprisingly capable GPU path. On our test machine, two narrowly scoped capability changes moved TinyLlama prompt processing from 432 to 4,787 tokens per second. With Gemma 4 12B, prompt processing moved from 71.66 to 515.76 tokens per second and generation from 3.41 to 49.67 while the workload stayed on Apple's existing GPU bridge.\n\nLume started as a way to make macOS VMs practical for developers. This result gives us a foundation to test across more Apple Silicon generations, guest releases, and Metal workloads.\n\nWant to help? [Star Cua on GitHub](https://github.com/trycua/cua) and test the shim on your setup. [Open an issue](https://github.com/trycua/cua/issues/new/choose) with your host chip, host and guest versions, exact workload, and both stock and unlocked results. If you validate a new combination or improve the shim, [send a pull request](https://github.com/trycua/cua/pulls).", "url": "https://wpnews.pro/news/apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp", "canonical_source": "https://github.com/trycua/cua/blob/main/blog/gpu-passthrough-macos-vms.md", "published_at": "2026-08-11 14:50:33+00:00", "updated_at": "2026-08-11 15:13:01.708779+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools", "ai-infrastructure"], "entities": ["Cua", "Apple", "Virtualization.framework", "llama.cpp", "M1 Ultra", "TinyLlama 1.1B", "Google", "Gemma 4 12B QAT Q4_0"], "alternates": {"html": "https://wpnews.pro/news/apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp", "markdown": "https://wpnews.pro/news/apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp.md", "text": "https://wpnews.pro/news/apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp.txt", "jsonld": "https://wpnews.pro/news/apple-silicon-and-macos-vms-11-16x-faster-llm-inference-with-llama-cpp.jsonld"}}