QEMU vmapple boot macos 14 through 26

A developer has contributed a QEMU patch that lets the vmapple machine model boot macOS 14 and later guests, which previously stalled in the kernel's first instructions. The fix patches 19 "cbnz x0, ." sites following "hvc #0" pointer-authentication hypercalls to nops and appends a /filesystems/DoNotUseSEP node to the device tree, allowing macOS 13.6, 14.6.1, 15.6.1 and 26.6.2 guests to reach their desktops.

| | commit e1c37ea4d4a74fb6addcfeede9a34d521de44fa5 | | | Author: Anees Iqbal <hello@aneesiqbal.ai | | | Date: Thu Oct 1 05:31:30 2026 +0300 | | | hw/vmapple: boot macOS 14 and later guests | | | macOS 14 and later guests stop in their kernel's first instructions: | | | the kernel spins with "hvc 0; cbnz x0, ." on each of its pointer | | | authentication hypercalls, which the public Hypervisor.framework API | | | fails without an exit. macOS 15 and later also panic when APFS mounts | | | their unencrypted data volume "unencrypted data volume is not | | | allowed" , unless the device tree has /filesystems/DoNotUseSEP, and | | | reset in a loop. | | | Make two changes to the guest when iBoot enters the kernel: | | | - Replace every "cbnz x0, ." that follows an "hvc 0" in the kernel's | | | memory, from the boot arguments' physBase to their topOfKernelData, | | | with a nop, so the kernel runs on the firmware's keys, as macOS 13's | | | does. 14.6.1, 15.6.1 and 26.6.2 have 19 such sites; 13.6 has none. | | | - If the device tree has no /filesystems node, append one that has | | | DoNotUseSEP as the root's last child, in the zeroes that pad the tree | | | to its last 16 KiB page, and cover it with the tree's length in the | | | boot arguments. 13.6's tree has the node, and is left as it is. | | | To catch the kernel's entry, RAM is not executable from reset. Each page | | | iBoot runs becomes executable on its first fetch, until a fetch lands in | | | the kernel's memory with x0 at the boot arguments; only the boot CPU | | | runs then. HVF reports these faults through a new hook, | | | hvf arm exec fault, which a machine can set to handle a stage 2 fault | | | on an instruction fetch itself. | | | With this, 13.6, 14.6.1, 15.6.1 and 26.6.2 guests reach their desktops. | | | diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst | | | index 0994d6d0f9..b60ecf7942 100644 | | | --- a/docs/system/arm/vmapple.rst | | | +++ b/docs/system/arm/vmapple.rst | | | @@ -14,10 +14,9 @@ To run the vmapple machine model, you need to | | | Run on Apple Silicon | | | Run on macOS 12.0 or above | | | - Have an already installed copy of a Virtualization.Framework macOS 12 or 13 | | | - virtual machine. Newer versions are NOT supported on the guest side see | | | - Guest versions . I will assume that you installed it using the | | | - macosvm <https://github.com/s-u/macosvm CLI. | | | + Have an already installed copy of a Virtualization.Framework macOS 12 or | | | + later virtual machine see Guest versions . I will assume that you | | | + installed it using the macosvm <https://github.com/s-u/macosvm CLI. | | | Build QEMU with apple-gfx, which the machine's display needs. apple-gfx | | | drives reims-vgpu's C API, a library that pkg-config finds as | | | reims-vgpu : pass --enable-pvg with PKG CONFIG PATH naming the | | | @@ -73,21 +72,43 @@ to get better interactive access into the target system: | | | Guest versions | | | -------------- | | | -macOS 12 and 13 guests boot. A vmapple guest kernel asks the hypervisor to | | | -manage its pointer authentication keys through paravirtualized hypercalls: | | | - hvc 0 with SMCCC function IDs 0xC1000000 to 0xC10000FF. | | | +macOS 12 and 13 guests boot without changes. macOS 14 and later need the | | | +two below, which the machine makes when iBoot enters the kernel. | | | + | | | +A vmapple guest kernel asks the hypervisor to manage its pointer | | | +authentication keys through paravirtualized hypercalls: hvc 0 with | | | +SMCCC function IDs 0xC1000000 to 0xC10000FF. | | | Hypervisor.framework answers these calls itself, without an exit to QEMU, | | | and for a VM created through its public API it answers each of them as not | | | supported. The macOS 12 and 13 kernels carry on with the keys the firmware | | | set up, which all processes then share. | | | From macOS 14 on, the kernel checks the result of every one of these calls | | | -and loops forever when one fails. Such a guest stops right after iBoot hands | | | -over to the kernel. The serial console ends with iBoot's End of | | | -iBootStage2 serial output. line, and the boot CPU keeps a host core busy | | | -without exiting to QEMU. Hypervisor.framework implements the calls only for | | | -VMs created with a private ISA setting, which needs an entitlement that only | | | -Apple-signed binaries can hold. QEMU does not use it. | | | +and loops forever when one fails, with hvc 0; cbnz x0, . . | | | +Hypervisor.framework implements the calls only for VMs created with a | | | +private ISA setting, which needs an entitlement that only Apple-signed | | | +binaries can hold. QEMU does not use it. Instead, when iBoot enters the | | | +kernel, the machine replaces each of those loops with a nop in the memory | | | +iBoot loaded the kernel into, so the kernel carries on as macOS 13's does. | | | +This takes the per-process keys away from the guest's pointer | | | +authentication. Without it, such a guest stops right after iBoot hands over | | | +to the kernel: the serial console ends with iBoot's End of iBootStage2 | | | +serial output. line, and the boot CPU keeps a host core busy without | | | +exiting to QEMU. | | | + | | | +The machine has no Secure Enclave. From macOS 15 on, the guest's APFS then | | | +panics on its unencrypted data volume "unencrypted data volume is not | | | +allowed" unless the device tree's /filesystems node has a | | | + DoNotUseSEP property, and the guest resets and boots again, over and | | | +over. When iBoot enters the kernel, the machine also adds such a node to the | | | +device tree iBoot hands the kernel, if the tree has no /filesystems | | | +node. macOS 13's tree has one, which is left as it is. | | | + | | | +To catch iBoot entering the kernel, guest RAM is not executable from reset. | | | +Each page iBoot runs becomes executable on its first instruction fetch, until | | | +a fetch lands between the boot arguments' physBase and | | | + topOfKernelData with x0 at those boot arguments, as iBoot enters the | | | +kernel. All of RAM is executable from then on. | | | Display | | | ------- | | | diff --git a/hw/vmapple/vmapple.c b/hw/vmapple/vmapple.c | | | index c0306bcf52..da11b91132 100644 | | | --- a/hw/vmapple/vmapple.c | | | +++ b/hw/vmapple/vmapple.c | | | @@ -16,6 +16,8 @@ | | | include "qemu/osdep.h" | | | include "qemu/bitops.h" | | | + include "qemu/cacheflush.h" | | | + include "qemu/cutils.h" | | | include "qemu/datadir.h" | | | include "qemu/error-report.h" | | | include "qemu/guest-random.h" | | | @@ -47,11 +49,13 @@ | | | include "qobject/qlist.h" | | | include "standard-headers/linux/input.h" | | | include "system/hvf.h" | | | + include "system/hvf int.h" | | | include "system/reset.h" | | | include "system/runstate.h" | | | include "system/system.h" | | | include "target/arm/gtimer.h" | | | include "target/arm/cpu.h" | | | + include "target/arm/hvf arm.h" | | | struct VMAppleMachineState { | | | MachineState parent; | | | @@ -457,12 +461,157 @@ static void create pcie VMAppleMachineState vms | | | } | | | } | | | +/ | | | + macOS 14 and later need two changes to the kernel's memory to boot here, | | | + made when iBoot enters the kernel. From reset, RAM is not executable: each | | | + page iBoot runs becomes executable on its first instruction fetch, until a | | | + fetch lands in the kernel's memory with x0 at the boot arguments. Only the | | | + boot CPU runs then. The changes leave macOS 13 as it was. | | | + / | | | + | | | +/ A host pointer to pa, pa + len , if guest RAM holds all of it. / | | | +static uint8 t vmapple ram VMAppleMachineState vms, uint64 t pa, | | | + uint64 t len | | | +{ | | | + MachineState ms = MACHINE vms ; | | | + uint64 t off = pa - vms- memmap VMAPPLE MEM .base; | | | + | | | + if pa < vms- memmap VMAPPLE MEM .base \|\| off ms- ram size \|\| | | | + len ms- ram size - off { | | | + return NULL; | | | + } | | | + return uint8 t memory region get ram ptr ms- ram + off; | | | +} | | | + | | | +/ | | | + Hypervisor.framework fails the kernel's pointer authentication hypercalls, | | | + and from macOS 14 on the kernel spins on each failure with | | | + "hvc 0; cbnz x0, .". Take out the spins: the kernel then runs on the | | | + firmware's keys, as macOS 13's does. | | | + / | | | +static void vmapple skip pac waits uint8 t mem, uint64 t size | | | +{ | | | + for uint64 t off = 4; off + 4 <= size; off += 4 { | | | + if ldl le p mem + off == 0xb5000000 && / cbnz x0, . / | | | + ldl le p mem + off - 4 == 0xd4000002 { / hvc 0 / | | | + stl le p mem + off, 0xd503201f ; / nop / | | | + flush idcache range uintptr t mem + off, uintptr t mem + off, | | | + 4 ; | | | + } | | | + } | | | +} | | | + | | | +/ | | | + A device tree node: a property count, a child count, its properties, then | | | + its children. A property is a 32-byte name, a 32-bit length whose top bit | | | + is a flag, and its value, padded to 4 bytes. | | | + / | | | +static const struct QEMU PACKED { | | | + uint32 t props, children; | | | + struct QEMU PACKED { | | | + char name 32 ; | | | + uint32 t len; | | | + char value 12 ; | | | + } name; | | | + struct QEMU PACKED { | | | + char name 32 ; | | | + uint32 t len, value; | | | + } no sep; | | | +} vmapple filesystems = { | | | + const le32 2 , 0, | | | + { "name", const le32 12 , "filesystems" }, | | | + { "DoNotUseSEP", const le32 4 , const le32 1 }, | | | +}; | | | + | | | +/ | | | + The machine has no Secure Enclave, and from macOS 15 on APFS panics on the | | | + unencrypted data volume unless the device tree has /filesystems/DoNotUseSEP. | | | + If the tree has no /filesystems, append that node as the root's last child, | | | + in the zeroes that pad the tree to its last 16 KiB page, and cover it with | | | + the tree's length in the boot arguments. | | | + / | | | +static void vmapple no sep VMAppleMachineState vms, uint8 t args | | | +{ | | | + / deviceTreeP - virtBase + physBase, and deviceTreeLength / | | | + uint64 t pa = ldq le p args + 0x60 - ldq le p args + 8 + | | | + ldq le p args + 16 ; | | | + uint64 t len = ldl le p args + 0x68 , end = 0, nodes, props; | | | + uint64 t room = ROUND UP pa + len, 16 KiB - pa; | | | + uint64 t size = sizeof vmapple filesystems ; | | | + uint8 t dt = vmapple ram vms, pa, room ; | | | + | | | + if dt { | | | + return; | | | + } | | | + / The tree's end: its nodes come in order, each before its children / | | | + for nodes = 1; nodes; nodes-- { | | | + if end + 8 len { | | | + return; | | | + } | | | + props = ldl le p dt + end ; | | | + nodes += ldl le p dt + end + 4 ; | | | + for end += 8; props; props-- { | | | + if end + 36 len { | | | + return; | | | + } | | | + end += 36 + ROUND UP ldl le p dt + end + 32 & INT32 MAX, 4 ; | | | + } | | | + } | | | + if end + size room \|\| buffer is zero dt + end, size \|\| | | | + memmem dt, end, &vmapple filesystems.name, | | | + sizeof vmapple filesystems.name { | | | + return; | | | + } | | | + memcpy dt + end, &vmapple filesystems, size ; | | | + stl le p dt + 4, ldl le p dt + 4 + 1 ; | | | + stl le p args + 0x68, MAX len, end + size ; | | | +} | | | + | | | +static bool vmapple exec fault CPUState cpu, hwaddr ipa | | | +{ | | | + VMAppleMachineState vms = VMAPPLE MACHINE qdev get machine ; | | | + uint64 t x0, base = 0, top = 0; | | | + uint8 t args, kernel; | | | + | | | + if vmapple ram vms, ipa, 1 { | | | + return false; | | | + } | | | + / XNU's boot args, version 2: physBase and topOfKernelData / | | | + assert hvf ok hv vcpu get reg cpu- accel- fd, HV REG X0, &x0 ; | | | + args = vmapple ram vms, x0, 0x70 ; | | | + if args && lduw le p args + 2 == 2 { | | | + base = ldq le p args + 16 ; | | | + top = ldq le p args + 32 ; | | | + } | | | + kernel = vmapple ram vms, base, top - base ; | | | + if ipa < base \|\| ipa = top \|\| kernel { | | | + / Still in iBoot / | | | + assert hvf ok hv vm protect ipa & qemu real host page mask , | | | + qemu real host page size , | | | + HV MEMORY READ \| HV MEMORY WRITE \| | | | + HV MEMORY EXEC ; | | | + return true; | | | + } | | | + vmapple no sep vms, args ; | | | + vmapple skip pac waits kernel, top - base ; | | | + hvf arm exec fault = NULL; | | | + assert hvf ok hv vm protect vms- memmap VMAPPLE MEM .base, | | | + MACHINE vms - ram size, | | | + HV MEMORY READ \| HV MEMORY WRITE \| | | | + HV MEMORY EXEC ; | | | + return true; | | | +} | | | + | | | static void vmapple reset void opaque | | | { | | | VMAppleMachineState vms = opaque; | | | hwaddr base = vms- memmap VMAPPLE FIRMWARE .base; | | | cpu set pc first cpu, base ; | | | + hvf arm exec fault = vmapple exec fault; | | | + assert hvf ok hv vm protect vms- memmap VMAPPLE MEM .base, | | | + MACHINE vms - ram size, | | | + HV MEMORY READ \| HV MEMORY WRITE ; | | | } | | | static void mach vmapple init MachineState machine | | | diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c | | | index 17064abba8..febeba2a80 100644 | | | --- a/target/arm/hvf/hvf.c | | | +++ b/target/arm/hvf/hvf.c | | | @@ -332,6 +332,8 @@ typedef struct ARMHostCPUFeatures { | | | static ARMHostCPUFeatures arm host cpu features; | | | +bool hvf arm exec fault CPUState cpu, hwaddr ipa ; | | | + | | | struct hvf reg match { | | | int reg; | | | uint64 t offset; | | | @@ -2797,6 +2799,10 @@ static int hvf handle exception CPUState cpu, hv vcpu exit exception t excp | | | trace hvf insn abort env- pc, set, fnv, ea, s1ptw, ifsc ; | | | + if hvf arm exec fault && | | | + hvf arm exec fault cpu, excp- physical address { | | | + break; | | | + } | | | / fall through / | | | } | | | default: | | | diff --git a/target/arm/hvf arm.h b/target/arm/hvf arm.h | | | index 59e19f6e84..643455570a 100644 | | | --- a/target/arm/hvf arm.h | | | +++ b/target/arm/hvf arm.h | | | @@ -23,6 +23,12 @@ void hvf arm init debug void ; | | | void hvf arm set cpu features from host ARMCPU cpu ; | | | +/ | | | + If set, called on a stage 2 fault on an instruction fetch, with the fault's | | | + IPA. Returns true if it handled the fault. | | | + / | | | +extern bool hvf arm exec fault CPUState cpu, hwaddr ipa ; | | | + | | | / | | | We need access to types from macOS SDK =15.2, so expose stubs if the | | | headers are not available until we raise our minimum macOS version. |