# Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs > Source: > Published: 2023-09-02 21:23:55+00:00 # Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs As of macOS 12 (Monterey), Apple's [Virtualization framework](https://developer.apple.com/documentation/virtualization) has nice support for macOS guest virtual machines, but with severe limitations: For example you can't install a macOS guest on Intel Macs, install guests with newer versions of macOS than the host, copy and paste between the host and the guest, or install third party kernel extensions in the guest. As usual for Apple, the functionality they do support is nicely implemented, but they've left out so much that the result is only marginally useful -- at least compared to third-party implementations available on Intel Macs, like VMware Fusion and Parallels. I've been working on the last of these limitations, and have found a workaround. It's rather complex, but should be very useful for those developing kernel extensions for macOS on Apple Silicon, or (more likely) porting existing Intel kexts to Apple Silicon. Developing kernel extensions on bare metal is a pain -- not least because you might end up damaging your nice new Apple Silicon Mac. But doing it in a virtual machine isolates the potential damage to the VM itself -- at worst you'll only need to trash it and create a replacement. Parts of my workaround are interesting in themselves, even if you don't use them to run third party kernel extensions on macOS guest VMs. For example I've found a way to patch the VM's kernel cache (containing the kernel and built-in kernel extensions), which should be very useful in reverse engineering macOS. Most of the discoveries here are ones I've made on my own, by trial and error. But I had a very helpful starting point -- [NyanSatan's Virtual-iBoot-Fun project](https://github.com/NyanSatan/Virtual-iBoot-Fun). ## Prerequisites - An Apple Silicon Mac running macOS 12 or higher. Mine is a 2020 Mac Mini (Macmini9,1). - A decent virtual machine host that uses Apple's Virtualization framework. I use [UTM](https://github.com/utmapp/UTM/). - One or more decent disassemblers. I use [Ghidra](https://github.com/NationalSecurityAgency/ghidra) and [Hopper Disassembler](https://www.hopperapp.com/). I also installed [Nick Botticelli's ghidra-iboot plugin](https://github.com/nick-botticelli/ghidra-iboot) - [Tihmstar's img4tool](https://github.com/tihmstar/img4tool). - A decent hex editor. I use [Hex Fiend](https://hexfiend.com/). I change its default edit mode to "Overwrite" and its line number format to "Hexadecimal". - A binary diff tool. I use [VBinDiff](https://www.cjmweb.net/vbindiff/). - A calculator with support for radix modes. Apple's Calculator in Programmer mode will do. ## Background My workaround patches three iBoot modules and the VM's kernel cache. The iBoot modules load early in the macOS (and IOS) boot sequence, before the kernel. There are at least three variants of iBoot, used in three diffent "stages" -- Stage 0 (as I call it), Stage 1 and Stage 2. On bare metal, Stage 0 is implemented in hardware (in Apple's ["Secure Enclave"](https://support.apple.com/guide/security/secure-enclave-sec59b0b31ff/web)). Stage 1 (aka LLB) and Stage 2 (iBoot properly so called) are [implemented in software](https://eclecticlight.co/2022/01/05/booting-an-m1-mac-from-hardware-to-kexts-2-llb-and-iboot/), though only the Stage 2 iBoot exists (inside `iBoot.img4`) in the macOS file system. By design, the Secure Enclave Processor (SEP) is inaccessible to ordinary mortals. And on bare metal the LLB and iBoot modules are encrypted (using a key held in the SEP). But for a VM, the modules for all three stages are unencrypted. Stage 0 is implemented (on the host) in `/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin`. Stage 1 is embedded (in `img4` format) in an `AuxiliaryStorage` file (on the host) associated with the DMG file (with an `img` extension) that stores the VM's image. Stage 2 (inside `iBoot.img4`) exists in the VM's file system. None of the iBoot variants call any external functions, and none have any symbols. So they're difficult to decipher. But the fact that the Virtualization framework uses them without encryption is a golden opportunity to reverse engineer them, and understand them better. It's apparently common knowledge that the iBoot modules all contain a function to check the "digests" (DGST) of the many `img4` images that govern the macOS boot process (including LLB, `iBoot.img4` and the kernel cache itself). Each "digest" is a hash, used to ensure that none of these images has changed since they were "signed" by Apple. The source code for a similar function (`image4_validate_property_callback()`) is available [here](https://github.com/xerub/img4lib/blob/master/libvfs/vfs_img4.c#L1327). The images in question are those to be loaded at the next stage. So (among other things) Stage 0 iBoot checks the digest of the Stage 1 iBoot `img4` image, Stage 1 iBoot checks the digest of the Stage 2 iBoot `img4` file, and Stage 2 iBoot checks the digest of the `kernelcache` file. Aside from the file system itself, nothing checks the integrity of the stage 0 iBoot file (`AVPBooter.vmapple2.bin`). So all you need to stop these integrity checks is to [defeat the file system's protection](https://gist.github.com/macshome/15f995a4e849acd75caf14f2e50e7e98) and patch this function (call it `image4_validate_property_callback()`) at every "stage". As NyanSatan mentions, the standard approach is to patch out `image4_validate_property_callback()` completely -- to replace it with code that "returns 0". I find this causes problems, so my patch is more surgical -- I allow `image4_validate_property_callback()` to perform all its normal checks, and only patch its return value (to '0') just before it returns. Some time ago I discovered that [failed calls to `_validate_acm_context()` in the VM's `AppleVPBootPolicy.kext` stop the VM from creating an auxiliary kext cache](https://github.com/utmapp/UTM/issues/4026#issuecomment-1338282487). Since third-party kexts are stored here, this prevents them from loading in a Virtualization framework VM. The failed calls originate from two other functions in `AppleVPBootPolicy.kext` -- `_command_create_linked_manifest()` and `_command_update_local_policy_for_kcos()`. My workaround is to patch out the calls to `_validate_acm_context()` from both of these functions. This is a more "surgical" approach than patching out `_validate_acm_context()` itself, which is called from many additional functions in `AppleVPBootPolicy.kext`. But this isn't enough by itself. An auxiliary kext cache does get created (you can see it using `kmutil inspect`). But macOS still complains that your third-party kext needs to be rebuilt. To fix this you need to go back to the Stage 2 iBoot module (inside `iBoot.img4`). One of its purposes is to fill the "device tree" with all the appropriate devices. But it (or the variant available in a Virtualization framework VM) refuses to allow any "AuxKC" entries to be added to `chosen/memory-map` in a VM, even when the auxiliary kext cache is present. Fixing this requires finding the function that checks whether a given boot object (originally in `img4` format) should be processed, and patching out the call to it. Its first parameter is a [four-character code](https://en.wikipedia.org/wiki/FourCC) -- for example `illb`, `ibot`, `krnl` or `auxk`. `validate_boot_object()` (my name for this function) indirectly calls `image4_validate_property_callback()`. So if you patch out the former you don't need to patch the latter. At some point I'll describe how I found `validate_boot_object()`. To display the contents of `chosen/memory-map`, run the following command in a Terminal prompt: ``` ioreg -p IODeviceTree -n "memory-map" -w 0 -r -t ``` ## Settings Changes (Host and Guest) Some settings changes are required on the host by [How to Defang macOS System Protections](https://gist.github.com/macshome/15f995a4e849acd75caf14f2e50e7e98), which allows changes to `AVPBooter.vmapple2.bin.` - In System Settings, under Software Update, disable "Download new updates when available". Otherwise your host can become unbootable. - Boot into Recovery Mode, run Terminal and do the following: ``` csrutil disable csrutil authenticated-root disable ``` You also, of course, need to create a macOS guest VM. Make sure it uses the Virtualization framework. In [UTM](https://github.com/utmapp/UTM) this is accomplished by choosing "Virtualize" and then "macOS 12+". Terminal will be used heavily below. So, to avoid lots of annoying prompts to give it permission to access files, it's best to give the Terminal app "full disk access" on the guest (in Privacy and Security in System Settings). Then boot into Recovery Mode on the guest. On macOS 12 you'll need to use [my hack](https://github.com/utmapp/UTM/issues/3904#issuecomment-1100924393) to do this. - Run the Startup Security Utility. Choose Reduced Security and "Allow user management of kernel extensions from identified developers". - Run Terminal and do the following: - `csrutil disable`, then `y` to "Allow booting unsigned operating systems and any kernel extensions". ## Special Considerations Once You've Made the Changes Described Here Below you'll create (and boot from) an APFS snapshot on your host computer to accomodate changes to the Stage 0 iBoot module (`AVMBooter.vmapple2.bin`). You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched `AVPBooter.vmapple2.bin` it will simply refuse to start.) You also need to keep csrutil's "authenticated-root" disabled, and keep "Download new updates when available" disabled under Software Update. But don't use Software Update to update macOS on your host while it's booted from this custom snapshot. I don't *know* that this will cause trouble, but I expect it might. Use the following command to revert your snapshot (and `AVPBooter.vmapple2.bin`), then reboot your computer: ``` sudo bless --mount / --last-sealed-snapshot ``` Guest VMs running release versions of macOS 13 and up can be upgraded to newer versions of macOS. But before you do so you should revert this document's changes to the Stage 2 iBoot module (`iBoot.img4`) and the kernel cache (`kernelcache`): Boot into Recovery Mode, run the Startup Security Utility, and choose Full Security. Afterwards you'll need to go through all the following steps again, from scratch. Among other things you'll need to create new `iBoot.img4.org` and `kernelcache.org` files, and possibly also a new `LLB.img4.org` file. macOS 12 guests can't be upgraded. The Virtualization process (`com.apple.Virtualization.VirtualMachine`) crashes and your VM becomes unbootable. This is an Apple bug. I've seen it myself, and have seen it documented [here](https://github.com/utmapp/UTM/issues/5307). The same [is true for beta-version macOS 14 and macOS 15 guests](https://github.com/insidegui/VirtualBuddy/discussions/194), though the consequences are less severe. In this case your only option is to create a new guest VM directly from an IPSW file, then work through the following steps. ## Finding the Modules to be Patched ### iBoot Stage 0 (`AVPBooter.vmapple2.bin`) This one's on the host, and is easy to find. Copy it and rename the copy to `AVPBooter.vmapple2.bin.org`. ``` /System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin ``` ### iBoot Stage 1 (LLB) This one's also on the host, but is harder to find. By default, VMs created by [UTM](https://github.com/utmapp/UTM/) are stored in the following directory: ``` ~/Library/Containers/com.utmapp.UTM/Data/Documents ``` Each VM is stored in a "package" with the extension `.utm`. Inside the package, in its `Data` subdirectory, is a file named `AuxiliaryStorage`. LLB is embedded here, in `img4` format. It's followed immediately by an `img4` format image of the "logo". You'll need to get copies of both. If you've upgraded your guest VM at least once to a newer version of macOS, there will be *two* copies of the LLB image in the `AuxiliaryStorage` file, each followed by a "logo" image. The first LLB image is always at offset `0x24000`. The second, if it exists, should be at offset `0x224000`. To be sure you've found all the LLB images, and their correct locations, search (in your hex editor) on "illb" (the [four character code](https://en.wikipedia.org/wiki/FourCC) for the LLB image). Only one of these two sets of images is active. Each time you upgrade macOS, the "other" LLB/logo image set becomes active. The original installation uses the set at offset `0x24000`. The first upgrade switches to the one at offset `0x224000` (and leaves the set at offset `0x24000` unchanged). The second upgrade switches back to the one at offset `0x24000` and overwrites it (leaving the one at `0x224000` unchanged). And so forth. Here's how to find out which set is active. At offsets `0x4000` and `0x5000` in `AuxiliaryStorage` are two more file images, each starting with "HUFA". Each corresponds to one of the LLB/logo image sets. If you only have one LLB/logo image set, only one "HUFA" image exists, at offset `0x4000`. The structure of these "HUFA" images is as follows: ``` struct HUFA { unsigned char magic[4]; // Always "HUFA" (unknown meaning). uint32_t file_version; // Always '1'. uint32_t upgrade_count; // 'n' == which (re)installation of macOS // created this HUFA image and its // corresponding LLB/logo images. '1' // means the original installation. '2' // and up mean each subsequent upgrade. uint32_t LLB_offset; // Offset of corresponding LLB/logo images // from the start of the first HUFA image // at offset 0x4000 -- either 0x20000 or // 0x220000. uint32_t unknown[4]; unsigned char hash[32]; // Some kind of hash value -- type and // target unknown. unsigned char fill[4032]; // Filled with 0xff. }; ``` If there are two sets of LLB/logo images, the one that's active has the highest `upgrade_count`. This is the one you'll need to work on. - Open `AuxiliaryStorage` in a hex editor. I use [Hex Fiend](https://hexfiend.com/), and will tailor my steps to it specifically. - Jump to the offset of the active set of LLB/logo images -- `0x24000` or `0x224000`. If an LLB image exists at this location, its first two bytes should be `3083`. - The `img4` image is in DER format (used by ASN1), so the first five bytes are `3083` followed by a three digit hexadecimal number in big endian format (for example `037FD1`). This number is the length of the image, exclusive of the length of its header. So the total length is `0x37FD6`. - Select the first five bytes of the image, then choose "Extend Selection". In this case, you'd extend it by `0x37FD1` bytes. Scroll down to the end of the selection (without disturbing it) and check that it's in the correct location (just before the "logo" image). If it is, CMD-C to copy the image, CMD-N to open a new window, CMD-V to paste in its contents, and save the file as `LLB.img4.org`. - The header for the "logo" image should be `3082` followed by a two digit hexdecimal number (for example `36DA`). Select the first four bytes, extend the selection by `0x36DA` bytes and scroll down to its end. The following bytes should be a bunch of NULLs (`00`). CMD-C to copy the image, CMD-N to open another new window, CMD-V to paste in its contents, and save this second new file as `logo.img4.org`. ### iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`) These modules are stored in files on the VM. So run the VM and do the following in it, at a Terminal prompt: - Run `kmutil inspect`, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case. ``` /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache ``` - Note the long hexadecimal number just before `/System/Library/Caches` -- in this case `FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83`. This is the "Next Stage Image4 Hash (nsih)", observable in the output of `sudo bputil -d`. - Copy this file and rename it to `kernelcache.org`. - `cd /` and `sudo find . -name iBoot.img4 -exec ls -al \{\} \;`. There will be at least two hits. Choose the one whose path contains the Next Stage Image4 Hash. Copy it and rename the copy to `iBoot.img4.org`. You might find the original `iBoot.img4` here, for example: ``` /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4 ``` ## Patching These Modules ### iBoot Stage 0 (`AVPBooter.vmapple2.bin.org`) For this I use [Ghidra](https://github.com/NationalSecurityAgency/ghidra) with the [ghidra-iboot plugin](https://github.com/nick-botticelli/ghidra-iboot). Copy `AVPBooter.vmapple2.bin.org` to `AVPBooter.vmapple2.bin`. Then run Ghidra and create a new project. Then: - Run Ghidra's CodeBrowser and import `AVPBooter.vmapple2.bin`. - Choose Search : Program Text : All Fields. Type "0x4447" ('DG' of DGST) in "Search for", then choose "Search All". You should find two hits, both in the same function. Click on one to move the cursor to its location. - Once again choose Search : Program Text : All Fields, and type "retab" in "Search For". Then choose "Next". Now you should see the code that runs just before the function returns. It will look something like as follows. In this example, you want to change the instruction at address `0x00102a0c` (`mov x0,x20`) to `mov x0,#0x0`. (AARCH64 machine code uses the `X0` register to store a return value.) ``` 001029f4 a8 83 5a f8 ldur x8,[x29, #local_68] 001029f8 29 f9 37 d0 adrp x9,0x70028000 001029fc 1f 20 03 d5 nop 00102a00 29 a1 40 f9 ldr x9,[x9, #0x140]=>DAT_70028140 00102a04 3f 01 08 eb cmp x9,x8 00102a08 21 06 00 54 b.ne LAB_00102acc 00102a0c e0 03 14 aa mov x0,x20 00102a10 fd 7b 4c a9 ldp x29=>local_10,x30,[sp, #0xc0] 00102a14 f4 4f 4b a9 ldp x20,x19,[sp, #local_20] 00102a18 f6 57 4a a9 ldp x22,x21,[sp, #local_30] 00102a1c f8 5f 49 a9 ldp x24,x23,[sp, #local_40] 00102a20 fa 67 48 a9 ldp x26,x25,[sp, #local_50] 00102a24 fc 6f 47 a9 ldp x28,x27,[sp, #local_60] 00102a28 ff 43 03 91 add sp,sp,#0xd0 00102a2c ff 0f 5f d6 retab ``` - Right click on the `mov x0,0x20` instruction and choose "Patch Instruction". Then change `x0,0x20` to `x0,#0x0`. - In Ghidra's CodeBrowser, choose File : Export Program. Then choose Format : Raw Bytes, and overwrite `AVPBooter.vmapple2.bin`. - Use `vbindiff AVPBooter.vmapple2.bin.org AVPBooter.vmapple2.bin` to check your results. There should be just one change, to a four-byte value -- the length of one AARCH64 instruction. ### iBoot Stage 1 (`LLB.img4`) All the other modules that need patching are "wrapped" in `img4` format. So to get at their actual content you need to use [`img4tool`](https://github.com/tihmstar/img4tool) to unpack them. Then you'll patch them and rewrap them in new `img4` format files. - `img4tool -e -p LLB.im4p.org LLB.img4.org` - `img4tool -e -m LLB.im4m.org LLB.img4.org` - `img4tool -e -o LLB.bin.org LLB.im4p.org` The iBoot Stage 1 module should now be in `LLB.bin.org`. Copy it to `LLB.bin` and patch it according to the instructions for the iBoot Stage 0 binary (`AVPBooter.vmapple2.bin`) above. - Run `img4tool LLB.im4p.org`, which will produce output something like what follows. Use the information from it to run the next command. The value for "desc" differs from one version of macOS to another. Note that you do *not* want to use compression, even though `img4tool` supports it. `img4tool` uses Apple's `libcompression.dylib` to implement its compression (and decompression). But, even though it's the same type ("bvx2"), it's incompatible with the decompression used by the iBoot binaries, which don't have access to any external modules. ``` img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE Compiled with plist: YES IM4P: --------- type: illb desc: iBoot-8422.141.2 size: 0x000369df Compression: bvx2 Uncompressed size: 0x0006dc90 IM4P does not contain KBAG values ``` - `img4tool -c LLB.im4p -t illb -d "iBoot-8422.141.2" LLB.bin` - `img4tool -c LLB.img4 -p LLB.im4p -m LLB.im4m.org` The patched iBoot Stage 1 module should now be in `LLB.img4`. ### iBoot Stage 2 (`iBoot.img4`) Once again you'll need to unwrap the iBoot Stage 2 binary, patch it, and then rewrap it in `img4` format. But this time the process is more complicated: `iBoot.im4p.org` contains a PAYP structure, tacked on to its end, that you'll need to append to `iBoot.im4p` by hand. - `img4tool -e -p iBoot.im4p.org iBoot.img4.org` - `img4tool -e -m iBoot.im4m.org iBoot.img4.org` - `img4tool -e -o iBoot.bin.org iBoot.im4p.org` Here's how to copy the PAYP structure from `iBoot.im4p.org` to a seperate file, which you'll later append to `iBoot.im4p`. - `openssl asn1parse -inform der -in iBoot.im4p.org -i -dump` This outputs the entire `iBoot.im4p.org` file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (`259719` in this case) in the next command. ``` 259719:d=1 hl=2 l= 28 cons: cont [ 0 ] 259721:d=2 hl=2 l= 26 cons: SEQUENCE 259723:d=3 hl=2 l= 4 prim: IA5STRING :PAYP 259729:d=3 hl=2 l= 18 cons: SET 259731:d=4 hl=7 l= 11 cons: priv [ 1768907638 ] 259738:d=5 hl=2 l= 9 cons: SEQUENCE 259740:d=6 hl=2 l= 4 prim: IA5STRING :iocv 259746:d=6 hl=2 l= 1 prim: INTEGER :03 ``` - `xxd -p -s 259719 iBoot.im4p.org iBoot.payp.hex` `iBoot.payp.hex` is a hex dump. The following command will convert it to a binary (in DER format): - `xxd -p -r iBoot.payp.hex iBoot.payp.bin` Check the contents of `iBoot.payp.bin` by running the following command: ``` openssl asn1parse -inform der -in iBoot.payp.bin -i ``` Now copy `iBoot.bin.org` to `iBoot.bin`. Then patch out the call to `validate_boot_object()`. As mentioned above, you can do this instead of patching `image4_validate_property_callback()`. ``` e5 03 04 aa 04 00 80 52 ``` - In the Ghidra CodeBrowser, choose Search : For Instruction Patterns, then Edit Bytes and Input Mode Hex. Copy the above two lines of hexadecimal code and paste it into the Edit Bytes box, then choose Apply. Choose Search All and you should find one hit, with code that looks like the following: ``` FUN_700ac1fc XREF[1]: FUN_70063a78:70063fbc(c) 700ac1fc e5 03 04 aa mov x5,x4 700ac200 04 00 80 52 mov w4,#0x0 700ac204 01 00 00 14 b LAB_700ac208 LAB_700ac208 XREF[1]: 700ac204(j) 700ac208 7f 23 03 d5 pacibsp 700ac20c ff 03 03 d1 sub sp,sp,#0xc0 ``` - Double-click on the cross reference (`FUN_70063a78:70063fbc(c)` in this case). That should take you to code that looks like this: ``` 70063fb0 e1 e3 0c 91 add param_2,sp,#0x338 70063fb4 e4 43 03 91 add param_5,sp,#0xd0 70063fb8 e3 03 16 aa mov param_4,x22 70063fbc 90 20 01 94 bl FUN_700ac1fc undefined FUN_700ac1fc() 70063fc0 a0 03 00 34 cbz param_1,LAB_70064034 70063fc4 14 7b 00 51 sub w20,w24,#0x1e 70063fc8 07 00 00 14 b LAB_70063fe4 ``` - In this example, right-click on the `bl FUN_700ac1fc` instruction and change it to `mov x0,#0x0`. - On recent versions of macOS, there may be more than one cross reference. In this case you should double-click on each one, in turn, and change its target to `mov x0,#0x0`. - Choose File : Export Program. Then choose Format : Raw Bytes, and overwrite `iBoot.bin`. - Run `img4tool iBoot.im4p.org`, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another. ``` img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE Compiled with plist: YES IM4P: --------- type: ibot desc: iBoot-8422.141.2 size: 0x0003f655 Compression: bvx2 Uncompressed size: 0x0007dde8 PAYP: iocv: iocv: 3 IM4P does not contain KBAG values ``` - `img4tool -c iBoot.im4p -t ibot -d "iBoot-8422.141.2" iBoot.bin` - `dd if=iBoot.payp.bin >> iBoot.im4p` - Use [Hex Fiend](https://hexfiend.com/) to open `iBoot.im4p` and correct its length value. Make sure File : Mode is set to Override. - Observe `iBoot.im4p`'s five-byte header and length value -- for example `308307DE0B`. Convert the length (`0x7DE0B`) to decimal (`515595`) and add `30` (for the length of `iBoot.payp.bin` in this case). So the new length in this case is `515625` (== `0x7DE29`). - Correct the length value. The header in this case will now be `308307DE29`. Save `iBoot.im4p`. - `img4tool -c iBoot.img4 -p iBoot.im4p -m iBoot.im4m.org` The patched Stage 2 module should now be in `iBoot.img4`. ### The Kernel Cache (`kernelcache`) As with the iBoot Stage 2 module, you'll need to unwrap the kernel cache, patch it, and rewrap it in `img4` format. - `img4tool -e -p kernelcache.im4p.org kernelcache.org` - `img4tool -e -m kernelcache.im4m.org kernelcache.org` - `img4tool -e -o kernelcache.bin.org kernelcache.im4p.org` `kernelcache.im4p.org`, like `iBoot.im4p.org`, has a PAYP structure at its end. Copy the PAYP structure to a separate file, which you'll later append to `kernelcache.im4p`. - `openssl asn1parse -inform der -in kernelcache.im4p.org -i -dump` This outputs the entire `kernelcache.im4p.org` file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (`18030138` in this case) in the next command. ``` 18030138:d=1 hl=3 l= 186 cons: cont [ 0 ] 18030141:d=2 hl=3 l= 183 cons: SEQUENCE 18030144:d=3 hl=2 l= 4 prim: IA5STRING :PAYP 18030150:d=3 hl=3 l= 174 cons: SET 18030153:d=4 hl=7 l= 19 cons: priv [ 1801676144 ] 18030160:d=5 hl=2 l= 17 cons: SEQUENCE 18030162:d=6 hl=2 l= 4 prim: IA5STRING :kcep 18030168:d=6 hl=2 l= 9 prim: INTEGER :FFFFFE0007B7D488 18030179:d=4 hl=7 l= 14 cons: priv [ 1801677926 ] 18030186:d=5 hl=2 l= 12 cons: SEQUENCE 18030188:d=6 hl=2 l= 4 prim: IA5STRING :kclf 18030194:d=6 hl=2 l= 4 prim: INTEGER :030E4000 18030200:d=4 hl=7 l= 19 cons: priv [ 1801677935 ] 18030207:d=5 hl=2 l= 17 cons: SEQUENCE 18030209:d=6 hl=2 l= 4 prim: IA5STRING :kclo 18030215:d=6 hl=2 l= 9 prim: INTEGER :FFFFFE0007004000 18030226:d=4 hl=7 l= 14 cons: priv [ 1801677946 ] 18030233:d=5 hl=2 l= 12 cons: SEQUENCE 18030235:d=6 hl=2 l= 4 prim: IA5STRING :kclz 18030241:d=6 hl=2 l= 4 prim: INTEGER :AF0000 18030247:d=4 hl=7 l= 11 cons: priv [ 1801679462 ] 18030254:d=5 hl=2 l= 9 cons: SEQUENCE 18030256:d=6 hl=2 l= 4 prim: IA5STRING :kcrf 18030262:d=6 hl=2 l= 1 prim: INTEGER :00 18030265:d=4 hl=7 l= 14 cons: priv [ 1801679482 ] 18030272:d=5 hl=2 l= 12 cons: SEQUENCE 18030274:d=6 hl=2 l= 4 prim: IA5STRING :kcrz 18030280:d=6 hl=2 l= 4 prim: INTEGER :02C64000 18030286:d=4 hl=7 l= 14 cons: priv [ 1801680742 ] 18030293:d=5 hl=2 l= 12 cons: SEQUENCE 18030295:d=6 hl=2 l= 4 prim: IA5STRING :kcwf 18030301:d=6 hl=2 l= 4 prim: INTEGER :02C64000 18030307:d=4 hl=7 l= 13 cons: priv [ 1801680762 ] 18030314:d=5 hl=2 l= 11 cons: SEQUENCE 18030316:d=6 hl=2 l= 4 prim: IA5STRING :kcwz 18030322:d=6 hl=2 l= 3 prim: INTEGER :480000 ``` - `xxd -p -s 18030138 kernelcache.im4p.org kernelcache.payp.hex` `kernelcache.payp.hex` is a hex dump. The following command will convert it to a binary (in DER format): - `xxd -p -r kernelcache.payp.hex kernelcache.payp.bin` Check the contents of `kernelcache.payp.bin` by running the following command: ``` openssl asn1parse -inform der -in kernelcache.payp.bin -i ``` Now copy `kernelcache.bin.org` to `kernelcache.bin`. [Ghidra](https://github.com/NationalSecurityAgency/ghidra) doesn't work well with kernel cache files, so I use [Hopper Disassembler](https://www.hopperapp.com/) to patch `kernelcache.bin`. - In Hopper choose Read Executable to Disassemble (CMD-SHIFT-O) and select `kernelcache.bin`. Then click on Loader, scroll down to `com.apple.security.AppleVPBootPolicy` and click OK. - Select Labels, type "_validate_acm_context" and click on it in the list below. This should take you to the function, whose top few lines should look like this: ``` __validate_acm_context: fffffe0008be57ec pacibsp fffffe0008be57f0 sub sp, sp, #0x40 fffffe0008be57f4 stp x20, x19, [sp, #0x20] fffffe0008be57f8 stp fp, lr, [sp, #0x30] fffffe0008be57fc add fp, sp, #0x30 fffffe0008be5800 sturb wzr, [fp, var_11] fffffe0008be5804 cbz w0, loc_fffffe0008be5868 ``` - You want to find the calls to `_validate_acm_context()` from `_command_create_linked_manifest()` and `_command_update_local_policy_for_kcos()`, and patch each one out by replacing it with a `nop` instruction. - Click on `_validate_acm_context()`'s first line (`pacibsp`), then press the "x" key. This should open a dialog listing all the calls to `_validate_acm_context()` from elsewhere in `AppleVPBootPolicy`. Find `_command_create_linked_manifest()` and `_command_update_local_policy_for_kcos()` in this list, and click on each of them in turn. The calls to `_validate_acm_context()` should look something like this: ``` fffffe0008bd4e74 add x22, x21, #0x66 fffffe0008bd4e78 mov w0, #0x1 fffffe0008bd4e7c mov x1, x22 fffffe0008bd4e80 bl __validate_acm_context fffffe0008bd4e84 tbnz w0, 0x0, loc_fffffe0008bd4ea8 ``` - Select the call to `_validate_acm_context()` (`bl __validate_acm_context`) and change to Hexadecimal mode (from ASM mode). For each of the following four hex values, double-click on it and replace it with values from the following list (binary code for a `nop` instruction). (Hit Enter to update each value after you've typed in its replacement.) ``` 1f 20 03 d5 ``` - Repeat the previous two steps until you've patched the calls to `_validate_acm_context()` from both `_command_create_linked_manifest()` and `_command_update_local_policy_for_kcos()`. - Choose File : New Executable and overwrite `kernelcache.bin` - Use `vbindiff kernelcache.bin.org kernelcache.bin` to check your results. There should be just two changes, each to a four-byte value -- the length of a single AARCH64 instruction. - Run `img4tool kernelcache.im4p.org`, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another. Note that the current version of `img4tool` can't yet deal with `kernelcache.im4p.org`'s PAYP structure. ``` img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE Compiled with plist: YES IM4P: --------- type: krnl desc: KernelManagement_host-354.140.3 size: 0x01131df6 Compression: bvx2 Uncompressed size: 0x03bd4000 PAYP: kcep: kcep: [Error] img4tool: failed with exception: [exception]: what=assure failed code=15597586 line=238 file=ASN1DERElement.cpp commit count=199 commit sha =ed194718f9d6a035a432f2fdfe9fc639b72cba6c ``` - `img4tool -c kernelcache.im4p -t krnl -d "KernelManagement_host-354.140.3" kernelcache.bin` - `dd if=kernelcache.payp.bin >> kernelcache.im4p` - Use [Hex Fiend](https://hexfiend.com/) to open `kernelcache.im4p` and correct its length value. Make sure File : Mode is set to Override. - Observe `kernelcache.im4p`'s six-byte header and length value -- for example `308403BD4033`. Convert the length (`0x03BD4033`) to decimal (`62734387`) and add `189` (for the length of `kernelcache.payp.bin` in this case). So the new length in this case is `62734576` (== `0x3BD40F0`). - Correct the length value. The header in this case will now be `308403BD40F0`. Save `kernelcache.im4p`. - `img4tool -c kernelcache -p kernelcache.im4p -m kernelcache.im4m.org` The patched kernel cache should now be in `kernelcache`. ## Copying the Patched Modules to their Final Destinations ### iBoot Stage 0 (`AVPBooter.vmapple2.bin`) `AVPBooter.vmapple2.bin` is a system file. By default it's protected by the macOS file system, and can't be changed. To get around this I borrow from [How to Defang macOS System Protections](https://gist.github.com/macshome/15f995a4e849acd75caf14f2e50e7e98). Before you follow these steps, you must make the settings changes I described above under Settings Changes. - `mkdir /tmp/mount` - Run `mount` at a Terminal prompt and observe its results, for example as follows. Use the contents of the first line in the next command. ``` /dev/disk5s1s1 on / (apfs, sealed, local, read-only, journaled) devfs on /dev (devfs, local, nobrowse) /dev/disk5s6 on /System/Volumes/VM (apfs, local, noexec, journaled, noatime, nobrowse) /dev/disk5s2 on /System/Volumes/Preboot (apfs, local, journaled, nobrowse) /dev/disk5s4 on /System/Volumes/Update (apfs, local, journaled, nobrowse) /dev/disk1s2 on /System/Volumes/xarts (apfs, local, noexec, journaled, noatime, nobrowse) /dev/disk1s1 on /System/Volumes/iSCPreboot (apfs, local, journaled, nobrowse) /dev/disk1s3 on /System/Volumes/Hardware (apfs, local, journaled, nobrowse) /dev/disk5s5 on /System/Volumes/Data (apfs, local, journaled, nobrowse, protect) /dev/disk2s3 on /Volumes/Boot3 (apfs, sealed, local, read-only, journaled) /dev/disk7s3 on /Volumes/Boot2 (apfs, sealed, local, read-only, journaled) /dev/disk4s1 on /Volumes/Boot4 - Data (apfs, local, journaled, nobrowse, protect) /dev/disk2s1 on /Volumes/Boot3 - Data (apfs, local, journaled, nobrowse, protect) /dev/disk4s3 on /Volumes/Boot4 (apfs, sealed, local, read-only, journaled) /dev/disk6s1 on /Volumes/Boot1 - Data (apfs, local, journaled, nobrowse, protect) /dev/disk6s3 on /Volumes/Boot1 (apfs, sealed, local, read-only, journaled) /dev/disk7s1 on /Volumes/Boot2 - Data (apfs, local, journaled, nobrowse, protect) map auto_home on /System/Volumes/Data/home (autofs, automounted, nobrowse) ``` - `sudo mount -o nobrowse -t apfs /dev/disk5s1 /tmp/mount` - `cd /tmp/mount/System/Library/Frameworks/Virtualization.framework/Resources` - `sudo cp /path/to/patched/AVPBooter.vmapple2.bin .` - `sudo bless --mount /tmp/mount --bootefi --create-snapshot` - Reboot your host computer. On reboot, check the contents of `/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin` to make sure it's what you expect. You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched `AVPBooter.vmapple2.bin` it will simply refuse to start.) If need be, use the following command to revert your snapshot (and `AVPBooter.vmapple2.bin`), then reboot your host computer: ``` sudo bless --mount / --last-sealed-snapshot ``` ### iBoot Stage 1 (`LLB.img4`) You'll need to copy both `LLB.img4` and `logo.img4.org` over their original contents in `AuxiliaryStorage`. This is because `LLB.img4` might be a different length than `LLB.img4.org`, and Apple's virtualization infrastructure expects the logo image to immediately follow the LLB image. [Above](#iboot-stage-1-llb) you determined which set of these images to work with. Now you'll be copying your changes over the original set. - Use [Hex Fiend](https://hexfiend.com/) to open `LLB.img4`, `logo.img4.org` and `AuxiliaryStorage`. Make sure Edit : Mode for `AuxilaryStorage` is Overwrite. - Jump to offset of the active set of LLB/logo images (either `0x24000` or `0x224000`) in `AuxiliaryStorage`. If an LLB image exists at this location, its first two bytes should be `3083`. - Copy the contents of `LLB.img4` (CMD-A, CMD-C) and paste them into `AuxilaryStorage`. - Copy the contents of `logo.img4.org` and paste them into `AuxiliaryStorage`, then save the file. ### iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`) Copy these files to your macOS guest VM and do the following there: - Run `kmutil inspect`, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case. ``` /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache ``` - Note the "Next Stage Image4 Hash" just before `/System/Library/Caches` -- in this case `FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83`. - `sudo cp kernelcache /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache` - Run `cd /` and `sudo find . -name iBoot.img4 -exec ls -al \{\} \;`. Observe the hit whose path contains the Next Stage Image4 Hash. Use the results in the following command. For example: - `sudo cp iBoot.img4 /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4` Shut down your your macOS guest VM and reboot it into Recovery Mode. Then perform the following steps. You'll be looking for the same Next Stage Image4 Hash as in the previous steps. Run Terminal in Recovery Mode, then do the following: - Run `cd /` and `find . -name iBoot.img4 -exec ls -al \{\} \;`. Expect two hits this time, which should look like the following: ``` /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4 /System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4 ``` - Copy `iBoot.img4` over both of them. You may also see another hit that looks like the following. Ignore it. You can tell by its date and file size that you already copied the patched `iBoot.img4` over it above. ``` /Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4 ``` - Run `cd /` and `find . -name kernelcache -exec ls -al \{\} \;`. Once again expect two hits, which should look like the following (maybe plus one superfluous hit, as above). ``` /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache /System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache ``` - Copy `kernelcache` over both of them. - Reboot your macOS guest VM and start playing with third-party kernel extensions on it!