# Building a VMM from scratch, #0.5: Booting Linux in 190 lines of C > Source: > Published: 2026-10-22 11:14:05.076148+00:00 In [#0](https://blog.boxlite.ai/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun) we set out to replace libkrun with a VMM of our own. Its boot path is landing one PR at a time, and its tests already boot a real Linux kernel to an init on four vCPUs ([#2052](https://github.com/boxlite-ai/boxlite/pull/2052)). Before the parts that walk through that Rust code, here is the whole path in miniature: [vmmlite](https://github.com/boxlite-ai/vmmlite), 190 lines of C, not counting comments, that boot a stock Ubuntu kernel on KVM to a PID 1 that prints one line. ``` $ ./vmmlite bzImage initramfs.cpio [ 0.000000] Linux version 7.0.0-1014-gcp (buildd@lcy02-amd64-091) ... ... [ 1.757880] Run /init as init process Hello from PID 1! [ 1.766049] reboot: Restarting system ``` About three seconds after it starts, vmmlite exits with status 0. It's C on purpose: one file, nothing between you and the `ioctl()` calls. Our Rust VMM follows the same path, with more care. **For learning only.** vmmlite is not for production. It doesn't validate its input files, runs without a sandbox, and emulates only a serial port and a reset. To run code in a micro-VM, use a production VMM such as [BoxLite](https://github.com/boxlite-ai/boxlite). `main()` is eight numbered sections. Four talk to KVM; the other four only write bytes into guest memory. *Figure 1. The eight sections of vmmlite.c. Sections 1, 2, 7 and 8 call KVM; sections 3 to 6 only write bytes into guest RAM.* KVM does the heavy lifting. It runs guest instructions on the real CPU and, after `KVM_CREATE_IRQCHIP` and `KVM_CREATE_PIT2`, emulates the interrupt controllers and the timer inside the host kernel. What's left for vmmlite is the job of a PC's firmware and bootloader, plus two devices: a serial port and a reset line. Nothing runs in the guest before Linux: no BIOS, no GRUB. Linux's [64-bit boot protocol](https://github.com/gregkh/linux/blob/v6.12.110/Documentation/arch/x86/boot.rst#L1374-L1412) allows that, if the loader leaves the machine as they would have: | The protocol asks for | vmmlite | |---|---| | the kernel at its load address | the bzImage's 64-bit part, copied to 1 MiB (3) | | `struct boot_params` , the "zero page" | at 0x7000: the setup header, command line, initramfs and E820 memory map (4) | | 64-bit mode, paging on, identity mapped | page tables at 0x9000 that map the first 1 GiB to itself (6) | | a GDT with flat `__BOOT_CS` (0x10) and`__BOOT_DS` (0x18) | four entries at 0x500 (6) | | interrupts off, RSI at the zero page | RFLAGS 0x2, RSI 0x7000 (7) | | a jump to the 64-bit entry, load address + 0x200 | RIP 0x100200 (7) | A bzImage starts with a boot sector and 16-bit setup code, written for a BIOS. vmmlite checks the header and skips both: ``` if (memcmp(k + 0x202, "HdrS", 4) || !(k[0x236] & 1)) // the header's magic; XLF_KERNEL_64 die("vmmlite: need a bzImage with a 64-bit entry\n"); size_t setup = ((k[0x1f1] ? k[0x1f1] : 4) + 1) * 512; // setup_sects (0 means 4) + boot sector memcpy(ram + KERNEL, k + setup, ksize - setup); ``` Figure 2 shows guest RAM when the vCPU starts. The addresses are the ones most VMMs we surveyed share, and the ones in [BoxLite's VMM design](https://github.com/boxlite-ai/boxlite/blob/main/docs/contributing/architecture/vmm/README.md#boot-path). *Figure 2. Guest RAM at the first instruction, not to scale. The arrows show how the CPU and the kernel find each piece: registers, zero page fields, and a scan for the MP table's signature.* Long mode is a handful of control-register bits, set through KVM: ``` s.cr3 = PML4; // the page tables s.cr4 |= 1 << 5; // CR4.PAE s.cr0 |= 1u << 31 | 1; // CR0.PG, CR0.PE: paging on s.efer |= 1 << 10 | 1 << 8; // EFER.LMA, EFER.LME: long mode IOCTL(cpu, KVM_SET_SREGS, &s); // apply struct kvm_regs r = { .rip = KERNEL + 0x200, // the 64-bit entry point .rsi = ZERO_PAGE, // boot protocol: RSI = the zero page .rflags = 2}; // interrupts off (bit 1 is always set) ``` At 0x100200 sits `startup_64`, the decompressor's 64-bit entry ([head_64.S](https://github.com/gregkh/linux/blob/v7.0.14/arch/x86/boot/compressed/head_64.S#L277-L278)). It unpacks the real kernel and jumps to it. The vCPU exits along the way, for CPUID or the first touch of a page, but KVM handles those exits itself. CPUID comes from `KVM_GET_SUPPORTED_CPUID`, with two bits set in leaf 1, [as Firecracker does](https://github.com/firecracker-microvm/firecracker/blob/6ec271414300fb8f0ff1ff79ad321bb7467d072f/src/vmm/src/cpu_config/x86_64/cpuid/normalize.rs#L245-L251): bit 31, "running under a hypervisor", and bit 24, the TSC-deadline timer that KVM emulates. Firmware also tells the kernel what it runs on: the CPUs, the interrupt controllers, and how interrupts are wired. Modern firmware uses ACPI tables. vmmlite writes the older, much smaller Intel MultiProcessor (MP) table: 240 bytes that list one CPU, an ISA bus, an IOAPIC at 0xFEC00000, and 16 routes from ISA IRQ n to IOAPIC pin n. Linux looks for the table's `_MP_` signature in a few fixed places, one of them the last KiB below 640 KiB ([mpparse.c](https://github.com/gregkh/linux/blob/v7.0.14/arch/x86/kernel/mpparse.c#L612-L614)). vmmlite writes it at 0x9FC00, as Firecracker, libkrun and our Rust VMM ([#2052](https://github.com/boxlite-ai/boxlite/pull/2052)) do, and leaves that range out of the E820 map. Without the table or ACPI, Linux turns the IOAPIC and SMP off, as the [design doc](https://github.com/boxlite-ai/boxlite/blob/main/docs/contributing/architecture/vmm/README.md#boot-path) explains. The routes match how KVM wires its in-kernel chips by default: line n drives PIC input n and IOAPIC pin n ([irq_comm.c](https://github.com/gregkh/linux/blob/v6.12.110/arch/x86/kvm/irq_comm.c#L366-L389)). So IRQ 4 from the serial port lands where Linux expects it. Then vmmlite calls `KVM_RUN` in a loop. It returns only when the guest does something KVM leaves to user space, and vmmlite answers three kinds of port I/O: ``` } else if (port >= 0x3f8 && port <= 0x3ff) { // COM1; an IN reads what com1() *io = com1(vm, port & 7, run->io.direction, *io); // returns, an OUT ignores it } else if (run->io.direction == KVM_EXIT_IO_IN) { // the guest reads another port: memset(io, 0, run->io.size); // it reads 0 } else if (port == 0x64 && *io == 0xfe) { // keyboard controller reset: return 0; // the guest rebooted; done } // any other port write: ignored ``` COM1 is a 16550 UART at ports 0x3F8 to 0x3FF. `com1()` emulates just enough of five registers for Linux's 8250 driver to find it ([8250_port.c](https://github.com/gregkh/linux/blob/v7.0.14/drivers/tty/serial/8250/8250_port.c#L1104-L1126)) and call it an original 8250. The kernel's own messages are the easy part. The console code polls the line status register before each byte ([8250_port.c](https://github.com/gregkh/linux/blob/v7.0.14/drivers/tty/serial/8250/8250_port.c#L3223-L3229)), and `com1()` always answers 0x60, "transmitter empty". PID 1 is different. Its `write()` lands in a tty buffer, and the driver drains that buffer from its interrupt handler: it turns on the "THR empty" interrupt ([8250_port.c](https://github.com/gregkh/linux/blob/v7.0.14/drivers/tty/serial/8250/8250_port.c#L1416-L1430)) and writes the next byte each time the UART reports it ([8250_port.c](https://github.com/gregkh/linux/blob/v7.0.14/drivers/tty/serial/8250/8250_port.c#L1826-L1828)). So `com1()` keeps one bit of state, a pending "THR empty" event, and drives IRQ 4 from it: ``` struct kvm_irq_level irq = {.irq = 4, .level = ier & 2 && thre}; IOCTL(vm, KVM_IRQ_LINE, &irq); // IRQ 4, up or down ``` *Figure 3. How "Hello from PID 1!" leaves the VM. Every port access is a VM exit that vmmlite answers; IRQ 4 tells the driver to send more.* Each character costs at least three exits: IIR, LSR, then the byte. That's the price of emulating a device one register at a time, and why real VMMs move bulk I/O to virtio. The last exit is the reset. init.c is three calls: ``` int main(void) { if (write(1, "Hello from PID 1!\n", 18) < 0) return 1; // fd 1: the console, ttyS0 tcdrain(1); // wait until the UART has sent it all reboot(RB_AUTOBOOT); // restart: Linux pulses the i8042 // reset line, and vmmlite exits } ``` Linux tries an ACPI reset, finds no ACPI, and falls back to the keyboard controller: it writes 0xFE to port 0x64, which pulses the CPU's reset line ([reboot.c](https://github.com/gregkh/linux/blob/v7.0.14/arch/x86/kernel/reboot.c#L657-L668)). vmmlite takes that write as the end of the VM and returns 0. Here's the boot on a Google Cloud VM with nested KVM and Ubuntu's 7.0.0-1014-gcp kernel, trimmed to the lines vmmlite caused: | Kernel log, trimmed | Caused by | |---|---| | `Command line: console=ttyS0 panic=-1` | 4: `cmd_line_ptr` | | `BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] System RAM` | 4: the first E820 entry | | `BIOS-e820: [gap 0x000000000009fc00-0x00000000000fffff]` | 4: the hole that hides the MP table | | `Hypervisor detected: KVM` | 7: CPUID leaf 1, bit 31 | | `found SMP MP-table at [mem 0x0009fc00-0x0009fc0f]` | 5 | | `RAMDISK: [mem 0x1ff40000-0x1fffffff]` | 4: the initramfs, at the top of RAM | | `ACPI BIOS Error (bug): A valid RSDP was not found` | no ACPI tables, on purpose | | `IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23` | 5 for the ID and address; KVM's IOAPIC for version 0x11 and 24 pins | | `TSC deadline timer available` | 7: CPUID leaf 1, bit 24 | | `serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 8250` | 8: `com1()` | | `i8042: Can't read CTR while initializing i8042` | 8: unknown ports read 0 | | `Hello from PID 1!` | init.c, through `com1()` and IRQ 4 | | `reboot: Restarting system` | init.c's `reboot()` , just before the write to port 0x64 | The keyboard controller line is the one vmmlite pays for. Every read of its ports returns 0, so Linux waits for a byte that never comes ([i8042.c](https://github.com/gregkh/linux/blob/v7.0.14/drivers/input/serio/i8042.c#L230-L238)): 0.73 seconds between `i8042: Probing ports directly.` and that error, out of 1.76 seconds to `Run /init`. You need x86_64 Linux with KVM and access to `/dev/kvm`, plus gcc with static glibc and cpio. vmmlite boots your host's own kernel: ``` git clone https://github.com/boxlite-ai/vmmlite && cd vmmlite gcc -O2 -Wall -Wextra -o vmmlite vmmlite.c gcc -O2 -Wall -Wextra -static -o init init.c echo init | cpio -o -H newc > initramfs.cpio sudo install -m 644 /boot/vmlinuz-$(uname -r) bzImage ./vmmlite bzImage initramfs.cpio ``` We've booted it on Intel with nested KVM on Google Cloud, and on AMD-V emulated by QEMU. The repo's CI boots a GitHub Actions runner's own kernel on every push. vmmlite is a map, not a VMM you should run. What it skips is what the rest of this series is about: | vmmlite | BoxLite's VMM | Part | |---|---|---| | one loop around `KVM_RUN` ; an exit it doesn't expect ends the VM | one exit contract for KVM, Hypervisor.framework and, later, WHP | #2 | | no way to stop the vCPU from outside | pulling a vCPU out of the guest, safely | #3 | | CPUID copied from KVM, two bits set | registers, CPUID and MSRs on x86 | #4 | | whatever kernel the host has | a kernel we build ourselves | #5 | | a bzImage, on x86_64 only | the VMM as the bootloader, on arm64 and x86_64 | #6 | | COM1 and a reset; other ports read 0 | serial, RTC and reset first, then virtio | #7, #1012 | | KVM only | Hypervisor.framework, where the VMM decodes the exits | #8 | | a PID 1 that prints one line | our agent as PID 1 | #13 | | no sandbox, no hardening | guest input must never panic the host | #14 |