# NightRun Boots Local LLMs Without a Conventional OS > Source: > Published: 2026-07-30 08:10:44+00:00 # NightRun Boots Local LLMs Without an Operating System On July 30, 2026, CNX Software reported that NightRun boots directly into a local LLM on Raspberry Pi 5 hardware and x86_64 PCs without loading a conventional operating system. The experimental open-source UEFI application loads quantized models into RAM, supports several 1B to 4B parameter models, and provides a framebuffer chat interface for offline CPU inference from USB media or a microSD card. NightRun is an experimental open-source UEFI application that boots a Raspberry Pi 5 or x86_64 PC directly into a local LLM, without a conventional operating system. According to CNX Software, the runtime can boot from USB media on x86_64 systems or a microSD card on Raspberry Pi 5, then load a quantized model into RAM and present a framebuffer-based chat interface. The project's GitHub repository describes NightRun as a Rust application with no Linux userspace, kernel, browser, network stack, or host process beneath it. Firmware starts the application directly. The repository states that its installer builds and flashes a bootable image, verifies a pinned model download with SHA-256, and supports x86_64 USB and Raspberry Pi 5 SD-card targets. ### Runtime and model support CNX Software reports that NightRun supports Llama 3.2 1B, Llama 3.2 3B, Granite 4.1 3B, and Qwen3 4B Instruct 2507 models converted to its .nrm format. Its listed memory requirements range from 4 GB for Llama 3.2 1B to 8 GB for Qwen3 4B Instruct 2507. The publication describes a zero-copy design for Q8_0, Q4_K, and Q6_K quantized models, meaning inference operates on the quantized weights in RAM rather than first dequantizing them. It also reports AVX2, FMA, and F16C kernels for x86 systems and NEON kernels for Raspberry Pi hardware. After model loading, NightRun seals storage, according to both the repository and CNX Software. The GitHub documentation states that subsequent disk reads cause a hard fault by design, while CNX Software reports CRC-32 verification during model loading. The documentation's design is consequently narrower than a typical local-LLM stack: it is an appliance-style runtime for one offline inference task rather than a general-purpose environment. ### Performance and deployment constraints CNX Software reports about 20 tokens per second for Llama 3.2 1B in Q8_0 format on an eight-core x86 QEMU virtual machine, and about 3.0 tokens per second for Granite 4.1 3B in Q4_K_M format on an 8 GB Raspberry Pi 5. The publication attributes decoding throughput primarily to memory bandwidth. CNX Software lists 64-bit x86_64 UEFI PCs with Secure Boot disabled among the supported hardware. NightRun's repository identifies USB and SD-card flashing as part of installation, so deployment requires dedicated removable boot media rather than installation into an existing OS environment. For practitioners, bare-metal inference runtimes are a distinct engineering tradeoff from conventional local serving stacks. Similar designs can remove operating-system memory and background-process overhead, but they also omit standard facilities such as networking, package management, observability, and multi-application scheduling. ## Key Points - 1NightRun runs quantized LLM inference directly from UEFI, removing the conventional OS layer from supported Raspberry Pi 5 and x86_64 deployments. - 2The runtime supports 1B to 4B parameter models and CPU-specific vector kernels, with memory bandwidth reported as the primary decoding constraint. - 3Bare-metal local inference can reduce software overhead, while comparable designs trade away networking, observability, package management, and general-purpose scheduling. ## Scoring Rationale NightRun is a technically unusual local-inference runtime that may interest edge AI and systems practitioners exploring minimal-footprint CPU deployments. Its early experimental status, narrow hardware support, and small-model focus limit its broad near-term impact relative to mainstream inference platforms. ## Sources Primary source and supporting public references used for this report. Practice interview problems based on real data 1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with. [Try 250 free problems](/problems)