NightRun UEFI application boots a local LLM on Raspberry Pi 5 and x86 PCs without an OS

NightRun, an experimental open-source UEFI application written in Rust, boots a local large language model directly into RAM on 64-bit x86_64 PCs (with Secure Boot disabled) and Raspberry Pi 5, bypassing a conventional operating system to free memory and bandwidth for AI inference. The project, reported by CNX Software, loads quantized models of 1.3 GB to 2.4 GB, verifies CRC-32 checksums, and seals storage after boot to prevent disk reads.

Running LLMs locally on an SBC or mini PC is a good way to keep your data private, but they typically require a conventional operating system, which uses a good portion of system memory. NightRun is an experimental open-source project that addresses this issue by booting a machine directly into a local LLM from a USB drive or microSD card, without loading a conventional OS. By removing the operating system, NightRun makes more RAM and memory bandwidth available for AI inference. The runtime is written in Rust, and during boot it loads a quantized model 1.3 GB to 2.4 GB directly into RAM while verifying its CRC-32 checksums. After the model is loaded, storage is “sealed,” meaning any later attempt to read from the disk will trigger a hard fault. NightRun key features: Hardware support – Supports 64-bit x86 64 UEFI PCs Secure Boot disabled and Raspberry Pi 5 via USB ... The post NightRun UEFI application boots a local LLM on Raspberry Pi 5 and x86 PCs without an OS https://www.cnx-software.com/2026/07/30/nightrun-uefi-application-boots-a-local-llm-on-raspberry-pi-5-and-x86-pcs-without-an-os/ appeared first on CNX Software - Embedded Systems News https://www.cnx-software.com .