NightRun boots a Raspberry Pi 5 or x86_64 UEFI machine directly into a local LLM without loading an OS. The Rust runtime loads a quantized model (1.3 GB to 2.4 GB in the .nrm format) directly into RAM during UEFI boot, verifies CRC-32 checksums, seals storage to prevent later disk reads, and opens a framebuffer chat interface. Llama 3.2 1B runs in 4 GB. Qwen3 4B Instruct runs in 8 GB. No Linux. No Python stack. No container runtime.
The constraint being removed is the OS as a coordination layer between hardware and inference. On resource-constrained edge hardware, the operating system can consume 30-40% of available RAM before the model loads. NightRun eliminates that overhead through zero-copy execution: Q8_0, Q4_K, and Q6_K quantized models run directly from RAM without dequantization, using AVX2/FMA kernels on x86 and NEON on Pi 5. The runtime is experimental and the project is a proof of concept, but the architecture is reproducible and Rust-native.
The embedded AI toolchain assumption that a general-purpose OS is a required dependency for LLM inference is starting to crack at the edges. NightRun is the same category of move as uLLaMA, LLVM bare-metal profiles, and custom RTOS inference harnesses, but it goes further by treating UEFI as the full runtime boundary. If you are shipping a hardware product where the AI inference endpoint does not need a general-purpose OS, this is the architecture to track. The firmware and OS teams that treat the inference runtime as an application on top of Linux will own more coordination cost than the teams that treat inference as firmware.