V.E.L.O.C.I.T.Y.-OS: Ditching the Web Stack & The 30MB Standalone IDE (Part 3)

A developer built V.E.L.O.C.I.T.Y.-OS, a bare-metal operating system that runs entirely inside the CPU's L3 cache. The project includes a custom 30MB standalone IDE written in Rust that replaces VS Code, and a neural document architecture binary format that achieves 61.32ns latency versus 846.45ns for equivalent JSON. The system also features a custom model runtime with a 4x compressed KV-cache, enabling concurrent execution of three times as many AI agents on the same memory budget.

With the Neural Document Architecture NDA binary format defined, the next logical bottleneck was the environment it ran in. I was building this as a VS Code extension, which meant dealing with TypeScript, JSON-RPC serialization, and Electron's massive memory footprint. VS Code regularly consumes 300MB+ of RAM just idling before you've even opened a file. Worse, parsing JSON text in the agent hot path was eating up microsecond cycles. I decided that if the format was bare-metal and binary, the development environment should be too. The V.E.L.O.C.I.T.Y.-OS 12-Part RoadmapWe are building a bare-metal, self-healing operating system running entirely inside the CPU's L3 cache. Here is the roadmap for this 12-part series: The first step was replacing JSON serialization. I wrote a standalone C class library Velocity.NDA and a Rust counterpart. By utilizing C MemoryMarshal and ReadOnlySpan , I mapped compiled .ndf files directly from memory buffers. No heap allocations, no garbage collection, and no text parsing: Here is the corresponding loading snippet from src/nda.rs illustrating how simple offset-based buffer index reads replace string/JSON parser passes: php // src/nda.rs — Zero-Allocation Binary Loading pub fn load path: &Path - Result<Self { let data = fs::read path ?; // Header structure: magic 4B + version 2B + rows 4B + cols 4B + scale 4B = 18B const HDR: usize = 18; let magic = u32::from le bytes data 0..4 .try into .unwrap ; let version = u16::from le bytes data 4..6 .try into .unwrap ; let rows = u32::from le bytes data 6..10 .try into .unwrap as usize; let cols = u32::from le bytes data 10..14 .try into .unwrap as usize; let scale = f32::from le bytes data 14..18 .try into .unwrap ; let bitmap bytes = rows cols + 7 / 8; // Map slice pointers directly out of the read byte buffer let sign = data HDR..HDR + bitmap bytes .to vec ; let extra = data HDR + bitmap bytes..HDR + 2 bitmap bytes .to vec ; Ok Self { rows, cols, scale, version, sign, extra } } As observed when reviewing these latency figures: "61.32ns vs 846.45ns on equivalent JSON — that's not an optimization, that's a different category of problem. Zero-allocation with MemoryMarshal and spans directly mapped from the buffer means you're not parsing, you're reading. The distinction matters at scale." Next, I bypassed VS Code completely. I built a custom, lightweight Agentic IDE in Rust. The design goals were strict: By eliminating the Chromium WebView and Electron Extension Host boundaries, the architectural performance gains were staggering: Arc<Graph memory space instead of serialized over IPC pipes.Here is the architectural comparison mapping the process boundary layouts: To support the agentic workflow, I built three core features: But a 30MB IDE isn't fully self-contained without a fast local model runtime. VS Code relies on massive background processes for AI. I decided to build a custom runtime for models , including a distillation layer that converts model weights like BitNet b1.58 directly into the NDA format. Instead of traditional FP16 floating-point tensors, the NDA-KV cache stores attention Key and Value matrices as semantic triplets decomposed into Active and Positive bitmaps . This structure leverages Vulkan Shared Virtual Memory SVM and allows the GPU to traverse a cryptographically chained linked list of NDA container frames. The results were staggering: As I mentioned to Pascal, this came with a one-time tradeoff: a 27% increase in base weight size over standard b1.58. However, because the KV-cache is what you continually consume, this 4x compression means you can run 3x as many agents concurrently with full context on the same memory budget, with full cryptographic auditability built-in. When I posted these memory and latency metrics, analyzed the L2 cache implications:"L2 cache execution for real-time transaction clearing — that explains the zero-allocation constraint... The one-time weight tradeoff for permanent KV-cache compression is the right way to think about it — you pay once at distillation time, you benefit on every inference." Pascal pointed out that by eliminating the serialization/deserialization boundary and shifting to a bitwise NDA-KV cache, I was doing the opposite of modern web frameworks—I was reclaiming the hardware. But local JIT compilation of my new language was still relying on closure chains and CPU-bound math. I needed to push the execution speeds further. In the next post, I'll document how I designed a two-tier closure JIT compiler and utilized Higher-Ranked Trait Bounds HRTBs to eliminate memory management overhead on the execution hot path. Are you building extensions or web-based interfaces for developer tools? Have you run into Electron's process boundaries or V8 garbage collection sweeps in the agent hot path? Would you consider a pure-native layout e.g. Rust + GPU UI to bypass the serialization tax? Let's discuss in the comments below Special thanks to Disclaimer: AI was used throughout this project, it is just fitting that it would co-author with me, so special thanks to the Foundry for its tireless hours toiling away and Gemini for producing the cover image.