# A 26B Model in 2 GB of RAM, Courtesy of Your SSD > Source: > Published: 2026-07-29 17:08:32+00:00 [AI](https://sourcefeed.dev/c/ai)Article # A 26B Model in 2 GB of RAM, Courtesy of Your SSD TurboFieldfare streams Gemma 4's experts from flash, and the trick will outlive the tool. [Priya Nair](https://sourcefeed.dev/u/priya_nair) A 26-billion-parameter model in 2 GB of RAM sounds like a benchmark scam. It isn't — but the interesting part isn't the number. It's what the trick reveals about where local inference is headed: mixture-of-experts routing has quietly turned your SSD into a usable tier of the memory hierarchy, and [TurboFieldfare](https://github.com/drumih/turbo-fieldfare) is the first project I've seen that builds an entire engine around that bet. The project, which hit the Hacker News front page this week, is a Swift-and-Metal runtime that runs Google's [Gemma 4](https://ai.google.dev/gemma) 26B-A4B — 4-bit quantized, 14.3 GB on disk — on any Apple Silicon Mac while holding roughly 2 GB resident. On a base M2 MacBook Air with 8 GB of RAM, the author reports 5.1–6.3 tokens per second. On an M5 Pro, 31–35. Those are self-reported numbers, but the HN thread stress-tested the design hard and it held up. ## Why this works now and didn't before Streaming weights from disk is old news, and mostly bad news. llama.cpp has mmap'd weights since 2023, and for dense models the result on a RAM-starved machine is thrashing: every token touches every parameter, so the page cache churns through the whole file per forward pass. Apple's own researchers mapped this territory in the ["LLM in a flash" paper](https://arxiv.org/abs/2312.11514) back in December 2023, using activation sparsity to predict which FFN neurons to pull from flash. Clever, but it fought the architecture. Dense models simply weren't built to be partially loaded. MoE models are. Gemma 4 26B-A4B activates only about 4B of its 26B parameters per token — the rest sit idle in expert blocks the router didn't pick. TurboFieldfare exploits the obvious corollary: keep the shared 1.35 GB core (embeddings, attention, router) and the FP16 KV cache pinned in RAM, and fetch only the experts each token actually routes to from SSD. The engineering details are where it gets good. The author found that mmap was the wrong tool — cold expert loads took 10 ms through the page-fault path versus 2.8 ms with explicit `pread` , which in their simulations was the difference between 0.5 and 4 tokens per second. And expert selection has real temporal locality: the full route changes almost every token, but around 40% of experts repeat on the next token and 57% recur within two, so a small 16-slot cache hits about 67% of the time. That locality is the load-bearing fact. Without it, you're just benchmarking your SSD. This is the same insight the community applied to Mixtral with CPU-RAM expert offloading in 2024, and that [ktransformers](https://github.com/kvcache-ai/ktransformers) uses to run DeepSeek-scale MoEs on single GPUs. TurboFieldfare pushes it one tier down the hierarchy — past DRAM, onto flash — which only pencils out because Apple ships fast NVMe in everything. ## What you'd actually do with it Be clear-eyed about the trade. The author is: fully loaded, the same model does 75 tok/s in 14 GB of RAM. Streaming trades away roughly 40 tok/s to save 12 GB. If you have a 24 GB or 32 GB machine, you should mostly just load the model — or note that on high-RAM Macs the macOS page cache ends up holding the hot experts anyway, which is part of why the M5 numbers look so much better than the M2's. The real audience is the enormous installed base of 8 GB M1 and M2 MacBooks — machines Apple sold by the millions before bumping the base config to 16 GB, and which have been effectively locked out of local models beyond the 3B–7B class. For those users, 5–6 tok/s from a model that lands top-ten on open-model leaderboards is a genuine unlock. That's comfortable reading speed for chat and drafting. It is not enough for agentic loops or an inline coding assistant, where you'll feel every second of a 2,000-token response. Adoption is low-friction: a streaming installer, a CLI, a native Mac app, and an OpenAI-compatible server on loopback, so anything that speaks the Chat Completions API can point at it. The constraints are sharper: macOS 26+, Metal 4, text-only, and — this is the big one — the engine is hard-wired to exactly one model. There's no GGUF loading, no swapping in next quarter's checkpoint. Two practical footnotes from the HN thread: sustained SSD reads don't wear flash (wear comes from writes), and the 2 GB figure includes an FP16 KV cache, so long contexts will push the footprint past the headline number. ## Point solution, durable idea My read: TurboFieldfare the product is a proof of concept you shouldn't build on, and TurboFieldfare the existence proof matters a lot. Single-model, single-platform engines don't survive contact with the model release cycle — Gemma 5 or the next Qwen MoE ships, and a bespoke runtime is instantly a museum piece. The Apache-2.0 code's real value is as a reference implementation, and history says the technique gets absorbed: [llama.cpp](https://github.com/ggml-org/llama.cpp) and [MLX](https://github.com/ml-explore/mlx) already have MoE plumbing and mmap machinery; a pread-based expert cache with locality-aware eviction is a bounded, well-specified feature for either. I'd expect a generalized version within a couple of quarters, because the incentive is enormous — every open-weights lab has converged on sparse MoE precisely because of the active-parameter economics this exploits. The larger shift is worth naming plainly. For the entire local-LLM era, RAM has been the wall, and Apple's RAM pricing made it an expensive one. Expert streaming re-prices the problem: total parameters now cost you disk (cheap, plentiful), while active parameters cost you RAM and compute. Once runtimes generalize this, the question when a model drops stops being "do I have 64 GB?" and becomes "how fast is my SSD, and how sparse is the router?" That's a much better question — and for once, the answer favors the hardware people already own. ## Sources & further reading - [Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac](https://github.com/drumih/turbo-fieldfare)— github.com - [Show HN discussion: TurboFieldfare](https://news.ycombinator.com/item?id=49098510)— news.ycombinator.com - [Gemma 4: Byte for byte, the most capable open models](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/)— blog.google - [Google Gemma 4 26B A4B now available on Workers AI](https://developers.cloudflare.com/changelog/post/2026-04-04-gemma-4-26b-a4b-workers-ai/)— developers.cloudflare.com - [LLM in a flash: Efficient Large Language Model Inference with Limited Memory](https://arxiv.org/abs/2312.11514)— arxiv.org [Priya Nair](https://sourcefeed.dev/u/priya_nair)· AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. ## Discussion 0 No comments yet Be the first to weigh in.