Nobody talks about RAM. Every local-LLM regret is a RAM problem.

A developer benchmarked local LLM memory usage on a Ryzen desktop with 32 GB of RAM and a 12 GB RTX 3060, finding that a 4.9 GB llama3.1:8b model reserved 7.0 GB of VRAM at a 32,000-token context window — a 43% "context tax" driven by the KV cache rather than model weights. The writeup argues system RAM, not VRAM, determines whether larger models like 14B and 27B run at all, since they can offload to 32 GB of RAM but not fit in a 12 GB card.

Originally published at mrsaynothing.dev https://mrsaynothing.dev/en/blog/2026-09-19/nobody-talks-about-ram?utm source=devto&utm medium=referral . Walk into any local-LLM thread and the arguments are about GPUs. VRAM benchmarks, 24 GB cards, CUDA versus ROCm, whether the 3060 is still the people's card. Meanwhile the number that actually decides whether your model runs is sitting in the other slot, unbenchmarkable and unmarketed: how much RAM the machine has. VRAM sells the dream. RAM decides whether the model boots at all — and how much context survives when it does. I tested this on the box in front of me while writing this post: a Ryzen desktop with 32 GB of RAM and a GeForce RTX 3060 with 12 GB of VRAM. Pulled llama3.1:8b — the download page says 4.9 GB. Watch what it actually reserved: That is the whole argument in one screen. A "4.9 GB model" reserved 7.0 GB before answering a single prompt — a 43% context tax — and kept 7,963 of 12,288 MiB of VRAM to itself. The weights were never the budget. The context was. The llama.cpp memory notes https://github.com/ggml-org/llama.cpp/blob/master/docs/memory.md spell out the arithmetic that download pages omit: total memory = model weights + KV cache + compute buffer. Only the first term is constant. The KV cache grows linearly with context length, and the compute buffer grows with batch and graph shape. Ollama wraps llama.cpp, so the same law applies — which is why ollama ps reported 7.0 GB for a 4.9 GB tag at a 32,000-token window, all of it resident on the GPU. A quantized model isn't a compromise. It's an admission that memory was always the real budget. This is also why the GGUF quantization ladder exists at all. Q4 isn't a religion; it's what makes the memory arithmetic land inside hardware people own. And it's why two "identical" 8B setups behave nothing alike: same model, different context length, different machines entirely. Same three model classes, both memory types, one 12 GB card and 32 GB of RAM — the configuration thousands of developers actually have: | Model class Q4 | Download | Loaded + 32k ctx | On 12 GB VRAM | On 32 GB RAM | |---|---|---|---|---| | 7–8B llama3.1:8b | 4.9 GB | 7.0 GB measured | 100% GPU, ~8 GiB used | barely noticed | | 13–14B qwen2.5:14b | 9.0 GB | ~12 GB | offload begins | fine | | 27–32B gemma3:27b | 17 GB | ~20+ GB | CPU pulls the weight | the only reason it runs | Read the last two rows again. On VRAM alone, a 14B model at a real context window is already a split-offload job, and a 27B is impossible. On 32 GB of system RAM, both are merely slow . That difference — between impossible and slow — is the entire practical difference between RAM and VRAM. If it fits in VRAM, it's fast. If it fits in RAM, it works. If it fits in neither, you're swapping to an NVMe drive and time loses meaning. Offload sits on PCIe, and the Ollama FAQ https://docs.ollama.com/faq/ is straight about the cost: layers that don't fit on the GPU run on the CPU, and throughput drops hard as the GPU share shrinks. Nobody chooses that trade-off consciously. It happens silently, one layer at a time, and the symptom is just "local models feel overrated." Things that broke or surprised me while writing this piece, in order: ps output. ollama rm cycle keeps the disk honest. None of this means GPUs don't matter — the 100% GPU line in that capture is why generation felt instant. It means the GPU is the second question. The runner choice comes after you know what fits, not before. RAM you need = model file + KV cache for your real context window + 4 GB for being a computer. For 7–8B at Q4, 16 GB is comfortable. For 14B–32B, 32 GB stops being a luxury and starts being the point. Buy VRAM for the speed you want at the context you use; buy RAM for everything you'll ever load. Check ollama ps once and the spec-sheet religion quietly ends. So, two questions. When you spec your next dev box, are you buying VRAM for the benchmarks you'll post — or RAM for the models you'll actually run? And honestly: how many of the models you pulled at 2 a.m. did you delete before breakfast? I did, tonight, mid-post. Tell me I'm not the only one in the comments — and tell me which side of the RAM/VRAM split your build sits on. More field notes at mrsaynothing.dev https://mrsaynothing.dev · code at GitHub https://github.com/mrsaynothing · say hi: contact@mrsaynothing.dev mailto:contact@mrsaynothing.dev