{"slug": "nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem", "title": "Nobody talks about RAM. Every local-LLM regret is a RAM problem.", "summary": "A developer benchmarked local LLM memory usage on a Ryzen desktop with 32 GB of RAM and a 12 GB RTX 3060, finding that a 4.9 GB llama3.1:8b model reserved 7.0 GB of VRAM at a 32,000-token context window — a 43% \"context tax\" driven by the KV cache rather than model weights. The writeup argues system RAM, not VRAM, determines whether larger models like 14B and 27B run at all, since they can offload to 32 GB of RAM but not fit in a 12 GB card.", "body_md": "*Originally published at [mrsaynothing.dev](https://mrsaynothing.dev/en/blog/2026-09-19/nobody-talks-about-ram?utm_source=devto&utm_medium=referral)*.\n\nWalk into any local-LLM thread and the arguments are about GPUs. VRAM benchmarks, 24 GB cards, CUDA versus ROCm, whether the 3060 is still the people's card. Meanwhile the number that actually decides whether your model runs is sitting in the other slot, unbenchmarkable and unmarketed: how much RAM the machine has.\n\n**VRAM sells the dream. RAM decides whether the model boots at all — and how much context survives when it does.**\n\nI tested this on the box in front of me while writing this post: a Ryzen desktop with 32 GB of RAM and a GeForce RTX 3060 with 12 GB of VRAM. Pulled `llama3.1:8b` — the download page says 4.9 GB. Watch what it actually reserved:\n\nThat is the whole argument in one screen. A \"4.9 GB model\" reserved **7.0 GB** before answering a single prompt — a **43% context tax** — and kept **7,963 of 12,288 MiB** of VRAM to itself. The weights were never the budget. The context was.\n\nThe [llama.cpp memory notes](https://github.com/ggml-org/llama.cpp/blob/master/docs/memory.md) spell out the arithmetic that download pages omit: total memory = model weights + KV cache + compute buffer. Only the first term is constant. The KV cache grows linearly with context length, and the compute buffer grows with batch and graph shape. Ollama wraps llama.cpp, so the same law applies — which is why `ollama ps` reported 7.0 GB for a 4.9 GB tag at a 32,000-token window, all of it resident on the GPU.\n\n**A quantized model isn't a compromise. It's an admission that memory was always the real budget.**\n\nThis is also why the GGUF quantization ladder exists at all. Q4 isn't a religion; it's what makes the memory arithmetic land inside hardware people own. And it's why two \"identical\" 8B setups behave nothing alike: same model, different context length, different machines entirely.\n\nSame three model classes, both memory types, one 12 GB card and 32 GB of RAM — the configuration thousands of developers actually have:\n\n| Model class (Q4) | Download | Loaded + 32k ctx | On 12 GB VRAM | On 32 GB RAM | \n|---|---|---|---|---|\n| 7–8B ( `llama3.1:8b` ) | 4.9 GB | 7.0 GB (measured) | 100% GPU, ~8 GiB used | barely noticed | \n| 13–14B ( `qwen2.5:14b` ) | 9.0 GB | ~12 GB | offload begins | fine | \n| 27–32B ( `gemma3:27b` ) | 17 GB | ~20+ GB | CPU pulls the weight | the only reason it runs | \n\nRead the last two rows again. On VRAM alone, a 14B model at a real context window is already a split-offload job, and a 27B is impossible. On 32 GB of system RAM, both are merely *slow*. That difference — between impossible and slow — is the entire practical difference between RAM and VRAM. If it fits in VRAM, it's fast. If it fits in RAM, it works. If it fits in neither, you're swapping to an NVMe drive and time loses meaning.\n\nOffload sits on PCIe, and the [Ollama FAQ](https://docs.ollama.com/faq/) is straight about the cost: layers that don't fit on the GPU run on the CPU, and throughput drops hard as the GPU share shrinks. Nobody chooses that trade-off consciously. It happens silently, one layer at a time, and the symptom is just \"local models feel overrated.\"\n\nThings that broke or surprised me while writing this piece, in order:\n\n`ps` output.`ollama rm` cycle keeps the disk honest.\nNone of this means GPUs don't matter — the 100% GPU line in that capture is why generation felt instant. It means the GPU is the second question. The runner choice comes *after* you know what fits, not before.\n\nRAM you need = model file + KV cache for your real context window + 4 GB for being a computer. For 7–8B at Q4, 16 GB is comfortable. For 14B–32B, 32 GB stops being a luxury and starts being the point. Buy VRAM for the speed you want at the context you use; buy RAM for everything you'll ever load.\n\n**Check `ollama ps` once and the spec-sheet religion quietly ends.**\n\nSo, two questions. When you spec your next dev box, are you buying VRAM for the benchmarks you'll post — or RAM for the models you'll actually run? And honestly: how many of the models you pulled at 2 a.m. did you delete before breakfast? I did, tonight, mid-post. Tell me I'm not the only one in the comments — and tell me which side of the RAM/VRAM split your build sits on.\n\n*More field notes at [mrsaynothing.dev](https://mrsaynothing.dev) · code at [GitHub](https://github.com/mrsaynothing) · say hi: [contact@mrsaynothing.dev](mailto:contact@mrsaynothing.dev)*", "url": "https://wpnews.pro/news/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem", "canonical_source": "https://dev.to/mrsaynothing/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem-177h", "published_at": "2026-09-19 16:53:47+00:00", "updated_at": "2026-09-19 17:23:12.257917+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "mlops"], "entities": ["Ollama", "llama.cpp", "llama3.1:8b", "qwen2.5:14b", "gemma3:27b", "Nvidia", "GeForce RTX 3060", "AMD Ryzen"], "alternates": {"html": "https://wpnews.pro/news/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem", "markdown": "https://wpnews.pro/news/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem.md", "text": "https://wpnews.pro/news/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem.txt", "jsonld": "https://wpnews.pro/news/nobody-talks-about-ram-every-local-llm-regret-is-a-ram-problem.jsonld"}}