{"slug": "running-local-llms-on-8gb-ram-laptops-what-s-actually-possible", "title": "Running Local LLMs on 8GB RAM Laptops: What's Actually Possible", "summary": "A 4-bit quantized Qwen 3 4B model is the practical ceiling for running a local LLM on an 8GB RAM laptop through llama.cpp, generating roughly 15 to 18 tokens per second on integrated graphics from an Intel Core i5 (Wildcat Lake) or Apple A18 Pro chip, according to hands-on memory and token-speed tests. The same tests found Qwen 3 8B at 4-bit does not fit: llama.cpp reported only 4.3GB to 5.3GB of GPU-accessible memory on two 8GB machines, forcing CPU spillover that cut generation to around 7 tokens per second on better hardware and under one token per second in the worst case. The tests conclude that 8GB is the floor for small quantized models rather than a comfortable baseline, and that 16GB configurations open meaningfully more headroom.", "body_md": "# Running Local LLMs on 8GB RAM Laptops: What's Actually Possible\n\nWhat quantized Qwen 3 models can realistically do on 8GB-RAM laptops with llama.cpp, based on real token speed and memory tests.\n\n## Can you actually run a local LLM on 8GB of RAM?\n\nYes, but only within limits. On an 8GB RAM laptop, a 4-bit quantized Qwen 3 4B model runs comfortably through llama.cpp, producing somewhere in the range of 15 to 18 tokens per second on integrated graphics from either an Intel Core i5 (Wildcat Lake) or an Apple A18 Pro chip. Step up to an 8B parameter model, though, and 8GB stops being enough. The GPU portion runs out of memory, work spills onto the CPU, and generation speed can fall to a handful of tokens per second or worse.\n\n## TL;DR\n\n- **4-bit quantization** shrinks a model from 16-bit weights down to roughly a quarter of the size, which is what makes a 4B-parameter model fit inside 8GB of shared system memory at all.\n- On integrated-graphics laptops, **Qwen 3 4B at 4-bit** is the practical ceiling for 8GB RAM, running at around 15 to 18 tokens per second for text generation in real tests.\n- **Qwen 3 8B does not fit** in the GPU-accessible portion of 8GB unified memory. In one test, only 4.3GB to 5.3GB of the total 8GB was actually available to the GPU, short of what an 8B 4-bit model needs.\n- When a model overflows into **system RAM and CPU compute** , speeds can crater from double digits down to under one token per second, since CPUs handle the parallel math of inference far worse than GPUs.\n- **Prompt processing and token generation are different workloads.** A chip can read a prompt faster than it writes a response, and the two stages don’t scale the same way across hardware.\n- Integrated graphics on both Intel and Apple Silicon chips have gotten good enough that **small quantized models run similarly well** across both, with differences measured in single-digit tokens per second rather than orders of magnitude.\n- **More RAM is the single biggest lever** for running larger local models. 8GB is the floor for small models, not a comfortable baseline, and configurations with 16GB open up meaningfully more headroom.\n\n## What does quantization actually buy you?\n\nA language model’s weights are normally stored in 16-bit precision. Quantization compresses those weights down to a lower bit depth, most commonly 4-bit for consumer hardware, trading some precision for a much smaller memory footprint. A 4-billion-parameter model at 4-bit quantization lands at roughly 5GB on disk and in memory. That’s small enough to load into the memory pool an 8GB laptop can spare, with a little room left over for the context window and runtime overhead.\n\nThis is why 4-bit quantized models dominate discussions of “local AI on a laptop.” Without quantization, even a 4B model at full precision would eat most or all of an 8GB machine’s memory before you’ve typed a single prompt.\n\n## Why does an 8GB laptop struggle with 8B models?\n\nThe core problem is that integrated GPUs don’t have their own dedicated video memory. They borrow from total system RAM, and only part of that RAM is ever made available to the GPU side of things. In real testing, llama.cpp reported just 4.3GB of GPU-accessible memory on one 8GB laptop and 5.3GB on another, despite both machines having 8GB installed in total.\n\nAn 8B parameter model at 4-bit quantization needs more memory than that just to hold its weights, before accounting for the active context. So the model can’t fully load onto the GPU. llama.cpp falls back to splitting the load, running part of the model on the GPU and the rest on the CPU, with the overflow pulling from general system RAM instead of a reserved GPU pool.\n\nCPUs are built for sequential, general-purpose work, not the dense parallel matrix math that transformer inference needs. The result in testing was a drop from high single-digit or double-digit tokens per second down to a crawl, in one case around 7 tokens per second on better-performing hardware and under a quarter of a token per second on the other, which works out to roughly one token every four seconds. At that point the system is likely also swapping to disk, which compounds the slowdown further.\n\n## How fast is “usable” for local inference?\n\nToken generation speed is what most people mean when they ask if a local model “runs well.” A rough rule of thumb from hands-on testing: double-digit tokens per second feels close to reading speed or faster, which is comfortable for chat-style use. Single-digit speeds, somewhere around 5 to 10 tokens per second, are still usable but noticeably slower than a person reading casually. Below a couple of tokens per second, the experience stops being interactive and becomes more like leaving a batch job running in the background.\n\nIn practice, the 4B model at 4-bit quantization landed solidly in the comfortable zone on 8GB laptops, generating text at roughly 15 to 18 tokens per second depending on the chip. The 8B model, once it spilled past GPU memory limits, dropped well below the “usable” threshold on at least one machine.\n\n- ✕a coding agent\n- ✕no-code\n- ✕vibe coding\n- ✕a faster Cursor\n\nThe one that tells the coding agents what to build.\n\nIt’s also worth separating prompt processing from generation. Prompt processing, sometimes shortened to PP, is how fast the model reads and encodes the input you give it. Token generation is how fast it writes the response. These two stages can behave differently across hardware. In one comparison, an Intel integrated GPU processed prompts faster than an Apple A18 Pro’s GPU, but the Apple chip generated response tokens faster. Neither number alone tells the whole story.\n\n## Does the chip (Intel vs Apple Silicon) matter for local AI?\n\nLess than you’d expect, at least for models small enough to fit in 8GB. Both Intel’s integrated graphics and Apple Silicon’s GPU cores have improved enough that a 4-bit Qwen 3 4B model ran at broadly comparable speeds on both platforms, with differences in the range of a few tokens per second rather than anything dramatic. llama.cpp also supports Apple’s Metal backend for GPU acceleration on Apple Silicon, and that backend has seen meaningful performance improvements recently, which narrows the gap further.\n\nThe bigger differentiator isn’t brand, it’s memory headroom. Because both platforms use unified or shared memory architectures for integrated graphics, the amount of total RAM and how much of it the GPU can actually claim matters more than which company made the chip. A machine with 16GB of shared memory will have meaningfully more room for larger quantized models than one maxed out at 8GB, regardless of whether it’s running Apple Silicon or an Intel integrated GPU.\n\n## Is 8GB of RAM enough for serious local AI work?\n\nNot really, if “serious” means running anything beyond small models. 8GB is enough to experiment with 4-bit quantized models in the 3B to 4B parameter range and get genuinely usable performance. It is not enough to reliably run 7B or 8B parameter models without hitting memory overflow and the steep performance penalty that comes with spilling onto the CPU and system RAM.\n\nFor anyone planning to do more than casual experimentation with local LLMs, laptops configurable with 16GB of memory (or more) give far more practical flexibility. It’s the single clearest upgrade path identified in direct hardware testing: the laptop’s compute speed mattered far less for AI workloads than whether the model fit in available memory in the first place.\n\n## Frequently Asked Questions\n\n### What is 4-bit quantization in simple terms?\n\nIt’s a method of compressing a model’s weights from their original 16-bit format down to 4 bits each, dramatically shrinking the file size and memory needed to run the model, at some cost to precision.\n\n### What’s the largest model I can run well on 8GB RAM?\n\nBased on real-world testing, a 4-bit quantized model around 4 billion parameters (such as Qwen 3 4B) is the practical ceiling for comfortable, double-digit-tokens-per-second performance on an 8GB laptop.\n\n### Why does an 8B model run so much slower than a 4B model on the same laptop?\n\nThe 8B model’s memory needs exceed what the integrated GPU can access out of the shared 8GB pool, forcing part of the model onto the CPU and general system RAM, which is far slower at the parallel computation LLM inference requires.\n\n### Does llama.cpp support both Intel and Apple Silicon laptops?\n\nYes. llama.cpp is open source and runs across a wide range of hardware, including Intel integrated graphics and Apple Silicon via its Metal GPU backend.\n\n### Is more RAM or a faster CPU more important for running local LLMs?\n\n### Built like a system. Not vibe-coded.\n\nRemy manages the project — every layer architected, not stitched together at the last second.\n\nMore RAM matters more. If a model doesn’t fit in available memory, performance collapses regardless of how fast the processor is, since the system has to offload work to slower CPU-based computation or disk swapping.", "url": "https://wpnews.pro/news/running-local-llms-on-8gb-ram-laptops-what-s-actually-possible", "canonical_source": "https://www.mindstudio.ai/blog/run-qwen-llama-cpp-low-ram-laptop/", "published_at": "2026-10-28 00:00:00+00:00", "updated_at": "2026-10-28 10:45:57.735639+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "machine-learning"], "entities": ["Qwen 3 4B", "Qwen 3 8B", "llama.cpp", "Intel Core i5", "Wildcat Lake", "Apple A18 Pro"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/running-local-llms-on-8gb-ram-laptops-what-s-actually-possible", "markdown": "https://wpnews.pro/news/running-local-llms-on-8gb-ram-laptops-what-s-actually-possible.md", "text": "https://wpnews.pro/news/running-local-llms-on-8gb-ram-laptops-what-s-actually-possible.txt", "jsonld": "https://wpnews.pro/news/running-local-llms-on-8gb-ram-laptops-what-s-actually-possible.jsonld"}}