How much RAM does a local LLM need on a Mac? Measurements on 16 GB and a 16–128 GB sizing table

A developer measured local LLM memory use on a 16 GB Apple M5 MacBook across Qwen 3.5 0.8B, Qwen 3.5 4B and Gemma 4 12B, finding that weights plus KV-cache context must stay under roughly three-quarters of system RAM (12.7 GB on a 16 GB machine) to avoid swapping. The tests showed Gemma 4 12B peaking at 9.21 GB on a 45-page, 32K-window task while slowing to 9 tok/s generation, and the author derived a sizing table from 16 to 128 GB showing a classic 70B model needs about 80 GB at a full 128K window. Setting OLLAMA_KV_CACHE_TYPE=q8_0 with flash attention halves the cache, letting a 32B model with a 128K window fit in 64 GB.

"Will this model run on my Mac?" usually gets answered with the model's file size. That's only a third of the answer. While a model runs, memory goes to three things: the weights, the context KV cache and macOS with everything else you have open. I measured three models on my MacBook M5 with 16 GB, calculated the context cost for classic 8B–70B models, and put together what actually fits into 16 to 128 GB. recommendedMaxWorkingSetSize is 12.7 GB, about three quarters. The rest belongs to macOS, the browser and the editor. So the working rule is: weights + context ≤ ¾ of your Mac's RAM. You can raise the GPU limit with sysctl iogpu.wired limit mb , but then apps get less and macOS starts swapping. Setup: Apple M5, 16 GB, Ollama 0.31, MLX builds. Each model ran in three modes: a short request a 150-word business email with a 4K window; a 4.2K-token document ~10 pages with an 8K window; an 18K-token document ~45 pages with a 32K window, asking for five key takeaways. Memory is what ollama ps reports. Answers were capped at 200 tokens. | Model, mode | Memory | Prompt reading | Generation | Total time | |---|---|---|---|---| | Qwen 3.5 0.8B, short | 1.08 GB | — | 109 tok/s | 3 s | | Qwen 3.5 0.8B, 10 pages | 1.14 GB | 2257 tok/s | 103 tok/s | 4 s | | Qwen 3.5 0.8B, 45 pages | 1.73 GB | 1952 tok/s | 86 tok/s | 12 s | | Qwen 3.5 4B, short | 3.43 GB | — | 34 tok/s | 8 s | | Qwen 3.5 4B, 10 pages | 3.59 GB | 390 tok/s | 32 tok/s | 18 s | | Qwen 3.5 4B, 45 pages | 4.81 GB | 312 tok/s | 27 tok/s | 67 s | | Gemma 4 12B, short | 7.64 GB | — | 25 tok/s | 14 s | | Gemma 4 12B, 10 pages | 8.29 GB | 102 tok/s | 16 tok/s | 57 s | | Gemma 4 12B, 45 pages | 9.21 GB | 72 tok/s | 9 tok/s | 4.8 min | What stands out: In classic architectures Llama 3, Qwen 3 and most 2025 models every layer caches the whole window. The size follows from the config: 2 × layers × KV heads × head dim × 2 bytes. From the config.json files on Hugging Face, at 16-bit the default in Ollama and llama.cpp : | Model | Q4 weights | Cache per token | 8K | 32K | 128K | |---|---|---|---|---|---| | Llama 3.1 8B 32 layers | ~5 GB | 128 KB | 1 GB | 4 GB | 16 GB | | Qwen3 14B 40 layers | ~9 GB | 160 KB | 1.25 GB | 5 GB | 20 GB | | Qwen3 32B 64 layers | ~19 GB | 256 KB | 2 GB | 8 GB | 32 GB | | Llama 3.3 70B 80 layers | ~40 GB | 320 KB | 2.5 GB | 10 GB | 40 GB | | Qwen 3.5 4B / Gemma 4 12B my measurement | 3.4–7.6 GB | ~50 KB | 0.4 GB | 1.5 GB | ~6 GB | At a full 128K window, a classic model's cache weighs as much as the 4-bit model itself. A 70B model with that window needs 80 GB. Applying "weights + context ≤ ¾ RAM" with the classic-model cache numbers above newer models with a lean cache fit with room to spare . This is a calculation, not a measurement on every machine: | Mac RAM | For the model | Short requests, 8K | Documents, 32K | Whole books, 128K | |---|---|---|---|---| | 16 GB | ~12 GB | up to 14B, tight | 8B; Gemma 4 12B 9.2 GB measured | only new-gen 4B | | 24 GB | ~18 GB | 14B comfortably | 14B 14 GB | only new-gen up to 12B | | 32 GB | ~24 GB | 32B, tight 21 GB | 14B comfortably | 8B 21 GB | | 48 GB | ~36 GB | 32B comfortably | 32B 27 GB | 14B 29 GB | | 64 GB | ~48 GB | 70B, tight 43 GB | 32B comfortably | 32B only with a compressed cache | | 96 GB | ~72 GB | 70B comfortably | 70B 50 GB | 32B 51 GB | | 128 GB | ~96 GB | 70B comfortably | 70B comfortably | 70B 80 GB | num ctx , in LM Studio it's Context Length. OLLAMA KV CACHE TYPE=q8 0 with OLLAMA FLASH ATTENTION=1 stores the cache in 8 bits and halves it, with little quality impact. That lets a 32B model with a 128K window fit in 64 GB. How much memory does your Mac have, and which local model do you actually run every day? Originally published at klukyanov.ru https://klukyanov.ru/notes/llm-memory-mac/ . Shorter weekly write-ups in Russian — on Telegram https://t.me/+KSAy8mfFOsgwYWMy .