"Will this model run on my Mac?" usually gets answered with the model's file size. That's only a third of the answer. While a model runs, memory goes to three things: the weights, the context (KV cache) and macOS with everything else you have open. I measured three models on my MacBook M5 with 16 GB, calculated the context cost for classic 8B–70B models, and put together what actually fits into 16 to 128 GB.
recommendedMaxWorkingSetSize is 12.7 GB, about three quarters. The rest belongs to macOS, the browser and the editor.
So the working rule is: weights + context ≤ ¾ of your Mac's RAM. You can raise the GPU limit with sysctl iogpu.wired_limit_mb, but then apps get less and macOS starts swapping.
Setup: Apple M5, 16 GB, Ollama 0.31, MLX builds. Each model ran in three modes: a short request (a 150-word business email) with a 4K window; a 4.2K-token document (~10 pages) with an 8K window; an 18K-token document (~45 pages) with a 32K window, asking for five key takeaways. Memory is what ollama ps reports. Answers were capped at 200 tokens.
| Model, mode | Memory | Prompt reading | Generation | Total time |
|---|---|---|---|---|
| Qwen 3.5 0.8B, short | 1.08 GB | — | 109 tok/s | 3 s |
| Qwen 3.5 0.8B, 10 pages | 1.14 GB | 2257 tok/s | 103 tok/s | 4 s |
| Qwen 3.5 0.8B, 45 pages | 1.73 GB | 1952 tok/s | 86 tok/s | 12 s |
| Qwen 3.5 4B, short | 3.43 GB | — | 34 tok/s | 8 s |
| Qwen 3.5 4B, 10 pages | 3.59 GB | 390 tok/s | 32 tok/s | 18 s |
| Qwen 3.5 4B, 45 pages | 4.81 GB | 312 tok/s | 27 tok/s | 67 s |
| Gemma 4 12B, short | 7.64 GB | — | 25 tok/s | 14 s |
| Gemma 4 12B, 10 pages | 8.29 GB | 102 tok/s | 16 tok/s | 57 s |
| Gemma 4 12B, 45 pages | 9.21 GB | 72 tok/s | 9 tok/s | 4.8 min |
What stands out:
In classic architectures (Llama 3, Qwen 3 and most 2025 models) every layer caches the whole window. The size follows from the config: 2 × layers × KV heads × head dim × 2 bytes. From the config.json files on Hugging Face, at 16-bit (the default in Ollama and llama.cpp):
| Model | Q4 weights | Cache per token | 8K | 32K | 128K |
|---|---|---|---|---|---|
| Llama 3.1 8B (32 layers) | ~5 GB | 128 KB | 1 GB | 4 GB | 16 GB |
| Qwen3 14B (40 layers) | ~9 GB | 160 KB | 1.25 GB | 5 GB | 20 GB |
| Qwen3 32B (64 layers) | ~19 GB | 256 KB | 2 GB | 8 GB | 32 GB |
| Llama 3.3 70B (80 layers) | ~40 GB | 320 KB | 2.5 GB | 10 GB | 40 GB |
| Qwen 3.5 4B / Gemma 4 12B (my measurement) | 3.4–7.6 GB | ~50 KB | 0.4 GB | 1.5 GB | ~6 GB |
At a full 128K window, a classic model's cache weighs as much as the 4-bit model itself. A 70B model with that window needs 80 GB.
Applying "weights + context ≤ ¾ RAM" with the classic-model cache numbers above (newer models with a lean cache fit with room to spare). This is a calculation, not a measurement on every machine:
| Mac RAM | For the model | Short requests, 8K | Documents, 32K | Whole books, 128K |
|---|---|---|---|---|
| 16 GB | ~12 GB | up to 14B, tight | 8B; Gemma 4 12B (9.2 GB measured) | only new-gen 4B |
| 24 GB | ~18 GB | 14B comfortably | 14B (14 GB) | only new-gen up to 12B |
| 32 GB | ~24 GB | 32B, tight (21 GB) | 14B comfortably | 8B (21 GB) |
| 48 GB | ~36 GB | 32B comfortably | 32B (27 GB) | 14B (29 GB) |
| 64 GB | ~48 GB | 70B, tight (43 GB) | 32B comfortably | 32B only with a compressed cache | | 96 GB | ~72 GB | 70B comfortably | 70B (50 GB) | 32B (51 GB) | | 128 GB | ~96 GB | 70B comfortably | 70B comfortably | 70B (80 GB) |
num_ctx, in LM Studio it's Context Length. OLLAMA_KV_CACHE_TYPE=q8_0 (with OLLAMA_FLASH_ATTENTION=1) stores the cache in 8 bits and halves it, with little quality impact. That lets a 32B model with a 128K window fit in 64 GB.
How much memory does your Mac have, and which local model do you actually run every day?
*Originally published at [klukyanov.ru](https://klukyanov.ru/notes/llm-memory-mac/).*
*Shorter weekly write-ups (in Russian) — on [Telegram](https://t.me/+KSAy8mfFOsgwYWMy).*