I kept paying a monthly subscription for a cloud coding assistant while a 16GB M4 Mac mini sat on my desk idling most of the day. So I ran the obvious experiment: can a 16GB Mac mini run a coding assistant entirely offline — no code leaving the machine, no subscription — and is it actually usable for real work?
Short answer: yes, with one hard constraint (RAM) and one soft one (context length). This article is the written version of the video above, with every command, config file, and benchmark number so you can reproduce it.
Three reasons, in the order that actually mattered to me:
The reason not to: raw capability. The frontier hosted models are better at large multi-file reasoning, and it isn't close. More on that below.
On Apple Silicon, the GPU and CPU share one pool of unified memory. A model has to fit in that pool alongside macOS, your browser, VS Code, and whatever containers you're running. On a 16GB machine, macOS + a normal dev environment eats 6–8GB before you've loaded anything.
That leaves you roughly 7–9GB of realistic headroom for the model. This single number determines everything else, and it's why "just run the 30B model" advice from people on 64GB machines doesn't transfer.
By default macOS allows the GPU to use about 75% of total RAM as VRAM. You can check what you're actually working with:
sysctl hw.memsize
memory_pressure | tail -5
top -o MEM -n 10 -l 1 | head -20
Two options. Homebrew is easier to script and update:
brew install --cask ollama
Or download the app directly from ollama.com. Either way, verify the daemon is up:
ollama --version
curl -s http://localhost:11434/api/tags | head
If that curl
returns JSON, the local API server is live on port 11434
. That endpoint is what VS Code will talk to — it is OpenAI-API-compatible enough for most tooling.
If it isn't running:
ollama serve
LM Studio alternative:if you'd rather have a GUI with a model browser and a built-in chat window, LM Studio does the same job and also exposes an OpenAI-compatible server (default port1234
). Everything below works with either — swap theapiBase
port.
This is where most local-LLM writeups go wrong. Here's the actual size on disk (and roughly in memory) for the Qwen coder family:
| Model | Download size | Fits in 16GB? | Use it for |
|---|---|---|---|
qwen2.5-coder:1.5b |
|||
| 986 MB | ✅ Trivially | Autocomplete only | |
qwen2.5-coder:3b |
|||
| 1.9 GB | ✅ Easily | Autocomplete, light chat | |
qwen2.5-coder:7b |
|||
| 4.7 GB | ✅ Sweet spot | ||
| Chat + edit + autocomplete | |||
qwen2.5-coder:14b |
|||
| 9.0 GB | ⚠️ Tight — close other apps | Best quality you can get | |
qwen2.5-coder:32b |
|||
| 20 GB | ❌ No | — | |
qwen3-coder:30b (a3b) |
|||
| 19 GB | ❌ No | Needs 32GB+ |
The 16GB recommendation: qwen2.5-coder:7b for chat and edits, qwen2.5-coder:1.5b for inline autocomplete. Running a small dedicated autocomplete model alongside the bigger chat model is the trick that makes the whole thing feel responsive — autocomplete needs to answer in milliseconds, and a 7B can't.
Pull them:
ollama pull qwen2.5-coder:7b
ollama pull qwen2.5-coder:1.5b
ollama pull nomic-embed-text
ollama list
If you have the RAM headroom and want to try 14B, pull an explicit quantization rather than the default — q4_K_M
is the best quality-per-gigabyte tradeoff:
ollama pull qwen2.5-coder:14b-instruct-q4_K_M
ollama run qwen2.5-coder:7b
Then, to see actual timings instead of vibes, use verbose mode:
ollama run --verbose qwen2.5-coder:7b "Write a Python function that parses an ISO 8601 duration string into seconds. Include edge cases."
--verbose
prints total duration
, prompt eval rate
, and eval rate
(tokens/sec) after every response. That's your benchmark instrument — no extra tooling needed.
While it's generating, watch memory in another terminal:
ollama ps # shows loaded models, size, and CPU/GPU split
The PROCESSOR
column in ollama ps
should say 100% GPU
. If it says anything with CPU
, the model spilled out of unified memory and your tokens/sec just fell off a cliff — drop to a smaller model or quantization.
Install the Continue extension, then edit ~/.continue/config.yaml
:
name: Local Mac Mini Config
version: 0.0.1
schema: v1
models:
- name: Qwen2.5 Coder 7B
provider: ollama
model: qwen2.5-coder:7b
apiBase: http://localhost:11434
roles:
- chat
- edit
- apply
defaultCompletionOptions:
contextLength: 8192
maxTokens: 2048
- name: Qwen2.5 Coder 1.5B (autocomplete)
provider: ollama
model: qwen2.5-coder:1.5b
apiBase: http://localhost:11434
roles:
- autocomplete
defaultCompletionOptions:
contextLength: 2048
maxTokens: 256
- name: Nomic Embed
provider: ollama
model: nomic-embed-text
roles:
- embed
context:
- provider: code
- provider: diff
- provider: terminal
- provider: currentFile
Two things worth calling out:
model:
must match ollama list
exactly.contextLength: 8192
Restart VS Code, open the Continue panel, and confirm the model dropdown shows your local models. Inline autocomplete should start firing as you type.
Three environment variables do most of the work. Set them where Ollama can see them — if you run the app, use launchctl
; if you run ollama serve
yourself, put them in your shell profile.
launchctl setenv OLLAMA_KEEP_ALIVE "30m"
launchctl setenv OLLAMA_MAX_LOADED_MODELS "1"
launchctl setenv OLLAMA_NUM_PARALLEL "1"
Then restart Ollama for them to take effect.
The counterintuitive one is OLLAMA_MAX_LOADED_MODELS=1
. It seems to fight the two-model setup from Step 4 — and it does mean a swap when you jump between chat and autocomplete. But on 16GB, having both a 7B and a 1.5B resident plus a browser open is what pushes you into swap, and swap on a local LLM is catastrophic, not slow. If you have the headroom (nothing else open), set it to 2
and enjoy the snappier switching.
If you want to reclaim memory immediately:
ollama stop qwen2.5-coder:7b
I'm deliberately not handing you a table of my numbers. Throughput on Apple Silicon swings with macOS version, thermal state, and whatever else is resident in unified memory — a benchmark from someone else's Mac mini tells you almost nothing about yours. Here's the two-minute version that gives you real figures:
for m in qwen2.5-coder:1.5b qwen2.5-coder:3b qwen2.5-coder:7b; do
echo "=== $m ==="
ollama run --verbose "$m" \\
"Write a Python function that retries an HTTP request with exponential backoff. Include type hints and docstring." \\
2>&1 | tail -8
done
The number to read is ** eval rate** (tokens/sec).
prompt eval rate
matters less for interactive coding — it's how fast it ingests your file, and it's rarely the bottleneck at 8K context.Run it a few times and take the median; the first run of any model includes load time and will look worse than reality.
Rules of thumb that held up across my runs:
And keep an eye on ollama ps
while it runs — if PROCESSOR
shows any CPU percentage, the model spilled out of unified memory and the numbers you're reading are meaningless. On 16GB that's the single most common reason people conclude "local models are too slow."
Genuinely good at:
Falls down on:
The honest framing: a local 7B is roughly a competent junior who has read all the docs, works instantly, never leaks your code, and cannot see past the current file.
Depends entirely on your work:
I've been running this setup as my default and reaching for the cloud only when the local model visibly struggles. That split has held up.
The full video walkthrough — install, model runs, VS Code hookup, and the live coding tests — is at the top of this post, or here: Goodbye GitHub Copilot? Building a Local AI Lab on a 16GB Mac Mini.
Which model should I benchmark next on 16GB? Drop it in the comments — I'll run it.