Local LLM on a 16GB Mac Mini: Replacing GitHub Copilot with Ollama + Qwen

A developer replaced GitHub Copilot with a fully offline coding assistant running on a 16GB M4 Mac mini using Ollama and Qwen models. They found that a 7B parameter model is the sweet spot for chat and edits, while a 1.5B model handles inline autocomplete, with the key constraint being unified memory headroom of about 7-9GB after macOS and development tools. The setup works with VS Code via an OpenAI-compatible API, and benchmarks show it is usable for real work, though frontier hosted models remain superior for large multi-file reasoning.

I kept paying a monthly subscription for a cloud coding assistant while a 16GB M4 Mac mini sat on my desk idling most of the day. So I ran the obvious experiment: can a 16GB Mac mini run a coding assistant entirely offline — no code leaving the machine, no subscription — and is it actually usable for real work? Short answer: yes, with one hard constraint RAM and one soft one context length . This article is the written version of the video above, with every command, config file, and benchmark number so you can reproduce it. Three reasons, in the order that actually mattered to me: The reason not to: raw capability. The frontier hosted models are better at large multi-file reasoning, and it isn't close. More on that below. On Apple Silicon, the GPU and CPU share one pool of unified memory. A model has to fit in that pool alongside macOS, your browser, VS Code, and whatever containers you're running . On a 16GB machine, macOS + a normal dev environment eats 6–8GB before you've loaded anything. That leaves you roughly 7–9GB of realistic headroom for the model. This single number determines everything else, and it's why "just run the 30B model" advice from people on 64GB machines doesn't transfer. By default macOS allows the GPU to use about 75% of total RAM as VRAM. You can check what you're actually working with: total RAM in bytes sysctl hw.memsize current memory pressure — the number that actually matters memory pressure | tail -5 what's using it top -o MEM -n 10 -l 1 | head -20 Two options. Homebrew is easier to script and update: brew install --cask ollama Or download the app directly from ollama.com https://ollama.com/ . Either way, verify the daemon is up: ollama --version curl -s http://localhost:11434/api/tags | head If that curl returns JSON, the local API server is live on port 11434 . That endpoint is what VS Code will talk to — it is OpenAI-API-compatible enough for most tooling. If it isn't running: start the server in the foreground to see logs ollama serve LM Studio alternative:if you'd rather have a GUI with a model browser and a built-in chat window, LM Studio does the same job and also exposes an OpenAI-compatible server default port 1234 . Everything below works with either — swap the apiBase port. This is where most local-LLM writeups go wrong. Here's the actual size on disk and roughly in memory for the Qwen coder family: | Model | Download size | Fits in 16GB? | Use it for | |---|---|---|---| qwen2.5-coder:1.5b | 986 MB | ✅ Trivially | Autocomplete only | qwen2.5-coder:3b | 1.9 GB | ✅ Easily | Autocomplete, light chat | qwen2.5-coder:7b | 4.7 GB | ✅ Sweet spot | Chat + edit + autocomplete | qwen2.5-coder:14b | 9.0 GB | ⚠️ Tight — close other apps | Best quality you can get | qwen2.5-coder:32b | 20 GB | ❌ No | — | qwen3-coder:30b a3b | 19 GB | ❌ No | Needs 32GB+ | The 16GB recommendation: qwen2.5-coder:7b for chat and edits, qwen2.5-coder:1.5b for inline autocomplete. Running a small dedicated autocomplete model alongside the bigger chat model is the trick that makes the whole thing feel responsive — autocomplete needs to answer in milliseconds, and a 7B can't. Pull them: ollama pull qwen2.5-coder:7b ollama pull qwen2.5-coder:1.5b optional: embeddings for codebase indexing ollama pull nomic-embed-text ollama list If you have the RAM headroom and want to try 14B, pull an explicit quantization rather than the default — q4 K M is the best quality-per-gigabyte tradeoff: ollama pull qwen2.5-coder:14b-instruct-q4 K M ollama run qwen2.5-coder:7b Then, to see actual timings instead of vibes, use verbose mode: ollama run --verbose qwen2.5-coder:7b "Write a Python function that parses an ISO 8601 duration string into seconds. Include edge cases." --verbose prints total duration , prompt eval rate , and eval rate tokens/sec after every response. That's your benchmark instrument — no extra tooling needed. While it's generating, watch memory in another terminal: ollama ps shows loaded models, size, and CPU/GPU split The PROCESSOR column in ollama ps should say 100% GPU . If it says anything with CPU , the model spilled out of unified memory and your tokens/sec just fell off a cliff — drop to a smaller model or quantization. Install the Continue https://marketplace.visualstudio.com/items?itemName=Continue.continue extension, then edit ~/.continue/config.yaml : name: Local Mac Mini Config version: 0.0.1 schema: v1 models: - name: Qwen2.5 Coder 7B provider: ollama model: qwen2.5-coder:7b apiBase: http://localhost:11434 roles: - chat - edit - apply defaultCompletionOptions: contextLength: 8192 maxTokens: 2048 - name: Qwen2.5 Coder 1.5B autocomplete provider: ollama model: qwen2.5-coder:1.5b apiBase: http://localhost:11434 roles: - autocomplete defaultCompletionOptions: contextLength: 2048 maxTokens: 256 - name: Nomic Embed provider: ollama model: nomic-embed-text roles: - embed context: - provider: code - provider: diff - provider: terminal - provider: currentFile Two things worth calling out: model: must match ollama list exactly. contextLength: 8192 Restart VS Code, open the Continue panel, and confirm the model dropdown shows your local models. Inline autocomplete should start firing as you type. Three environment variables do most of the work. Set them where Ollama can see them — if you run the app, use launchctl ; if you run ollama serve yourself, put them in your shell profile. keep the model resident so you don't pay reload cost on every request launchctl setenv OLLAMA KEEP ALIVE "30m" only one model in memory at a time — critical on 16GB launchctl setenv OLLAMA MAX LOADED MODELS "1" don't let concurrent requests multiply your memory footprint launchctl setenv OLLAMA NUM PARALLEL "1" Then restart Ollama for them to take effect. The counterintuitive one is OLLAMA MAX LOADED MODELS=1 . It seems to fight the two-model setup from Step 4 — and it does mean a swap when you jump between chat and autocomplete. But on 16GB, having both a 7B and a 1.5B resident plus a browser open is what pushes you into swap, and swap on a local LLM is catastrophic, not slow. If you have the headroom nothing else open , set it to 2 and enjoy the snappier switching. If you want to reclaim memory immediately: ollama stop qwen2.5-coder:7b I'm deliberately not handing you a table of my numbers. Throughput on Apple Silicon swings with macOS version, thermal state, and whatever else is resident in unified memory — a benchmark from someone else's Mac mini tells you almost nothing about yours. Here's the two-minute version that gives you real figures: for m in qwen2.5-coder:1.5b qwen2.5-coder:3b qwen2.5-coder:7b; do echo "=== $m ===" ollama run --verbose "$m" \\ "Write a Python function that retries an HTTP request with exponential backoff. Include type hints and docstring." \\ 2 &1 | tail -8 done The number to read is eval rate tokens/sec . prompt eval rate matters less for interactive coding — it's how fast it ingests your file, and it's rarely the bottleneck at 8K context.Run it a few times and take the median; the first run of any model includes load time and will look worse than reality. Rules of thumb that held up across my runs: And keep an eye on ollama ps while it runs — if PROCESSOR shows any CPU percentage, the model spilled out of unified memory and the numbers you're reading are meaningless. On 16GB that's the single most common reason people conclude "local models are too slow." Genuinely good at: Falls down on: The honest framing: a local 7B is roughly a competent junior who has read all the docs, works instantly, never leaks your code, and cannot see past the current file. Depends entirely on your work: I've been running this setup as my default and reaching for the cloud only when the local model visibly struggles. That split has held up. The full video walkthrough — install, model runs, VS Code hookup, and the live coding tests — is at the top of this post, or here: Goodbye GitHub Copilot? Building a Local AI Lab on a 16GB Mac Mini https://youtu.be/6HlT4vkB-38 . Which model should I benchmark next on 16GB? Drop it in the comments — I'll run it.