{"slug": "local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen", "title": "Local LLM on a 16GB Mac Mini: Replacing GitHub Copilot with Ollama + Qwen", "summary": "A developer replaced GitHub Copilot with a fully offline coding assistant running on a 16GB M4 Mac mini using Ollama and Qwen models. They found that a 7B parameter model is the sweet spot for chat and edits, while a 1.5B model handles inline autocomplete, with the key constraint being unified memory headroom of about 7-9GB after macOS and development tools. The setup works with VS Code via an OpenAI-compatible API, and benchmarks show it is usable for real work, though frontier hosted models remain superior for large multi-file reasoning.", "body_md": "I kept paying a monthly subscription for a cloud coding assistant while a 16GB M4 Mac mini sat on my desk idling most of the day. So I ran the obvious experiment: **can a 16GB Mac mini run a coding assistant entirely offline** — no code leaving the machine, no subscription — and is it actually usable for real work?\n\nShort answer: yes, with one hard constraint (RAM) and one soft one (context length). This article is the written version of the video above, with every command, config file, and benchmark number so you can reproduce it.\n\nThree reasons, in the order that actually mattered to me:\n\nThe reason *not* to: raw capability. The frontier hosted models are better at large multi-file reasoning, and it isn't close. More on that below.\n\nOn Apple Silicon, the GPU and CPU share one pool of unified memory. A model has to fit in that pool **alongside macOS, your browser, VS Code, and whatever containers you're running**. On a 16GB machine, macOS + a normal dev environment eats 6–8GB before you've loaded anything.\n\nThat leaves you roughly **7–9GB of realistic headroom** for the model. This single number determines everything else, and it's why \"just run the 30B model\" advice from people on 64GB machines doesn't transfer.\n\nBy default macOS allows the GPU to use about 75% of total RAM as VRAM. You can check what you're actually working with:\n\n```\n# total RAM in bytes\nsysctl hw.memsize\n\n# current memory pressure — the number that actually matters\nmemory_pressure | tail -5\n\n# what's using it\ntop -o MEM -n 10 -l 1 | head -20\n```\n\nTwo options. Homebrew is easier to script and update:\n\n```\nbrew install --cask ollama\n```\n\nOr download the app directly from [ollama.com](https://ollama.com/). Either way, verify the daemon is up:\n\n```\nollama --version\ncurl -s http://localhost:11434/api/tags | head\n```\n\nIf that `curl`\n\nreturns JSON, the local API server is live on port `11434`\n\n. That endpoint is what VS Code will talk to — it is OpenAI-API-compatible enough for most tooling.\n\nIf it isn't running:\n\n```\n# start the server in the foreground to see logs\nollama serve\n```\n\nLM Studio alternative:if you'd rather have a GUI with a model browser and a built-in chat window, LM Studio does the same job and also exposes an OpenAI-compatible server (default port`1234`\n\n). Everything below works with either — swap the`apiBase`\n\nport.\n\nThis is where most local-LLM writeups go wrong. Here's the actual size on disk (and roughly in memory) for the Qwen coder family:\n\n| Model | Download size | Fits in 16GB? | Use it for |\n|---|---|---|---|\n`qwen2.5-coder:1.5b` |\n986 MB | ✅ Trivially | Autocomplete only |\n`qwen2.5-coder:3b` |\n1.9 GB | ✅ Easily | Autocomplete, light chat |\n`qwen2.5-coder:7b` |\n4.7 GB | ✅ Sweet spot\n|\nChat + edit + autocomplete |\n`qwen2.5-coder:14b` |\n9.0 GB | ⚠️ Tight — close other apps | Best quality you can get |\n`qwen2.5-coder:32b` |\n20 GB | ❌ No | — |\n`qwen3-coder:30b` (a3b) |\n19 GB | ❌ No | Needs 32GB+ |\n\n**The 16GB recommendation: qwen2.5-coder:7b for chat and edits, qwen2.5-coder:1.5b for inline autocomplete.** Running a small dedicated autocomplete model alongside the bigger chat model is the trick that makes the whole thing feel responsive — autocomplete needs to answer in milliseconds, and a 7B can't.\n\nPull them:\n\n```\nollama pull qwen2.5-coder:7b\nollama pull qwen2.5-coder:1.5b\n\n# optional: embeddings for codebase indexing\nollama pull nomic-embed-text\n\nollama list\n```\n\nIf you have the RAM headroom and want to try 14B, pull an explicit quantization rather than the default — `q4_K_M`\n\nis the best quality-per-gigabyte tradeoff:\n\n```\nollama pull qwen2.5-coder:14b-instruct-q4_K_M\nollama run qwen2.5-coder:7b\n```\n\nThen, to see actual timings instead of vibes, use verbose mode:\n\n```\nollama run --verbose qwen2.5-coder:7b \"Write a Python function that parses an ISO 8601 duration string into seconds. Include edge cases.\"\n```\n\n`--verbose`\n\nprints `total duration`\n\n, `prompt eval rate`\n\n, and `eval rate`\n\n(tokens/sec) after every response. That's your benchmark instrument — no extra tooling needed.\n\nWhile it's generating, watch memory in another terminal:\n\n```\nollama ps        # shows loaded models, size, and CPU/GPU split\n```\n\nThe `PROCESSOR`\n\ncolumn in `ollama ps`\n\nshould say `100% GPU`\n\n. If it says anything with `CPU`\n\n, the model spilled out of unified memory and your tokens/sec just fell off a cliff — drop to a smaller model or quantization.\n\nInstall the [Continue](https://marketplace.visualstudio.com/items?itemName=Continue.continue) extension, then edit `~/.continue/config.yaml`\n\n:\n\n```\nname: Local Mac Mini Config\nversion: 0.0.1\nschema: v1\n\nmodels:\n  - name: Qwen2.5 Coder 7B\n    provider: ollama\n    model: qwen2.5-coder:7b\n    apiBase: http://localhost:11434\n    roles:\n      - chat\n      - edit\n      - apply\n    defaultCompletionOptions:\n      contextLength: 8192\n      maxTokens: 2048\n\n  - name: Qwen2.5 Coder 1.5B (autocomplete)\n    provider: ollama\n    model: qwen2.5-coder:1.5b\n    apiBase: http://localhost:11434\n    roles:\n      - autocomplete\n    defaultCompletionOptions:\n      contextLength: 2048\n      maxTokens: 256\n\n  - name: Nomic Embed\n    provider: ollama\n    model: nomic-embed-text\n    roles:\n      - embed\n\ncontext:\n  - provider: code\n  - provider: diff\n  - provider: terminal\n  - provider: currentFile\n```\n\nTwo things worth calling out:\n\n`model:`\n\nmust match `ollama list`\n\nexactly.`contextLength: 8192`\n\nRestart VS Code, open the Continue panel, and confirm the model dropdown shows your local models. Inline autocomplete should start firing as you type.\n\nThree environment variables do most of the work. Set them where Ollama can see them — if you run the app, use `launchctl`\n\n; if you run `ollama serve`\n\nyourself, put them in your shell profile.\n\n```\n# keep the model resident so you don't pay reload cost on every request\nlaunchctl setenv OLLAMA_KEEP_ALIVE \"30m\"\n\n# only one model in memory at a time — critical on 16GB\nlaunchctl setenv OLLAMA_MAX_LOADED_MODELS \"1\"\n\n# don't let concurrent requests multiply your memory footprint\nlaunchctl setenv OLLAMA_NUM_PARALLEL \"1\"\n```\n\nThen restart Ollama for them to take effect.\n\nThe counterintuitive one is `OLLAMA_MAX_LOADED_MODELS=1`\n\n. It seems to fight the two-model setup from Step 4 — and it does mean a swap when you jump between chat and autocomplete. But on 16GB, having both a 7B and a 1.5B resident *plus* a browser open is what pushes you into swap, and swap on a local LLM is catastrophic, not slow. If you have the headroom (nothing else open), set it to `2`\n\nand enjoy the snappier switching.\n\nIf you want to reclaim memory immediately:\n\n```\nollama stop qwen2.5-coder:7b\n```\n\nI'm deliberately not handing you a table of my numbers. Throughput on Apple Silicon swings with macOS version, thermal state, and whatever else is resident in unified memory — a benchmark from someone else's Mac mini tells you almost nothing about yours. Here's the two-minute version that gives you real figures:\n\n```\nfor m in qwen2.5-coder:1.5b qwen2.5-coder:3b qwen2.5-coder:7b; do\n  echo \"=== $m ===\"\n  ollama run --verbose \"$m\" \\\\\n    \"Write a Python function that retries an HTTP request with exponential backoff. Include type hints and docstring.\" \\\\\n    2>&1 | tail -8\ndone\n```\n\nThe number to read is ** eval rate** (tokens/sec).\n\n`prompt eval rate`\n\nmatters less for interactive coding — it's how fast it ingests your file, and it's rarely the bottleneck at 8K context.Run it a few times and take the median; the first run of any model includes load time and will look worse than reality.\n\nRules of thumb that held up across my runs:\n\nAnd keep an eye on `ollama ps`\n\nwhile it runs — if `PROCESSOR`\n\nshows any CPU percentage, the model spilled out of unified memory and the numbers you're reading are meaningless. On 16GB that's the single most common reason people conclude \"local models are too slow.\"\n\n**Genuinely good at:**\n\n**Falls down on:**\n\nThe honest framing: a local 7B is roughly a competent junior who has read all the docs, works instantly, never leaks your code, and cannot see past the current file.\n\nDepends entirely on your work:\n\nI've been running this setup as my default and reaching for the cloud only when the local model visibly struggles. That split has held up.\n\n**The full video walkthrough** — install, model runs, VS Code hookup, and the live coding tests — is at the top of this post, or here: [Goodbye GitHub Copilot? Building a Local AI Lab on a 16GB Mac Mini](https://youtu.be/6HlT4vkB-38).\n\nWhich model should I benchmark next on 16GB? Drop it in the comments — I'll run it.", "url": "https://wpnews.pro/news/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen", "canonical_source": "https://dev.to/chennarao_vemula_aa375143/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen-n47", "published_at": "2026-08-14 18:35:16+00:00", "updated_at": "2026-08-14 19:05:49.984776+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "artificial-intelligence"], "entities": ["GitHub Copilot", "Ollama", "Qwen", "Mac mini", "VS Code", "LM Studio", "Apple Silicon"], "alternates": {"html": "https://wpnews.pro/news/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen", "markdown": "https://wpnews.pro/news/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen.md", "text": "https://wpnews.pro/news/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen.txt", "jsonld": "https://wpnews.pro/news/local-llm-on-a-16gb-mac-mini-replacing-github-copilot-with-ollama-qwen.jsonld"}}