{"slug": "local-llms-with-ollama-on-macos", "title": "Local LLMs with Ollama on macOS", "summary": "A developer documented a working setup for running local LLMs with Ollama on a 24GB MacBook Pro M4 Pro, storing models on an external drive via the OLLAMA_MODELS environment variable. The writeup covers terminal, menu-bar app, symlink and LaunchAgent methods for persisting the path, plus fixes for port conflicts and unmounted drives. On the test machine, gpt-oss:20b loaded from the external drive and generated at roughly 52 tok/s under Ollama 0.40.14.", "body_md": "Setup for a MacBook Pro M4 Pro (24GB) with models stored on the external drive `/Volumes/ROG_PRO`.\nVerified on this machine: `gpt-oss:20b` loads from the drive and generates at ~52 tok/s (Ollama 0.40.14).\n\n```\nbrew install ollama      # CLI\n# or download the app from https://ollama.com\n```\n\nCheck: `ollama -v`\n\nModels live in `OLLAMA_MODELS`. Default is `~/.ollama/models`.\n\n```\nmkdir -p /Volumes/ROG_PRO/ollama/models\ntouch /Volumes/ROG_PRO/ollama/models/.t && rm /Volumes/ROG_PRO/ollama/models/.t   # writable?\n```\n\nIf not writable: Finder → drive → Cmd+I → turn off \"Ignore ownership\". NTFS drives are read-only on macOS; use APFS.\n\n| Setup | Command | \n|---|---|\n| Terminal ( `ollama serve` ) | `echo 'export OLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models' >> ~/.zshrc && source ~/.zshrc` | \n| Menu bar app (until reboot) | `launchctl setenv OLLAMA_MODELS /Volumes/ROG_PRO/ollama/models` , then quit and reopen the app | \n| Symlink (works everywhere) | `mv ~/.ollama/models /Volumes/ROG_PRO/ollama/ ; ln -sfn /Volumes/ROG_PRO/ollama/models ~/.ollama/models` | \n\nPersistent menu-bar setting: create `~/Library/LaunchAgents/ollama.models.plist`:\n\n```\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n<plist version=\"1.0\"><dict>\n  <key>Label</key><string>ollama.models</string>\n  <key>ProgramArguments</key><array>\n    <string>/bin/launchctl</string><string>setenv</string>\n    <string>OLLAMA_MODELS</string><string>/Volumes/ROG_PRO/ollama/models</string>\n  </array>\n  <key>RunAtLoad</key><true/>\n</dict></plist>\n```\n\nLoad it: `launchctl load ~/Library/LaunchAgents/ollama.models.plist`\n\n- **\"address already in use\" on `ollama serve`** : an instance is already running (menu bar app or brew service) and ignores your env var. Stop it first:\n\n```\npkill -f ollama; brew services stop ollama 2>/dev/null; lsof -i :11434   # should print nothing\n```\n\n- **Drive unplugged** : Ollama can't see your models or creates an empty folder. Mount the drive before starting it.\n- macOS may ask for Removable Volumes / Full Disk Access for Ollama. Allow it.\n\n```\nOLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models ollama serve\n```\n\nLeave it running; use a second terminal for everything else. Server: `http://127.0.0.1:11434`.\n\nUseful server variables:\n\n| Variable | Purpose | Example | \n|---|---|---|\n| `OLLAMA_MODELS` | Model storage path | `/Volumes/ROG_PRO/ollama/models` | \n| `OLLAMA_CONTEXT_LENGTH` | Default context window (default 4096) | `32768` | \n| `OLLAMA_KEEP_ALIVE` | How long a model stays in RAM (default 5m) | `30m` | \n| `OLLAMA_MAX_LOADED_MODELS` | Concurrent models | `1` | \n| `OLLAMA_HOST` | Bind address | `127.0.0.1:11434` | \n\n```\nollama pull gpt-oss:20b        # download\nollama list                    # installed models\nollama run gpt-oss:20b         # interactive chat (/bye to exit)\nollama run gpt-oss:20b \"text\"  # one-shot\nollama ps                      # loaded models\nollama stop gpt-oss:20b        # unload from RAM\nollama show gpt-oss:20b        # details\nollama cp a b                  # copy/rename\nollama rm gpt-oss:20b          # delete (frees drive space)\n```\n\nVerify storage: `ls /Volumes/ROG_PRO/ollama/models/blobs` and `du -sh /Volumes/ROG_PRO/ollama/models`.\n\nBudget about 17–20GB for the model. Q4_K_M costs ~0.6 GB per billion parameters. Speeds below are third-party estimates (modelfit.io), not benchmarks. Only `gpt-oss:20b` has been measured here (~52 tok/s).\n\n| Model | Size | Est. speed | Use | \n|---|---|---|---|\n| `gpt-oss:20b` | ~14 GB | ~53 tok/s | Daily driver, reasoning, coding | \n| `qwen3.5:9b` | ~7 GB | ~43 tok/s | Multimodal, light on RAM | \n| `gemma4:26b` | ~16 GB | ~53 tok/s | Text + images | \n| `lfm2:24b-a2b` | ~14 GB | ~92 tok/s | Agents / tool calling | \n| `qwen3.8:27b` | ~17.5 GB | ~14 tok/s | Max quality, tight fit | \n| `ministral-3:14b` | ~9 GB | ~24 tok/s | Multimodal 14B | \n\nCoding: `gpt-oss:20b` is the easy choice in Ollama. JetBrains Mellum2.1 12B-A2.5B Thinking is coding-specific but has no Ollama tag (see §8).\nAvoid: 70B models (~43GB, won't fit), Q8 above 14B, 32B at Q4 (swaps).\n\nDocker:\n\n```\ndocker run -d -p 3000:8080 \\\n  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \\\n  -v open-webui:/app/backend/data \\\n  --name open-webui --restart always \\\n  ghcr.io/open-webui/open-webui:main\n```\n\nWithout Docker (Python 3.11): `pip install open-webui && open-webui serve` (port 8080).\n\nOpen `http://localhost:3000`, create the local admin account, pick the model. If none appear: Settings → Connections → Ollama URL `http://host.docker.internal:11434` (Docker) or `http://localhost:11434`.\n\nOpen WebUI's own data (chats, settings) is on the internal disk, not ROG_PRO.\n\n```\nbrew install sst/tap/opencode      # or: npm i -g opencode-ai\n```\n\nRaise the context first. The 4096 default is too small for agent prompts:\n\n```\nOLLAMA_CONTEXT_LENGTH=32768 OLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models ollama serve\n```\n\n32K fits comfortably with `gpt-oss:20b`. Drop to 16K if memory gets tight.\n\n`~/.config/opencode/opencode.json` (global) or `opencode.json` in the project:\n\n```\n{\n  \"$schema\": \"https://opencode.ai/config.json\",\n  \"provider\": {\n    \"ollama\": {\n      \"npm\": \"@ai-sdk/openai-compatible\",\n      \"name\": \"Ollama (local)\",\n      \"options\": { \"baseURL\": \"http://localhost:11434/v1\" },\n      \"models\": { \"gpt-oss:20b\": { \"name\": \"GPT-OSS 20B\" } }\n    }\n  }\n}\n```\n\nRun `opencode` in a project directory, then `/models` and choose `Ollama (local)`.\n\nA 20B local model handles small edits and questions. It is weaker than hosted models at long multi-step agent work, so review its changes.\n\n- **LM Studio** : My Models tab → change models directory to the drive.\n- **llama.cpp** (e.g. Mellum2.1):`LLAMA_CACHE=/Volumes/ROG_PRO/llama llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M`\n- **Hugging Face / MLX** :`export HF_HOME=/Volumes/ROG_PRO/hf`\n\n```\n# Native\ncurl http://localhost:11434/api/chat -d '{\"model\":\"gpt-oss:20b\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],\"stream\":false}'\n\n# OpenAI-compatible (what OpenCode and most tools use)\ncurl http://localhost:11434/v1/chat/completions -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"gpt-oss:20b\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n```\n\n| Symptom | Fix | \n|---|---|\n| Models missing after reboot | `OLLAMA_MODELS` not persisted, or drive not mounted. See §2. | \n| Pull writes to `~/.ollama` | Server started without the env var. Restart per §2 gotchas. | \n| Slow or swapping | Model too big, or too many apps open. Use a smaller model, lower context, `ollama stop` . | \n| Agent forgets or truncates | Context still 4096. Set `OLLAMA_CONTEXT_LENGTH` . | \n| Open WebUI sees no models | Wrong Ollama URL. Use `host.docker.internal` from Docker. | \n| Slow first response | Initial model load (~18s here). Raise `OLLAMA_KEEP_ALIVE` . |", "url": "https://wpnews.pro/news/local-llms-with-ollama-on-macos", "canonical_source": "https://gist.github.com/nprasad2077/b8c33fbec67a160b23772e642ffcb1fa", "published_at": "2026-11-03 13:17:19+00:00", "updated_at": "2026-11-03 13:20:18.089525+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["Ollama", "Apple", "MacBook Pro", "gpt-oss", "Qwen", "Gemma", "JetBrains"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/local-llms-with-ollama-on-macos", "markdown": "https://wpnews.pro/news/local-llms-with-ollama-on-macos.md", "text": "https://wpnews.pro/news/local-llms-with-ollama-on-macos.txt", "jsonld": "https://wpnews.pro/news/local-llms-with-ollama-on-macos.jsonld"}}