# Local LLMs with Ollama on macOS

> Source: <https://gist.github.com/nprasad2077/b8c33fbec67a160b23772e642ffcb1fa>
> Published: 2026-11-03 13:17:19+00:00

Setup for a MacBook Pro M4 Pro (24GB) with models stored on the external drive `/Volumes/ROG_PRO`.
Verified on this machine: `gpt-oss:20b` loads from the drive and generates at ~52 tok/s (Ollama 0.40.14).

```
brew install ollama      # CLI
# or download the app from https://ollama.com
```

Check: `ollama -v`

Models live in `OLLAMA_MODELS`. Default is `~/.ollama/models`.

```
mkdir -p /Volumes/ROG_PRO/ollama/models
touch /Volumes/ROG_PRO/ollama/models/.t && rm /Volumes/ROG_PRO/ollama/models/.t   # writable?
```

If not writable: Finder → drive → Cmd+I → turn off "Ignore ownership". NTFS drives are read-only on macOS; use APFS.

| Setup | Command | 
|---|---|
| Terminal ( `ollama serve` ) | `echo 'export OLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models' >> ~/.zshrc && source ~/.zshrc` | 
| Menu bar app (until reboot) | `launchctl setenv OLLAMA_MODELS /Volumes/ROG_PRO/ollama/models` , then quit and reopen the app | 
| Symlink (works everywhere) | `mv ~/.ollama/models /Volumes/ROG_PRO/ollama/ ; ln -sfn /Volumes/ROG_PRO/ollama/models ~/.ollama/models` | 

Persistent menu-bar setting: create `~/Library/LaunchAgents/ollama.models.plist`:

```
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>ollama.models</string>
  <key>ProgramArguments</key><array>
    <string>/bin/launchctl</string><string>setenv</string>
    <string>OLLAMA_MODELS</string><string>/Volumes/ROG_PRO/ollama/models</string>
  </array>
  <key>RunAtLoad</key><true/>
</dict></plist>
```

Load it: `launchctl load ~/Library/LaunchAgents/ollama.models.plist`

- **"address already in use" on `ollama serve`** : an instance is already running (menu bar app or brew service) and ignores your env var. Stop it first:

```
pkill -f ollama; brew services stop ollama 2>/dev/null; lsof -i :11434   # should print nothing
```

- **Drive unplugged** : Ollama can't see your models or creates an empty folder. Mount the drive before starting it.
- macOS may ask for Removable Volumes / Full Disk Access for Ollama. Allow it.

```
OLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models ollama serve
```

Leave it running; use a second terminal for everything else. Server: `http://127.0.0.1:11434`.

Useful server variables:

| Variable | Purpose | Example | 
|---|---|---|
| `OLLAMA_MODELS` | Model storage path | `/Volumes/ROG_PRO/ollama/models` | 
| `OLLAMA_CONTEXT_LENGTH` | Default context window (default 4096) | `32768` | 
| `OLLAMA_KEEP_ALIVE` | How long a model stays in RAM (default 5m) | `30m` | 
| `OLLAMA_MAX_LOADED_MODELS` | Concurrent models | `1` | 
| `OLLAMA_HOST` | Bind address | `127.0.0.1:11434` | 

```
ollama pull gpt-oss:20b        # download
ollama list                    # installed models
ollama run gpt-oss:20b         # interactive chat (/bye to exit)
ollama run gpt-oss:20b "text"  # one-shot
ollama ps                      # loaded models
ollama stop gpt-oss:20b        # unload from RAM
ollama show gpt-oss:20b        # details
ollama cp a b                  # copy/rename
ollama rm gpt-oss:20b          # delete (frees drive space)
```

Verify storage: `ls /Volumes/ROG_PRO/ollama/models/blobs` and `du -sh /Volumes/ROG_PRO/ollama/models`.

Budget about 17–20GB for the model. Q4_K_M costs ~0.6 GB per billion parameters. Speeds below are third-party estimates (modelfit.io), not benchmarks. Only `gpt-oss:20b` has been measured here (~52 tok/s).

| Model | Size | Est. speed | Use | 
|---|---|---|---|
| `gpt-oss:20b` | ~14 GB | ~53 tok/s | Daily driver, reasoning, coding | 
| `qwen3.5:9b` | ~7 GB | ~43 tok/s | Multimodal, light on RAM | 
| `gemma4:26b` | ~16 GB | ~53 tok/s | Text + images | 
| `lfm2:24b-a2b` | ~14 GB | ~92 tok/s | Agents / tool calling | 
| `qwen3.8:27b` | ~17.5 GB | ~14 tok/s | Max quality, tight fit | 
| `ministral-3:14b` | ~9 GB | ~24 tok/s | Multimodal 14B | 

Coding: `gpt-oss:20b` is the easy choice in Ollama. JetBrains Mellum2.1 12B-A2.5B Thinking is coding-specific but has no Ollama tag (see §8).
Avoid: 70B models (~43GB, won't fit), Q8 above 14B, 32B at Q4 (swaps).

Docker:

```
docker run -d -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main
```

Without Docker (Python 3.11): `pip install open-webui && open-webui serve` (port 8080).

Open `http://localhost:3000`, create the local admin account, pick the model. If none appear: Settings → Connections → Ollama URL `http://host.docker.internal:11434` (Docker) or `http://localhost:11434`.

Open WebUI's own data (chats, settings) is on the internal disk, not ROG_PRO.

```
brew install sst/tap/opencode      # or: npm i -g opencode-ai
```

Raise the context first. The 4096 default is too small for agent prompts:

```
OLLAMA_CONTEXT_LENGTH=32768 OLLAMA_MODELS=/Volumes/ROG_PRO/ollama/models ollama serve
```

32K fits comfortably with `gpt-oss:20b`. Drop to 16K if memory gets tight.

`~/.config/opencode/opencode.json` (global) or `opencode.json` in the project:

```
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": { "gpt-oss:20b": { "name": "GPT-OSS 20B" } }
    }
  }
}
```

Run `opencode` in a project directory, then `/models` and choose `Ollama (local)`.

A 20B local model handles small edits and questions. It is weaker than hosted models at long multi-step agent work, so review its changes.

- **LM Studio** : My Models tab → change models directory to the drive.
- **llama.cpp** (e.g. Mellum2.1):`LLAMA_CACHE=/Volumes/ROG_PRO/llama llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M`
- **Hugging Face / MLX** :`export HF_HOME=/Volumes/ROG_PRO/hf`

```
# Native
curl http://localhost:11434/api/chat -d '{"model":"gpt-oss:20b","messages":[{"role":"user","content":"hi"}],"stream":false}'

# OpenAI-compatible (what OpenCode and most tools use)
curl http://localhost:11434/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"gpt-oss:20b","messages":[{"role":"user","content":"hi"}]}'
```

| Symptom | Fix | 
|---|---|
| Models missing after reboot | `OLLAMA_MODELS` not persisted, or drive not mounted. See §2. | 
| Pull writes to `~/.ollama` | Server started without the env var. Restart per §2 gotchas. | 
| Slow or swapping | Model too big, or too many apps open. Use a smaller model, lower context, `ollama stop` . | 
| Agent forgets or truncates | Context still 4096. Set `OLLAMA_CONTEXT_LENGTH` . | 
| Open WebUI sees no models | Wrong Ollama URL. Use `host.docker.internal` from Docker. | 
| Slow first response | Initial model load (~18s here). Raise `OLLAMA_KEEP_ALIVE` . |