A practical setup guide for running a local AI troubleshooting and support environment on either macOS or Windows using:
Open WebUI for the browser-based chat interfaceOllama as the local model runtimeHugging Face as an optional source for GGUF modelsDocker Compose for running Open WebUIApple Silicon / Metal acceleration through native Ollama
Recommended architecture on macOS: run
Ollama natively on the Macand runOpen WebUI in Docker. This allows Ollama to use Apple Metal acceleration while Open WebUI stays containerized.
The recommended design on both macOS and Windows is to run Ollama natively on the host operating system and run Open WebUI in Docker Desktop.
flowchart TD
A[Browser] -->|http://localhost:3000| B[Open WebUI]
B -->|Docker host bridge| C[Ollama API]
C --> D[Local LLM]
D --> E{Host Hardware}
E -->|macOS| F[Apple Metal / Unified Memory]
E -->|Windows NVIDIA| G[CUDA]
E -->|Windows AMD| H[ROCm / Vulkan where supported]
E -->|CPU fallback| I[CPU]
The network path is:
sequenceDiagram
participant U as User Browser
participant W as Open WebUI Container
participant O as Ollama on Host
participant M as Local Model
U->>W: Prompt via http://localhost:3000
W->>O: API request to host.docker.internal:11434
O->>M: Run inference
M-->>O: Generated tokens
O-->>W: Stream response
W-->>U: Display response
host.docker.internal
is the important hostname here. It allows the Open WebUI container to reach Ollama running on the Windows or macOS host.
You need:
- Docker Desktop
- Docker Compose v2
- Ollama
- Sufficient RAM / unified memory
- Free disk space for model files
- A supported version of macOS or Windows
The installation path depends on your operating system.
flowchart LR
A[Start] --> B{Operating System}
B -->|macOS| C[Install Docker Desktop for Mac]
C --> D[Install Ollama for macOS]
B -->|Windows| E[Enable / Update WSL 2]
E --> F[Install Docker Desktop for Windows]
F --> G[Install Ollama for Windows]
D --> H[Verify Docker + Ollama]
G --> H
H --> I[Pull Model]
I --> J[Deploy Open WebUI]
Current Ollama documentation requires:
macOS Sonoma 14 or newer- Apple M-series Macs support CPU and GPU acceleration.
- Intel Macs can run Ollama using CPU execution.
Check your version:
sw_vers
Check your hardware:
system_profiler SPHardwareDataType | grep -E "Chip|Processor Name|Memory"
Example:
Chip: Apple M3 Pro
Memory: 36 GB
Homebrew is useful for command-line tools such as Git, wget
, jq
, and other utilities.
Check whether it is installed:
brew --version
If it is not installed:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Follow the PATH instructions printed by the Homebrew installer.
Verify:
brew --version
brew install git jq wget
Verify:
git --version
jq --version
wget --version
Docker Desktop supports current and recent macOS releases and requires at least 4 GB of RAM, although running local LLMs generally requires considerably more memory.
-
Download Docker Desktop for Mac from: - Select the correct build:
-
Apple Silicon
-
Intel
-
Open
Docker.dmg
. - Drag Docker into
/Applications
. - Launch Docker Desktop.
- Accept the Docker Desktop agreement.
- Use the recommended settings unless you have a reason to customize them.
If Homebrew is available:
brew install --cask docker
Launch it:
open -a Docker
Wait until Docker Desktop reports that the engine is running.
Verify:
docker --version
docker compose version
docker info
On Apple Silicon, most of this stack runs natively. Some AMD64-only tools may still require Rosetta.
Install it if needed:
softwareupdate --install-rosetta --agree-to-license
You generally do not need Rosetta just to run Ollama and Open WebUI.
Current Ollama documentation requires:
Windows 10 22H2 or newer, Home or Pro- NVIDIA users should keep GPU drivers current.
- AMD acceleration depends on supported ROCm/HIP or Vulkan-capable drivers.
Check Windows version:
winver
Or:
Get-ComputerInfo | Select-Object WindowsProductName, WindowsVersion, OsBuildNumber
Windows Terminal provides a better PowerShell and command-line experience.
With winget
:
winget install --id Microsoft.WindowsTerminal -e
winget install --id Git.Git -e
Verify in a new terminal:
git --version
Docker Desktop normally uses the WSL 2 backend for Linux containers.
Open PowerShell as Administrator:
wsl --install
Restart Windows if requested.
After reboot:
wsl --update
wsl --status
Check installed distributions:
wsl --list --verbose
You should see WSL version 2
for your Linux distribution.
If necessary:
wsl --set-default-version 2
- Download Docker Desktop from:
- Run
Docker Desktop Installer.exe
. - For most users, select the WSL 2 backend. - Complete installation.
- Start Docker Desktop.
- Accept the Docker Desktop agreement.
winget install --id Docker.DockerDesktop -e
Start Docker Desktop from the Start menu.
Verify in PowerShell:
docker --version
docker compose version
docker info
If Docker commands fail immediately after installation, close and reopen Windows Terminal or PowerShell.
For NVIDIA:
nvidia-smi
If the command is unavailable, install or update the NVIDIA driver before expecting GPU acceleration.
For AMD, use the current AMD driver package appropriate for your GPU.
Ollama itself runs natively on Windows. You do not need to place Ollama inside WSL for the setup in this guide.
Before continuing, all of these should work.
docker --version
docker compose version
ollama --version
docker --version
docker compose version
ollama --version
After Ollama is started, verify its API.
curl http://localhost:11434/api/version
Invoke-RestMethod http://localhost:11434/api/version
Expected conceptually:
{
"version": "..."
}
The preferred official macOS installation is the Ollama application.
Download it from:
https://ollama.com/download
Install Ollama.app
in:
/Applications
Launch it:
open -a Ollama
On first launch, Ollama can create the ollama
CLI link in:
/usr/local/bin/ollama
If you already used the shell installer:
curl -fsSL https://ollama.com/install.sh | sh
verify:
ollama --version
If the CLI exists but the service is not running, you may see:
Warning: could not connect to a running Ollama instance
Start the application:
open -a Ollama
or start the server manually:
ollama serve
Verify:
curl http://localhost:11434/api/version
The simplest supported method is the native Windows installer.
Download:
https://ollama.com/download
Run:
OllamaSetup.exe
Ollama installs into the current user's profile by default and makes the ollama
command available to PowerShell, Command Prompt, and Windows Terminal.
After installation, open a new PowerShell window:
ollama --version
Ollama normally runs in the background.
Verify the API:
Invoke-RestMethod http://localhost:11434/api/version
If Ollama is not running, launch it from the Start menu.
You can also start the server explicitly:
ollama serve
By default, downloaded models are stored under your user profile.
If you want models on a larger disk, create the user environment variable:
OLLAMA_MODELS
For example:
D:\AI\Models\Ollama
After changing the variable, completely exit Ollama and relaunch it.
ollama --version
ollama list
curl http://localhost:11434/api/tags
ollama --version
ollama list
Invoke-RestMethod http://localhost:11434/api/tags
Check running processes:
ps aux | grep -i ollama
Check the local port:
lsof -i :11434
Check the API:
curl http://localhost:11434/api/tags
A successful response looks roughly like:
{
"models": []
}
An empty model list is fine if you have not downloaded a model yet.
For technical support, troubleshooting, cloud, Kubernetes, shell commands, code analysis, and customer-facing explanations, a strong starting point is Qwen3.
Suggested sizing:
| Mac Unified Memory | Suggested Starting Model |
|---|---|
| 8 GB | Qwen3 1.7B–4B |
| 16 GB | Qwen3 4B–8B |
| 24 GB | Qwen3 8B–14B |
| 32 GB | Qwen3 14B or larger quantized models |
| 64 GB+ | Larger 30B-class quantized models |
For most MacBooks with 16 GB or more:
ollama pull qwen3:8b
For a lower-memory machine:
ollama pull qwen3:4b
For a more capable machine:
ollama pull qwen3:14b
List installed models:
ollama list
Test the model:
ollama run qwen3:8b
Try:
You are a senior cloud support engineer.
Analyze the following error. Separate confirmed facts from assumptions,
identify likely root causes, provide validation steps, and explain what
information should be requested from the customer.
Error:
connection refused to localhost:443
Exit Ollama chat with:
/bye
Create a working directory:
mkdir -p ~/open-webui
cd ~/open-webui
The final layout will look like:
~/open-webui/
├── compose.yaml
├── .env
└── .gitignore
Generate a secret:
openssl rand -hex 32
Create .env
:
nano .env
Add:
WEBUI_SECRET_KEY=PASTE_YOUR_GENERATED_SECRET_HERE
Save and exit.
For nano
:
Ctrl+O
Enter
Ctrl+X
Protect the environment file from accidental Git commits:
printf ".env\n*.gguf\n" > .gitignore
Create the Compose file:
nano compose.yaml
Paste:
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
environment:
OLLAMA_BASE_URL: http://host.docker.internal:11434
WEBUI_AUTH: "true"
WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY}
volumes:
- open-webui-data:/app/backend/data
ports:
- "3000:8080"
volumes:
open-webui-data:
Save the file.
Ollama is running directly on macOS.
Open WebUI is running inside a Linux Docker container.
Inside the container:
localhost
means the Open WebUI container itself, not your Mac.
Therefore this will not work:
http://localhost:11434
The Docker-provided hostname for reaching the Mac host is:
host.docker.internal
Therefore Open WebUI should use:
http://host.docker.internal:11434
From:
cd ~/open-webui
start the container:
docker compose up -d
Check its status:
docker compose ps
Expected output will resemble:
NAME IMAGE STATUS
open-webui ghcr.io/open-webui/open-webui:main Up
Check logs:
docker compose logs -f open-webui
Press:
Ctrl+C
to stop following the logs. This does not stop the container.
Open:
http://localhost:3000
On macOS, you can also launch it from Terminal:
open http://localhost:3000
On Windows PowerShell:
Start-Process http://localhost:3000
On first launch:
- Create the initial account.
- The first account normally becomes the administrator.
- Open a new chat.
- Select your Ollama model from the model selector.
For example:
qwen3:8b
First verify Ollama from macOS:
curl http://localhost:11434/api/tags
Then verify it from inside the Open WebUI container:
docker exec open-webui \
curl http://host.docker.internal:11434/api/tags
If both commands work, connectivity is correct.
Check locally installed models:
ollama list
Example:
NAME ID SIZE
qwen3:8b abc123... 5.2 GB
Restart Open WebUI:
cd ~/open-webui
docker compose restart open-webui
Then reload:
http://localhost:3000
If necessary, inspect:
docker compose logs --tail=100 open-webui
Verify the environment variable:
docker exec open-webui env | grep OLLAMA
Expected:
OLLAMA_BASE_URL=http://host.docker.internal:11434
You do not need Hugging Face to use Ollama's normal model library.
For example:
ollama pull qwen3:8b
is the simplest approach.
However, you may want to download a specific quantized model from Hugging Face.
For Ollama on a MacBook, prefer GGUF models.
Example filename:
Qwen3-8B-Q4_K_M.gguf
Quantization names frequently include:
Q4_K_M
Q5_K_M
Q6_K
Q8_0
For MacBooks, Q4_K_M
is often a useful balance between memory consumption and model quality.
Create a model directory:
mkdir -p ~/Models/my-support-model
cd ~/Models/my-support-model
Place your downloaded GGUF file there.
Example:
~/Models/my-support-model/
└── qwen3-8b-q4_k_m.gguf
Create a Modelfile
:
nano Modelfile
Example:
FROM ./qwen3-8b-q4_k_m.gguf
PARAMETER temperature 0.3
PARAMETER num_ctx 8192
SYSTEM """
You are a senior technical support engineer specializing in:
- AWS
- Azure
- GCP
- Kubernetes
- Docker
- Terraform
- CI/CD
- Linux
- macOS
- networking
- identity and access management
- security tooling
- application troubleshooting
When investigating technical issues:
1. Separate confirmed evidence from assumptions.
2. Do not invent undocumented product behavior.
3. Identify the most likely root cause.
4. Offer alternative hypotheses when appropriate.
5. Give validation commands before recommending disruptive changes.
6. Explain what logs, configuration, or screenshots are needed.
7. Clearly distinguish workaround from permanent fix.
8. Write customer-facing responses professionally and concisely.
9. Avoid claiming engineering confirmation unless evidence exists.
10. Call out uncertainty explicitly.
"""
Create the Ollama model:
ollama create support-engineer -f Modelfile
Verify:
ollama list
Run:
ollama run support-engineer
The model should also become available inside Open WebUI.
For a general-purpose support workflow, start with:
ollama pull qwen3:8b
Use it for:
- analyzing logs
- Kubernetes troubleshooting
- Terraform errors
- Docker problems
- shell commands
- API errors
- cloud configuration
- IAM analysis
- CI/CD debugging
- customer response drafting
- case summaries
- troubleshooting plans
If memory permits, compare it against:
ollama pull qwen3:14b
The larger model may provide better reasoning but will consume more unified memory and run more slowly.
You can save the following as a system prompt in Open WebUI:
You are a senior technical support engineer.
Your job is to help investigate technical support cases accurately and
conservatively.
For every investigation:
1. Summarize the reported issue.
2. Separate confirmed evidence from assumptions.
3. Identify the most likely root cause.
4. List alternative hypotheses when evidence is incomplete.
5. Provide safe validation commands.
6. Avoid destructive commands unless clearly labeled.
7. Distinguish:
- expected behavior
- configuration issue
- product limitation
- suspected defect
- confirmed defect
8. Clearly distinguish workaround from permanent resolution.
9. Never invent internal tickets, engineering findings, documentation,
product behavior, or feature commitments.
10. State confidence levels where appropriate.
11. When asked for a customer response, write concise, professional,
technically accurate language without unnecessary internal details.
List models:
ollama list
Pull a model:
ollama pull qwen3:8b
Run a model:
ollama run qwen3:8b
Show model details:
ollama show qwen3:8b
Remove a model:
ollama rm qwen3:8b
Show running models:
ollama ps
Start the Ollama API server:
ollama serve
Start Open WebUI:
docker compose up -d
Stop Open WebUI:
docker compose down
Restart:
docker compose restart
Check status:
docker compose ps
Follow logs:
docker compose logs -f open-webui
Show recent logs:
docker compose logs --tail=100 open-webui
Pull the newest Open WebUI image:
docker compose pull
Recreate the container after updating:
docker compose up -d
From the project directory:
cd ~/open-webui
docker compose pull
docker compose up -d
Check:
docker compose ps
Your conversations and configuration are retained in the Docker volume:
open-webui-data
If installed through the macOS application, update Ollama through its normal application update path.
Verify afterward:
ollama --version
Confirm the server:
curl http://localhost:11434/api/version
Open WebUI stores its persistent data in a Docker volume associated with your Compose project.
See the actual volume:
docker volume ls | grep open-webui
Create a backup directory:
mkdir -p ~/open-webui-backups
Then use the actual volume name returned above. For example:
docker run --rm \
-v open-webui_open-webui-data:/data \
-v ~/open-webui-backups:/backup \
alpine \
tar czf /backup/open-webui-data.tar.gz -C /data .
The exact Docker volume name may differ depending on the Compose project name. Run
docker volume ls
first and use the actual value.
To stop and remove the container while retaining data:
docker compose down
To delete the Open WebUI persistent volume too:
docker compose down -v
Warning:docker compose down -v
deletes Open WebUI's stored application data for that Compose project, including local configuration and potentially chat history.
Check:
which ollama
Try:
ls -l /usr/local/bin/ollama
If necessary:
export PATH="/usr/local/bin:$PATH"
Add permanently:
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc
Start Ollama:
open -a Ollama
or:
ollama serve
Verify:
curl http://localhost:11434/api/version
Check:
docker compose ps
Then:
docker compose logs --tail=100 open-webui
Confirm Docker Desktop is running.
Check whether port 3000
is already in use:
lsof -i :3000
If another service is using port 3000, change:
ports:
- "3001:8080"
Then access:
http://localhost:3001
Verify Ollama:
curl http://localhost:11434/api/tags
Verify from the Docker container:
docker exec open-webui \
curl http://host.docker.internal:11434/api/tags
Confirm compose.yaml
contains:
environment:
OLLAMA_BASE_URL: http://host.docker.internal:11434
Restart:
docker compose down
docker compose up -d
Check memory:
system_profiler SPHardwareDataType | grep Memory
Check running models:
ollama ps
Try a smaller model:
ollama pull qwen3:4b
Avoid having multiple memory-heavy applications open at the same time.
Check Activity Monitor or:
vm_stat
A model that technically loads may still perform poorly if macOS is heavily swapping.
Use a smaller quantization or smaller model.
When importing your own model:
PARAMETER num_ctx 16384
Be aware that larger context windows increase memory consumption.
Start with:
8192
and increase only when necessary.
For a machine that is strictly local and never exposed to other systems, you could configure:
WEBUI_AUTH: "false"
However, leaving authentication enabled is generally safer:
WEBUI_AUTH: "true"
Do not disable authentication if Open WebUI is exposed to your LAN, a VPN, a tunnel, or the Internet.
The normal local endpoint is:
http://localhost:11434
There is normally no need to publish Ollama publicly.
Your desired design is:
Local browser
|
v
Open WebUI
|
v
Ollama
If remote access is required later, put authentication, TLS, and appropriate access controls in front of Open WebUI rather than exposing the Ollama API directly.
For reference, the basic workflow is:
curl -fsSL https://ollama.com/install.sh | sh
open -a Ollama
curl http://localhost:11434/api/version
ollama pull qwen3:8b
mkdir -p ~/open-webui
cd ~/open-webui
openssl rand -hex 32
Create .env
:
WEBUI_SECRET_KEY=YOUR_SECRET
Create compose.yaml
:
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
environment:
OLLAMA_BASE_URL: http://host.docker.internal:11434
WEBUI_AUTH: "true"
WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY}
volumes:
- open-webui-data:/app/backend/data
ports:
- "3000:8080"
volumes:
open-webui-data:
Start:
docker compose up -d
Open:
open http://localhost:3000
php
flowchart TD
A[Install host prerequisites] --> B[Start Docker Desktop]
B --> C[Install and start Ollama]
C --> D[Verify localhost:11434]
D --> E[Pull or import a model]
E --> F[Test model with Ollama CLI]
F --> G[Create Open WebUI compose.yaml]
G --> H[docker compose up -d]
H --> I[Open localhost:3000]
I --> J{Model visible?}
J -->|Yes| K[Start using Open WebUI]
J -->|No| L[Test host.docker.internal:11434]
L --> M[Check OLLAMA_BASE_URL]
M --> N[Restart Open WebUI]
N --> J
php
flowchart TD
A[Open WebUI problem] --> B{Does localhost:3000 load?}
B -->|No| C[docker compose ps]
C --> D{Container running?}
D -->|No| E[docker compose logs open-webui]
D -->|Yes| F[Check port 3000 conflict]
B -->|Yes| G{Is model listed?}
G -->|No| H[Run ollama list]
H --> I{Model installed?}
I -->|No| J[ollama pull model]
I -->|Yes| K[Test Ollama API]
K --> L{Host API works?}
L -->|No| M[Start or restart Ollama]
L -->|Yes| N[Test from Open WebUI container]
N --> O{Container reaches Ollama?}
O -->|No| P[Check host.docker.internal]
P --> Q[Check OLLAMA_BASE_URL]
Q --> R[Restart container]
O -->|Yes| S[Refresh Open WebUI connection/models]
php
flowchart TD
A[Choose local model] --> B{Available RAM / Unified Memory}
B -->|8 GB| C[1B to 4B quantized model]
B -->|16 GB| D[4B to 8B quantized model]
B -->|24 GB| E[8B to 14B quantized model]
B -->|32 GB| F[14B or selected 30B quantized]
B -->|64 GB+| G[Larger 30B-class models]
C --> H[Test response quality]
D --> H
E --> H
F --> H
G --> H
H --> I{Fast enough?}
I -->|No| J[Use smaller model or quantization]
I -->|Yes| K{Quality sufficient?}
K -->|No| L[Try larger / specialized model]
K -->|Yes| M[Use in Open WebUI]
Check each item:
- Docker Desktop is running.
ollama --version
works. -
curl http://localhost:11434/api/version
works. -
ollama list
shows at least one model. -
ollama run qwen3:8b
works. -
docker compose ps
showsopen-webui
as running. -
http://localhost:3000
loads. - Open WebUI displays the Ollama model.
-
A test prompt returns a response.
-
Open WebUI persists after container restart.
-
Ollama macOS: https://docs.ollama.com/macos - Ollama Windows: https://docs.ollama.com/windows - Ollama downloads: https://ollama.com/download - Docker Desktop for Mac: https://docs.docker.com/desktop/setup/install/mac-install/ - Docker Desktop for Windows: https://docs.docker.com/desktop/setup/install/windows-install/ - Open WebUI Quick Start: https://docs.openwebui.com/getting-started/quick-start/ - Open WebUI + Ollama: https://docs.openwebui.com/getting-started/quick-start/connect-a-provider/starting-with-ollama/
For a macOS or Windows workstation used for technical support work:
Open WebUI
|
+-- qwen3:8b General troubleshooting/support
|
+-- Larger Qwen3 Optional, if enough unified memory
|
+-- Custom GGUF Optional Hugging Face model
Use native Ollama for model execution and Docker Compose for Open WebUI.
That combination is simple, fast on Apple Silicon, easy to maintain, and does not require a Linux VM or NVIDIA GPU.