Open WebUI + Ollama + Hugging Face on macOS and Windows

A developer has published a practical guide for setting up a local AI troubleshooting and support environment on macOS and Windows using Open WebUI, Ollama, and Hugging Face. The recommended architecture runs Ollama natively on the host OS for hardware acceleration (Apple Metal, CUDA, or ROCm) while Open WebUI runs in Docker, communicating via host.docker.internal. The guide covers installation steps, prerequisites, and verification commands for both platforms.

A practical setup guide for running a local AI troubleshooting and support environment on either macOS or Windows using: Open WebUI for the browser-based chat interface Ollama as the local model runtime Hugging Face as an optional source for GGUF models Docker Compose for running Open WebUI Apple Silicon / Metal acceleration through native Ollama Recommended architecture on macOS: run Ollama natively on the Macand runOpen WebUI in Docker. This allows Ollama to use Apple Metal acceleration while Open WebUI stays containerized. The recommended design on both macOS and Windows is to run Ollama natively on the host operating system and run Open WebUI in Docker Desktop . php flowchart TD A Browser -- |http://localhost:3000| B Open WebUI B -- |Docker host bridge| C Ollama API C -- D Local LLM D -- E{Host Hardware} E -- |macOS| F Apple Metal / Unified Memory E -- |Windows NVIDIA| G CUDA E -- |Windows AMD| H ROCm / Vulkan where supported E -- |CPU fallback| I CPU The network path is: sequenceDiagram participant U as User Browser participant W as Open WebUI Container participant O as Ollama on Host participant M as Local Model U- W: Prompt via http://localhost:3000 W- O: API request to host.docker.internal:11434 O- M: Run inference M-- O: Generated tokens O-- W: Stream response W-- U: Display response host.docker.internal is the important hostname here. It allows the Open WebUI container to reach Ollama running on the Windows or macOS host. You need: - Docker Desktop - Docker Compose v2 - Ollama - Sufficient RAM / unified memory - Free disk space for model files - A supported version of macOS or Windows The installation path depends on your operating system. php flowchart LR A Start -- B{Operating System} B -- |macOS| C Install Docker Desktop for Mac C -- D Install Ollama for macOS B -- |Windows| E Enable / Update WSL 2 E -- F Install Docker Desktop for Windows F -- G Install Ollama for Windows D -- H Verify Docker + Ollama G -- H H -- I Pull Model I -- J Deploy Open WebUI Current Ollama documentation requires: macOS Sonoma 14 or newer - Apple M-series Macs support CPU and GPU acceleration. - Intel Macs can run Ollama using CPU execution. Check your version: sw vers Check your hardware: system profiler SPHardwareDataType | grep -E "Chip|Processor Name|Memory" Example: Chip: Apple M3 Pro Memory: 36 GB Homebrew is useful for command-line tools such as Git, wget , jq , and other utilities. Check whether it is installed: brew --version If it is not installed: /bin/bash -c "$ curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh " Follow the PATH instructions printed by the Homebrew installer. Verify: brew --version brew install git jq wget Verify: git --version jq --version wget --version Docker Desktop supports current and recent macOS releases and requires at least 4 GB of RAM, although running local LLMs generally requires considerably more memory. - Download Docker Desktop for Mac from: - Select the correct build: - Apple Silicon - Intel - Open Docker.dmg . - Drag Docker into /Applications . - Launch Docker Desktop. - Accept the Docker Desktop agreement. - Use the recommended settings unless you have a reason to customize them. If Homebrew is available: brew install --cask docker Launch it: open -a Docker Wait until Docker Desktop reports that the engine is running. Verify: docker --version docker compose version docker info On Apple Silicon, most of this stack runs natively. Some AMD64-only tools may still require Rosetta. Install it if needed: softwareupdate --install-rosetta --agree-to-license You generally do not need Rosetta just to run Ollama and Open WebUI. Current Ollama documentation requires: Windows 10 22H2 or newer , Home or Pro- NVIDIA users should keep GPU drivers current. - AMD acceleration depends on supported ROCm/HIP or Vulkan-capable drivers. Check Windows version: winver Or: Get-ComputerInfo | Select-Object WindowsProductName, WindowsVersion, OsBuildNumber Windows Terminal provides a better PowerShell and command-line experience. With winget : winget install --id Microsoft.WindowsTerminal -e winget install --id Git.Git -e Verify in a new terminal: git --version Docker Desktop normally uses the WSL 2 backend for Linux containers. Open PowerShell as Administrator : wsl --install Restart Windows if requested. After reboot: wsl --update wsl --status Check installed distributions: wsl --list --verbose You should see WSL version 2 for your Linux distribution. If necessary: wsl --set-default-version 2 - Download Docker Desktop from: - Run Docker Desktop Installer.exe . - For most users, select the WSL 2 backend . - Complete installation. - Start Docker Desktop. - Accept the Docker Desktop agreement. winget install --id Docker.DockerDesktop -e Start Docker Desktop from the Start menu. Verify in PowerShell: docker --version docker compose version docker info If Docker commands fail immediately after installation, close and reopen Windows Terminal or PowerShell. For NVIDIA: nvidia-smi If the command is unavailable, install or update the NVIDIA driver before expecting GPU acceleration. For AMD, use the current AMD driver package appropriate for your GPU. Ollama itself runs natively on Windows. You do not need to place Ollama inside WSL for the setup in this guide. Before continuing, all of these should work. docker --version docker compose version ollama --version docker --version docker compose version ollama --version After Ollama is started, verify its API. curl http://localhost:11434/api/version Invoke-RestMethod http://localhost:11434/api/version Expected conceptually: { "version": "..." } The preferred official macOS installation is the Ollama application. Download it from: https://ollama.com/download Install Ollama.app in: /Applications Launch it: open -a Ollama On first launch, Ollama can create the ollama CLI link in: /usr/local/bin/ollama If you already used the shell installer: curl -fsSL https://ollama.com/install.sh | sh verify: ollama --version If the CLI exists but the service is not running, you may see: Warning: could not connect to a running Ollama instance Start the application: open -a Ollama or start the server manually: ollama serve Verify: curl http://localhost:11434/api/version The simplest supported method is the native Windows installer. Download: https://ollama.com/download Run: OllamaSetup.exe Ollama installs into the current user's profile by default and makes the ollama command available to PowerShell, Command Prompt, and Windows Terminal. After installation, open a new PowerShell window : ollama --version Ollama normally runs in the background. Verify the API: Invoke-RestMethod http://localhost:11434/api/version If Ollama is not running, launch it from the Start menu. You can also start the server explicitly: ollama serve By default, downloaded models are stored under your user profile. If you want models on a larger disk, create the user environment variable: OLLAMA MODELS For example: D:\AI\Models\Ollama After changing the variable, completely exit Ollama and relaunch it. ollama --version ollama list curl http://localhost:11434/api/tags ollama --version ollama list Invoke-RestMethod http://localhost:11434/api/tags Check running processes: ps aux | grep -i ollama Check the local port: lsof -i :11434 Check the API: curl http://localhost:11434/api/tags A successful response looks roughly like: { "models": } An empty model list is fine if you have not downloaded a model yet. For technical support, troubleshooting, cloud, Kubernetes, shell commands, code analysis, and customer-facing explanations, a strong starting point is Qwen3 . Suggested sizing: | Mac Unified Memory | Suggested Starting Model | |---|---| | 8 GB | Qwen3 1.7B–4B | | 16 GB | Qwen3 4B–8B | | 24 GB | Qwen3 8B–14B | | 32 GB | Qwen3 14B or larger quantized models | | 64 GB+ | Larger 30B-class quantized models | For most MacBooks with 16 GB or more: ollama pull qwen3:8b For a lower-memory machine: ollama pull qwen3:4b For a more capable machine: ollama pull qwen3:14b List installed models: ollama list Test the model: ollama run qwen3:8b Try: You are a senior cloud support engineer. Analyze the following error. Separate confirmed facts from assumptions, identify likely root causes, provide validation steps, and explain what information should be requested from the customer. Error: connection refused to localhost:443 Exit Ollama chat with: /bye Create a working directory: mkdir -p ~/open-webui cd ~/open-webui The final layout will look like: ~/open-webui/ ├── compose.yaml ├── .env └── .gitignore Generate a secret: openssl rand -hex 32 Create .env : nano .env Add: WEBUI SECRET KEY=PASTE YOUR GENERATED SECRET HERE Save and exit. For nano : Ctrl+O Enter Ctrl+X Protect the environment file from accidental Git commits: printf ".env\n .gguf\n" .gitignore Create the Compose file: nano compose.yaml Paste: services: open-webui: image: ghcr.io/open-webui/open-webui:main container name: open-webui restart: unless-stopped environment: OLLAMA BASE URL: http://host.docker.internal:11434 WEBUI AUTH: "true" WEBUI SECRET KEY: ${WEBUI SECRET KEY} volumes: - open-webui-data:/app/backend/data ports: - "3000:8080" volumes: open-webui-data: Save the file. Ollama is running directly on macOS. Open WebUI is running inside a Linux Docker container. Inside the container: localhost means the Open WebUI container itself , not your Mac. Therefore this will not work: http://localhost:11434 The Docker-provided hostname for reaching the Mac host is: host.docker.internal Therefore Open WebUI should use: http://host.docker.internal:11434 From: cd ~/open-webui start the container: docker compose up -d Check its status: docker compose ps Expected output will resemble: NAME IMAGE STATUS open-webui ghcr.io/open-webui/open-webui:main Up Check logs: docker compose logs -f open-webui Press: Ctrl+C to stop following the logs. This does not stop the container. Open: http://localhost:3000 On macOS, you can also launch it from Terminal: open http://localhost:3000 On Windows PowerShell: Start-Process http://localhost:3000 On first launch: - Create the initial account. - The first account normally becomes the administrator. - Open a new chat. - Select your Ollama model from the model selector. For example: qwen3:8b First verify Ollama from macOS: curl http://localhost:11434/api/tags Then verify it from inside the Open WebUI container: docker exec open-webui \ curl http://host.docker.internal:11434/api/tags If both commands work, connectivity is correct. Check locally installed models: ollama list Example: NAME ID SIZE qwen3:8b abc123... 5.2 GB Restart Open WebUI: cd ~/open-webui docker compose restart open-webui Then reload: http://localhost:3000 If necessary, inspect: docker compose logs --tail=100 open-webui Verify the environment variable: docker exec open-webui env | grep OLLAMA Expected: OLLAMA BASE URL=http://host.docker.internal:11434 You do not need Hugging Face to use Ollama's normal model library. For example: ollama pull qwen3:8b is the simplest approach. However, you may want to download a specific quantized model from Hugging Face. For Ollama on a MacBook, prefer GGUF models. Example filename: Qwen3-8B-Q4 K M.gguf Quantization names frequently include: Q4 K M Q5 K M Q6 K Q8 0 For MacBooks, Q4 K M is often a useful balance between memory consumption and model quality. Create a model directory: mkdir -p ~/Models/my-support-model cd ~/Models/my-support-model Place your downloaded GGUF file there. Example: ~/Models/my-support-model/ └── qwen3-8b-q4 k m.gguf Create a Modelfile : nano Modelfile Example: FROM ./qwen3-8b-q4 k m.gguf PARAMETER temperature 0.3 PARAMETER num ctx 8192 SYSTEM """ You are a senior technical support engineer specializing in: - AWS - Azure - GCP - Kubernetes - Docker - Terraform - CI/CD - Linux - macOS - networking - identity and access management - security tooling - application troubleshooting When investigating technical issues: 1. Separate confirmed evidence from assumptions. 2. Do not invent undocumented product behavior. 3. Identify the most likely root cause. 4. Offer alternative hypotheses when appropriate. 5. Give validation commands before recommending disruptive changes. 6. Explain what logs, configuration, or screenshots are needed. 7. Clearly distinguish workaround from permanent fix. 8. Write customer-facing responses professionally and concisely. 9. Avoid claiming engineering confirmation unless evidence exists. 10. Call out uncertainty explicitly. """ Create the Ollama model: ollama create support-engineer -f Modelfile Verify: ollama list Run: ollama run support-engineer The model should also become available inside Open WebUI. For a general-purpose support workflow, start with: ollama pull qwen3:8b Use it for: - analyzing logs - Kubernetes troubleshooting - Terraform errors - Docker problems - shell commands - API errors - cloud configuration - IAM analysis - CI/CD debugging - customer response drafting - case summaries - troubleshooting plans If memory permits, compare it against: ollama pull qwen3:14b The larger model may provide better reasoning but will consume more unified memory and run more slowly. You can save the following as a system prompt in Open WebUI: You are a senior technical support engineer. Your job is to help investigate technical support cases accurately and conservatively. For every investigation: 1. Summarize the reported issue. 2. Separate confirmed evidence from assumptions. 3. Identify the most likely root cause. 4. List alternative hypotheses when evidence is incomplete. 5. Provide safe validation commands. 6. Avoid destructive commands unless clearly labeled. 7. Distinguish: - expected behavior - configuration issue - product limitation - suspected defect - confirmed defect 8. Clearly distinguish workaround from permanent resolution. 9. Never invent internal tickets, engineering findings, documentation, product behavior, or feature commitments. 10. State confidence levels where appropriate. 11. When asked for a customer response, write concise, professional, technically accurate language without unnecessary internal details. List models: ollama list Pull a model: ollama pull qwen3:8b Run a model: ollama run qwen3:8b Show model details: ollama show qwen3:8b Remove a model: ollama rm qwen3:8b Show running models: ollama ps Start the Ollama API server: ollama serve Start Open WebUI: docker compose up -d Stop Open WebUI: docker compose down Restart: docker compose restart Check status: docker compose ps Follow logs: docker compose logs -f open-webui Show recent logs: docker compose logs --tail=100 open-webui Pull the newest Open WebUI image: docker compose pull Recreate the container after updating: docker compose up -d From the project directory: cd ~/open-webui docker compose pull docker compose up -d Check: docker compose ps Your conversations and configuration are retained in the Docker volume: open-webui-data If installed through the macOS application, update Ollama through its normal application update path. Verify afterward: ollama --version Confirm the server: curl http://localhost:11434/api/version Open WebUI stores its persistent data in a Docker volume associated with your Compose project. See the actual volume: docker volume ls | grep open-webui Create a backup directory: mkdir -p ~/open-webui-backups Then use the actual volume name returned above. For example: docker run --rm \ -v open-webui open-webui-data:/data \ -v ~/open-webui-backups:/backup \ alpine \ tar czf /backup/open-webui-data.tar.gz -C /data . The exact Docker volume name may differ depending on the Compose project name. Run docker volume ls first and use the actual value. To stop and remove the container while retaining data: docker compose down To delete the Open WebUI persistent volume too: docker compose down -v Warning: docker compose down -v deletes Open WebUI's stored application data for that Compose project, including local configuration and potentially chat history. Check: which ollama Try: ls -l /usr/local/bin/ollama If necessary: export PATH="/usr/local/bin:$PATH" Add permanently: echo 'export PATH="/usr/local/bin:$PATH"' ~/.zshrc source ~/.zshrc Start Ollama: open -a Ollama or: ollama serve Verify: curl http://localhost:11434/api/version Check: docker compose ps Then: docker compose logs --tail=100 open-webui Confirm Docker Desktop is running. Check whether port 3000 is already in use: lsof -i :3000 If another service is using port 3000, change: ports: - "3001:8080" Then access: http://localhost:3001 Verify Ollama: curl http://localhost:11434/api/tags Verify from the Docker container: docker exec open-webui \ curl http://host.docker.internal:11434/api/tags Confirm compose.yaml contains: environment: OLLAMA BASE URL: http://host.docker.internal:11434 Restart: docker compose down docker compose up -d Check memory: system profiler SPHardwareDataType | grep Memory Check running models: ollama ps Try a smaller model: ollama pull qwen3:4b Avoid having multiple memory-heavy applications open at the same time. Check Activity Monitor or: vm stat A model that technically loads may still perform poorly if macOS is heavily swapping. Use a smaller quantization or smaller model. When importing your own model: PARAMETER num ctx 16384 Be aware that larger context windows increase memory consumption. Start with: 8192 and increase only when necessary. For a machine that is strictly local and never exposed to other systems, you could configure: WEBUI AUTH: "false" However, leaving authentication enabled is generally safer: WEBUI AUTH: "true" Do not disable authentication if Open WebUI is exposed to your LAN, a VPN, a tunnel, or the Internet. The normal local endpoint is: http://localhost:11434 There is normally no need to publish Ollama publicly. Your desired design is: Local browser | v Open WebUI | v Ollama If remote access is required later, put authentication, TLS, and appropriate access controls in front of Open WebUI rather than exposing the Ollama API directly. For reference, the basic workflow is: Install Ollama curl -fsSL https://ollama.com/install.sh | sh Start Ollama open -a Ollama Verify curl http://localhost:11434/api/version Download model ollama pull qwen3:8b Create Open WebUI project mkdir -p ~/open-webui cd ~/open-webui Generate secret openssl rand -hex 32 Create .env : WEBUI SECRET KEY=YOUR SECRET Create compose.yaml : services: open-webui: image: ghcr.io/open-webui/open-webui:main container name: open-webui restart: unless-stopped environment: OLLAMA BASE URL: http://host.docker.internal:11434 WEBUI AUTH: "true" WEBUI SECRET KEY: ${WEBUI SECRET KEY} volumes: - open-webui-data:/app/backend/data ports: - "3000:8080" volumes: open-webui-data: Start: docker compose up -d Open: open http://localhost:3000 php flowchart TD A Install host prerequisites -- B Start Docker Desktop B -- C Install and start Ollama C -- D Verify localhost:11434 D -- E Pull or import a model E -- F Test model with Ollama CLI F -- G Create Open WebUI compose.yaml G -- H docker compose up -d H -- I Open localhost:3000 I -- J{Model visible?} J -- |Yes| K Start using Open WebUI J -- |No| L Test host.docker.internal:11434 L -- M Check OLLAMA BASE URL M -- N Restart Open WebUI N -- J php flowchart TD A Open WebUI problem -- B{Does localhost:3000 load?} B -- |No| C docker compose ps C -- D{Container running?} D -- |No| E docker compose logs open-webui D -- |Yes| F Check port 3000 conflict B -- |Yes| G{Is model listed?} G -- |No| H Run ollama list H -- I{Model installed?} I -- |No| J ollama pull model I -- |Yes| K Test Ollama API K -- L{Host API works?} L -- |No| M Start or restart Ollama L -- |Yes| N Test from Open WebUI container N -- O{Container reaches Ollama?} O -- |No| P Check host.docker.internal P -- Q Check OLLAMA BASE URL Q -- R Restart container O -- |Yes| S Refresh Open WebUI connection/models php flowchart TD A Choose local model -- B{Available RAM / Unified Memory} B -- |8 GB| C 1B to 4B quantized model B -- |16 GB| D 4B to 8B quantized model B -- |24 GB| E 8B to 14B quantized model B -- |32 GB| F 14B or selected 30B quantized B -- |64 GB+| G Larger 30B-class models C -- H Test response quality D -- H E -- H F -- H G -- H H -- I{Fast enough?} I -- |No| J Use smaller model or quantization I -- |Yes| K{Quality sufficient?} K -- |No| L Try larger / specialized model K -- |Yes| M Use in Open WebUI Check each item: - Docker Desktop is running. - ollama --version works. - curl http://localhost:11434/api/version works. - ollama list shows at least one model. - ollama run qwen3:8b works. - docker compose ps shows open-webui as running. - http://localhost:3000 loads. - Open WebUI displays the Ollama model. - A test prompt returns a response. - Open WebUI persists after container restart. - Ollama macOS: https://docs.ollama.com/macos https://docs.ollama.com/macos - Ollama Windows: https://docs.ollama.com/windows https://docs.ollama.com/windows - Ollama downloads: https://ollama.com/download https://ollama.com/download - Docker Desktop for Mac: https://docs.docker.com/desktop/setup/install/mac-install/ https://docs.docker.com/desktop/setup/install/mac-install/ - Docker Desktop for Windows: https://docs.docker.com/desktop/setup/install/windows-install/ https://docs.docker.com/desktop/setup/install/windows-install/ - Open WebUI Quick Start: https://docs.openwebui.com/getting-started/quick-start/ https://docs.openwebui.com/getting-started/quick-start/ - Open WebUI + Ollama: https://docs.openwebui.com/getting-started/quick-start/connect-a-provider/starting-with-ollama/ https://docs.openwebui.com/getting-started/quick-start/connect-a-provider/starting-with-ollama/ For a macOS or Windows workstation used for technical support work: Open WebUI | +-- qwen3:8b General troubleshooting/support | +-- Larger Qwen3 Optional, if enough unified memory | +-- Custom GGUF Optional Hugging Face model Use native Ollama for model execution and Docker Compose for Open WebUI. That combination is simple, fast on Apple Silicon, easy to maintain, and does not require a Linux VM or NVIDIA GPU.