{"slug": "llm-server-mit-open-webui-mit-ollama", "title": "LLM-Server mit Open WebUI mit Ollama", "summary": "A developer provides a step-by-step guide to set up an LLM server with Open WebUI and Ollama on Ubuntu 24.04, including Docker and NVIDIA container toolkit installation. The guide offers two deployment options: one with a Traefik ingress router for HTTPS and a simpler variant without.", "body_md": "Für Ubuntu 24.04:\n\n```\nsudo apt update\nsudo apt upgrade\nsudo apt install nvidia-utils-580 nvidia-driver-580\n```\n\nDocker installieren und aktuellen Nutzer berechtigen:\n\n```\ncurl -fsSL https://get.docker.com | sudo sh\nsudo usermod -aG docker $USER\n```\n\nNvidia-Repos einrichten\n\n```\ncurl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \\\n  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \\\n    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \\\n    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list\n```\n\nNvidia-Toolkit für Container installieren:\n\n```\nsudo apt-get update\nsudo apt install -y nvidia-container-toolkit\n```\n\nToolkit in Docker aktivieren:\n\n```\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n```\n\nFür Ubuntu 24.04:\n\n␃WPNCODE0␃\n\nDocker installieren und aktuellen Nutzer berechtigen:\n\n␃WPNCODE1␃\n\nNvidia-Repos einrichten\n\n␃WPNCODE2␃\n\nNvidia-Toolkit für Container installieren:\n\n␃WPNCODE3␃\n\nToolkit in Docker aktivieren:\n\n␃WPNCODE4␃\n\n| # Variante mit Ingress-Router für HTTPS | |\n| services: | |\n| openwebui: | |\n| image: ghcr.io/open-webui/open-webui:main | |\n| labels: | |\n| - \"traefik.http.routers.webui.rule=Host(`ai.example.org`)\" | |\n| - \"traefik.http.routers.webui.tls.certResolver=default\" | |\n| - \"traefik.http.routers.webui.tls=true\" | |\n| - \"traefik.http.services.webui.loadbalancer.server.port=8080\" | |\n| volumes: | |\n| - ./data/open-webui:/app/backend/data | |\n| ollama: | |\n| image: ollama/ollama:latest | |\n| container_name: ollama | |\n| volumes: | |\n| - ./data/ollama:/root/.ollama | |\n| deploy: | |\n| resources: | |\n| reservations: | |\n| devices: | |\n| - driver: nvidia | |\n| count: all | |\n| capabilities: [gpu] | |\n| restart: unless-stopped | |\n| traefik: | |\n| image: traefik:v3 | |\n| command: --providers.docker | |\n| restart: always | |\n| ports: | |\n| - 80:80 | |\n| - 443:443 | |\n| volumes: | |\n| - /var/run/docker.sock:/var/run/docker.sock:rw | |\n| - ./data/traefik/static.yml:/etc/traefik/traefik.yml | |\n| - ./data/traefik/dynamic.yml:/etc/traefik/dynamic/dynamic.yml | |\n| - ./data/traefik/acme.json:/etc/traefik/acme/acme.json |\n\n| #Einfache Variante ohne Ingress-Router | |\n| services: | |\n| openwebui: | |\n| image: ghcr.io/open-webui/open-webui:main | |\n| ports: | |\n| - 8080:8080 | |\n| volumes: | |\n| - ./data/open-webui:/app/backend/data | |\n| ollama: | |\n| image: ollama/ollama:latest | |\n| container_name: ollama | |\n| volumes: | |\n| - ./data/ollama:/root/.ollama | |\n| deploy: | |\n| resources: | |\n| reservations: | |\n| devices: | |\n| - driver: nvidia |\n\n| http: | |\n| routers: | |\n| https-redirect: | |\n| rule: \"HostRegexp(`{any:.*}`)\" | |\n| middlewares: | |\n| - https-redirect | |\n| service: redirect-all | |\n| middlewares: | |\n| https-redirect: | |\n| redirectScheme: | |\n| scheme: https | |\n| services: | |\n| redirect-all: | |\n| loadBalancer: | |\n| servers: | |\n| - url: \"\" |\n\n| entryPoints: | |\n| web: | |\n| address: \":80\" | |\n| web-secure: | |\n| address: \":443\" | |\n| providers: | |\n| docker: | |\n| watch: true | |\n| file: | |\n| directory: /etc/traefik/dynamic | |\n| watch: true | |\n| filename: dynamic.yml | |\n| certificatesResolvers: | |\n| default: | |\n| acme: | |\n| email: ihremail@example.org | |\n| storage: /etc/traefik/acme/acme.json | |\n| httpChallenge: | |\n| entryPoint: web |", "url": "https://wpnews.pro/news/llm-server-mit-open-webui-mit-ollama", "canonical_source": "https://gist.github.com/jamct/bb9a7ab3a38249168b05c7a831275028", "published_at": "2026-06-30 11:39:52+00:00", "updated_at": "2026-07-28 16:31:05.777441+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models"], "entities": ["Open WebUI", "Ollama", "Docker", "NVIDIA", "Traefik", "Ubuntu"], "alternates": {"html": "https://wpnews.pro/news/llm-server-mit-open-webui-mit-ollama", "markdown": "https://wpnews.pro/news/llm-server-mit-open-webui-mit-ollama.md", "text": "https://wpnews.pro/news/llm-server-mit-open-webui-mit-ollama.txt", "jsonld": "https://wpnews.pro/news/llm-server-mit-open-webui-mit-ollama.jsonld"}}