5 Docker Containers You Need for a Home AI Lab

A home AI lab is best built from five separate Docker containers — Ollama for model serving on port 11434, Open WebUI for the browser interface, Qdrant for vector search, n8n for automation and Portainer CE for management — rather than one combined AI box, according to the guide. The guide warns that models stored inside the Ollama container are lost on recreate unless a named volume such as `ollama:/root/.ollama` is used, and that a regenerated `WEBUI_SECRET_KEY` logs all Open WebUI users out. It notes Docker's documented GPU pass-through routes are Linux and Windows with WSL2, so Docker Desktop on macOS cannot hand containers an NVIDIA GPU.

5 Docker Containers You Need for a Home AI Lab Learn which five Docker containers make a reliable home AI lab, from Ollama and Open WebUI to Qdrant, n8n and Portainer CE, what job each one does, and the first mistake to avoid with each. The first weekend goes well. A model runs locally, a chat window answers questions, and the laptop fan finally has a reason to exist. The third weekend is where home AI labs die. Three containers sit on ports nobody wrote down. A folder of PDFs never got indexed, a cron job lives in someone's shell history, and an update wiped the models because they were stored inside the container. Not one of those failures is an AI problem; every one of them is a lab problem. A useful home lab is a small set of containers with clear jobs, not one giant AI box. KDnuggets took the same layered view in its look at the local AI stack https://www.kdnuggets.com/the-local-ai-stack-for-productive-slms . Model serving, an interface, retrieval memory, automation and management are separate layers, and each one is a container you can replace without touching the others. The five below are picked for the friction they remove rather than for how much AI is in the name. Two of them have no AI in them at all. | Container | Job in the stack | First caution | |---|---|---| | Ollama | Serve: runs local models, exposes port 11434 | Models kept inside the container vanish on recreate; use a named volume | | Open WebUI | Talk: browser chat, users, files, provider links | A regenerated WEBUI SECRET KEY logs everyone out; set it once | | Qdrant | Remember: vector search over your own documents | Not needed for casual chat; add it when retrieval matters | | n8n | Automate: scheduled ingestion, webhooks, glue jobs | Holds credentials; back up its volume and keep the port private | | Portainer CE | Manage: restarts, logs, volumes, updates, cleanup | Docker socket access controls the host; trusted network only | 1. Ollama: The Model Runtime Ollama https://ollama.com/ is the engine room. It pulls and runs local models and exposes an HTTP endpoint on port 11434. Every other container in the lab talks to that endpoint, so nothing else needs to know how a model is loaded. The official Docker Hub image https://hub.docker.com/r/ollama/ollama ships three routes: CPU only, NVIDIA GPUs through the NVIDIA Container Toolkit with --gpus=all , and AMD GPUs through the rocm tag. The install detail that matters most is the volume. Models are large, and a container recreated without a named volume downloads them all again. Worse, it can lose a model you pulled from a tag that has since moved. The documented run command is: docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama Drop --gpus=all for the CPU route. One honest caveat: GPU pass-through depends on the host, not on Ollama. The documented Docker GPU routes are Linux and Windows with WSL2. Docker Desktop on macOS does not hand containers an NVIDIA GPU, so a Mac lab either runs Ollama natively for speed or accepts CPU inference inside Docker. 2. Open WebUI: The Interface Layer A terminal is fine for testing a model and useless for living with one. Open WebUI https://openwebui.com/ puts a browser interface on the lab: chats, users, file uploads, and connections to Ollama and to any other provider you add. Docker is the officially recommended install https://docs.openwebui.com/getting-started/quick-start/ , and the quick start is one command: docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI SECRET KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main Two of those flags earn their place. The volume at /app/backend/data holds users, chats and uploaded files. Without it, every recreate starts the lab from a blank login page. WEBUI SECRET KEY signs sessions, so set it once with something like openssl rand -hex 32 and keep it, because a container that regenerates the key logs everyone out. The :cuda and :ollama image variants exist for GPU acceleration and for bundling Ollama inside the same container. Both are options rather than the default, since a separate Ollama container is easier to upgrade and to share between tools. 3. Qdrant: Retrieval Memory for the Lab Do you need a vector database at all? For casual chat, no, and skipping it is a legitimate decision. The moment the lab is supposed to answer questions from your own documents, notes or project files, those files have to be embedded and searched. That is what Qdrant https://qdrant.tech/ does: it stores vectors and finds the nearest ones. It is the memory the AI consults, which is a different thing from the AI itself. The local quick start https://qdrant.tech/documentation/quick-start/ runs Qdrant with two ports, 6333 for the REST API and dashboard and 6334 for gRPC. The quick start also mounts a storage path so collections survive restarts. The dashboard at localhost:6333/dashboard is worth opening once, because seeing a collection fill up is the fastest way to confirm an ingestion pipeline is doing anything. A model alone is not a useful system, a point the earlier KDnuggets piece on self-hosted LLMs in the real world https://www.kdnuggets.com/self-hosted-llms-in-the-real-world-limits-workarounds-and-hard-lessons makes at length. Retrieval is usually the layer that turns a demo into something you reach for. 4. n8n: The Automation Spine Every lab eventually needs glue. Files land in a folder and need embedding, model endpoints need calling on a schedule, and summaries need pushing somewhere a human will read them. n8n https://n8n.io/ is a workflow tool that runs those jobs visually. Its Docker Compose install https://docs.n8n.io/deploy/host-n8n/install-options/install-using-docker-compose is the sensible route for a lab, because Compose is how the rest of the stack should be described anyway. It serves on localhost:5678 and defaults to SQLite, with Postgres as the option once workflows multiply. Three workflows show what it is for: - Nightly ingestion. A scheduled job watches a folder, chunks new files and writes them into Qdrant. - Summarize on demand. A webhook takes a URL or a PDF, sends it through Ollama for a summary and posts the result to a notes file. - Weekly review. A job asks the model to read the week's lab notes and flag anything unfinished. The caution is the obvious one. n8n holds credentials and workflow definitions, so its volume needs a backup, and its port has no business being exposed beyond the home network. 5. Portainer CE: The Control Panel After four containers, the boring work becomes the work: restarts, logs, volume checks, spotting an image with an update, cleaning up the six stopped containers from last month. Portainer CE https://www.portainer.io/ gives all of that a browser UI. The official install https://docs.portainer.io/start/install-ce/server/docker/linux runs on HTTPS port 9443 and keeps its own state in a portainer data volume. The install also mounts the Docker socket at /var/run/docker.sock . The socket mount is the part to respect. Whatever can talk to the Docker socket can control every container on the host. That is what makes Portainer useful, and it is also why it belongs on a trusted machine behind a home network, never on a public port. Treat it as the admin console it is. The lab won't need it on day one; it will need it the first time someone asks why the chat interface stopped answering and nobody can remember which container to restart. Starting With Two Containers and Adding the Rest When the Lab Demands Them The five jobs are serve, talk, remember, automate and manage. Ollama and Open WebUI cover the first two and make a working lab in an afternoon. Add Qdrant when you catch yourself pasting the same document into a chat for the third time. Add n8n when a manual step has happened often enough to be annoying. Add Portainer before the stack becomes opaque, which is usually the week after n8n arrives. Describe all of it in one Compose file from the start. Compose gives every service a name the others can use as a hostname, so Open WebUI reaches the model at http://ollama:11434 with no IP addresses involved. Named volumes https://docs.docker.com/reference/compose-file/volumes/ are the difference between "tried this once" and a lab that survives updates. Pin image versions for anything other people rely on, because :main and :latest tags move without asking. When the stack is stable and the itch to tune it arrives, the earlier KDnuggets piece on practical Docker configurations https://www.kdnuggets.com/5-practical-docker-configurations is the next read. Until then, five containers with clear jobs beat fifteen with clever names. \ Nahla Davies\ http://nahlawrites.com/ http://nahlawrites.com/ is a software developer and tech writer. Before devoting her work full time to technical writing, she managed—among other intriguing things—to serve as a lead programmer at an Inc. 5,000 experiential branding organization whose clients include Samsung, Time Warner, Netflix, and Sony.