Day 7: Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally

A developer published a walkthrough for assembling a local retrieval-augmented generation stack with Docker Compose, wiring a Qdrant vector store, an Ollama inference engine and a gateway into a single bridge network. The setup publishes only the gateway port, uses service-name DNS for discovery, gates startup on healthchecks via depends_on conditions, and keeps the vector index and model weights in named volumes so containers stay disposable. The author caps the inference engine at 10 GB of memory on a 16 GB host and notes the limit should be verified with docker stats.

A production-ready RAG platform is not one script. It needs several specialised services on a shared internal network. Docker Compose is the declarative, infrastructure-as-code way to assemble them on a single host, and it closes Week 1 by pulling together everything so far. Diagram: FIG 1 · A gateway publishes one port. The vector store and inference engine sit on a private bridge network, each with its own persistent volume. See the animated version https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose . Three services make up the stack: name: ai-stack networks: ai-internal-net: driver: bridge volumes: qdrant storage: ollama storage: services: vector-store: image: qdrant/qdrant:v1.9.0 networks: ai-internal-net volumes: - qdrant storage:/qdrant/storage healthcheck: A TCP probe avoids needing curl in the image. Adjust to your image version. test: "CMD-SHELL", "bash -c ': /dev/tcp/127.0.0.1/6333' || exit 1" interval: 10s timeout: 5s retries: 10 inference-engine: image: ollama/ollama:latest pin a tested tag in production networks: ai-internal-net volumes: - ollama storage:/root/.ollama deploy: resources: limits: memory: 10G leave about 6 GB of a 16 GB host free healthcheck: test: "CMD", "ollama", "list" interval: 15s timeout: 10s retries: 10 ai-gateway: build: ./gateway networks: ai-internal-net ports: - "80:8000" the only port published to the host environment: VECTOR DB HOST: vector-store VECTOR DB PORT: "6333" LLM HOST: inference-engine LLM PORT: "11434" depends on: vector-store: condition: service healthy inference-engine: condition: service healthy Start it with docker compose up -d --build , check it with docker compose ps , and stop it with docker compose down . Add -v to down only when you want the volumes deleted too. 1. Service discovery by name. Inside a Compose network every service name is a DNS name. Use it, and never hard-code IP addresses or localhost : container IPs change on restart, and localhost points at the calling container itself. Diagram: FIG 2 · Inside a Compose network, each service name is a DNS name. Hard-coded IPs break on restart, and localhost points at the calling container itself. See the animated version https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose . 2. Healthchecks and dependency ordering. depends on with condition: service healthy makes the gateway wait until both services actually pass their checks, so it never queries a database or model server that is still starting. Diagram: FIG 3 · The gateway waits until the vector store and the inference engine pass their healthchecks, so it never queries a service that is still starting. See the animated version https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose . 3. Volume decoupling. Keep persistent data in named volumes, outside the containers. You can then upgrade, restart or rebuild the stack without losing the vector index or re-downloading model weights. Diagram: FIG 4 · Persistent data lives in named volumes, so containers stay disposable. A memory limit on the inference engine protects the host. See the animated version https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose . Two practical notes. The memory limit under deploy.resources is honoured by current docker compose , but check it with docker stats on your setup. And the healthcheck commands depend on what each image ships, so confirm they work with docker compose ps showing healthy . That completes Week 1: lean images, GPU access, quantized weights, dedicated runtimes, a validated API and a stack that starts in the right order. Next week moves it all onto Kubernetes, starting with a lightweight cluster on a 16 GB machine. This is the exact example from the video. Docker Compose itself reads this file, and a short check prints what it found. Nothing is started, and the gateway image is a placeholder. name: ai-stack services: vector-store: image: qdrant/qdrant:v1.9.0 volumes: qdrant data:/qdrant/storage inference-engine: image: ollama/ollama:latest volumes: ollama data:/root/.ollama ai-gateway: image: ai-gateway:dev ports: "80:8000" environment: VECTOR DB HOST: vector-store LLM HOST: inference-engine depends on: vector-store, inference-engine volumes: {qdrant data: {}, ollama data: {}} Helper file check.py , used by the run: python import json, subprocess raw = subprocess.check output "docker", "compose", "config", "--format", "json" c = json.loads raw svc = c "services" print "services:", len svc for n, s in svc.items : for p in s.get "ports", : print "published:", n, p "published" + "- " + str p "target" g = svc "ai-gateway" print "db host:", g "environment" "VECTOR DB HOST" for d in sorted g "depends on" : print "waits for:", d print "volumes:", ", ".join sorted c "volumes" Run it: python check.py Output I got while recording the video: php services: 3 published: ai-gateway 80- 8000 db host: vector-store waits for: inference-engine waits for: vector-store volumes: ollama data, qdrant data Day 8: K3d and lightweight Kubernetes: running multi-node clusters on 16 GB of RAM. Originally published at https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose , where this post includes animated diagrams.