{"slug": "day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack", "title": "Day 7: Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally", "summary": "A developer published a walkthrough for assembling a local retrieval-augmented generation stack with Docker Compose, wiring a Qdrant vector store, an Ollama inference engine and a gateway into a single bridge network. The setup publishes only the gateway port, uses service-name DNS for discovery, gates startup on healthchecks via depends_on conditions, and keeps the vector index and model weights in named volumes so containers stay disposable. The author caps the inference engine at 10 GB of memory on a 16 GB host and notes the limit should be verified with docker stats.", "body_md": "A production-ready RAG platform is not one script. It needs several specialised services on a shared internal network. **Docker Compose** is the declarative, infrastructure-as-code way to assemble them on a single host, and it closes Week 1 by pulling together everything so far.\n\n**Diagram:** FIG 1 · A gateway publishes one port. The vector store and inference engine sit on a private bridge network, each with its own persistent volume. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).\n\nThree services make up the stack:\n\n```\nname: ai-stack\n\nnetworks:\n  ai-internal-net:\n    driver: bridge\n\nvolumes:\n  qdrant_storage:\n  ollama_storage:\n\nservices:\n  vector-store:\n    image: qdrant/qdrant:v1.9.0\n    networks: [ai-internal-net]\n    volumes:\n      - qdrant_storage:/qdrant/storage\n    healthcheck:\n      # A TCP probe avoids needing curl in the image. Adjust to your image version.\n      test: [\"CMD-SHELL\", \"bash -c ':> /dev/tcp/127.0.0.1/6333' || exit 1\"]\n      interval: 10s\n      timeout: 5s\n      retries: 10\n\n  inference-engine:\n    image: ollama/ollama:latest          # pin a tested tag in production\n    networks: [ai-internal-net]\n    volumes:\n      - ollama_storage:/root/.ollama\n    deploy:\n      resources:\n        limits:\n          memory: 10G                    # leave about 6 GB of a 16 GB host free\n    healthcheck:\n      test: [\"CMD\", \"ollama\", \"list\"]\n      interval: 15s\n      timeout: 10s\n      retries: 10\n\n  ai-gateway:\n    build: ./gateway\n    networks: [ai-internal-net]\n    ports:\n      - \"80:8000\"                        # the only port published to the host\n    environment:\n      VECTOR_DB_HOST: vector-store\n      VECTOR_DB_PORT: \"6333\"\n      LLM_HOST: inference-engine\n      LLM_PORT: \"11434\"\n    depends_on:\n      vector-store:\n        condition: service_healthy\n      inference-engine:\n        condition: service_healthy\n```\n\nStart it with `docker compose up -d --build`, check it with `docker compose ps`, and stop it with `docker compose down`. Add `-v` to `down` only when you want the volumes deleted too.\n\n**1. Service discovery by name.** Inside a Compose network every service name is a DNS name. Use it, and never hard-code IP addresses or `localhost`: container IPs change on restart, and `localhost` points at the calling container itself.\n\n**Diagram:** FIG 2 · Inside a Compose network, each service name is a DNS name. Hard-coded IPs break on restart, and localhost points at the calling container itself. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).\n\n**2. Healthchecks and dependency ordering.** `depends_on` with `condition: service_healthy` makes the gateway wait until both services actually pass their checks, so it never queries a database or model server that is still starting.\n\n**Diagram:** FIG 3 · The gateway waits until the vector store and the inference engine pass their healthchecks, so it never queries a service that is still starting. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).\n\n**3. Volume decoupling.** Keep persistent data in named volumes, outside the containers. You can then upgrade, restart or rebuild the stack without losing the vector index or re-downloading model weights.\n\n**Diagram:** FIG 4 · Persistent data lives in named volumes, so containers stay disposable. A memory limit on the inference engine protects the host. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).\n\nTwo practical notes. The memory limit under `deploy.resources` is honoured by current `docker compose`, but check it with `docker stats` on your setup. And the healthcheck commands depend on what each image ships, so confirm they work with `docker compose ps` showing `healthy`.\n\nThat completes Week 1: lean images, GPU access, quantized weights, dedicated runtimes, a validated API and a stack that starts in the right order. Next week moves it all onto Kubernetes, starting with a lightweight cluster on a 16 GB machine.\n\nThis is the exact example from the video. Docker Compose itself reads this file, and a short check prints what it found. Nothing is started, and the gateway image is a placeholder.\n\n```\nname: ai-stack\nservices:\n  vector-store:\n    image: qdrant/qdrant:v1.9.0\n    volumes: [qdrant_data:/qdrant/storage]\n  inference-engine:\n    image: ollama/ollama:latest\n    volumes: [ollama_data:/root/.ollama]\n  ai-gateway:\n    image: ai-gateway:dev\n    ports: [\"80:8000\"]\n    environment:\n      VECTOR_DB_HOST: vector-store\n      LLM_HOST: inference-engine\n    depends_on: [vector-store, inference-engine]\nvolumes: {qdrant_data: {}, ollama_data: {}}\n```\n\nHelper file `check.py`, used by the run:\n\n``` python\nimport json, subprocess\nraw = subprocess.check_output([\"docker\", \"compose\", \"config\", \"--format\", \"json\"])\nc = json.loads(raw)\nsvc = c[\"services\"]\nprint(\"services:\", len(svc))\nfor n, s in svc.items():\n    for p in s.get(\"ports\", []):\n        print(\"published:\", n, p[\"published\"] + \"->\" + str(p[\"target\"]))\ng = svc[\"ai-gateway\"]\nprint(\"db host:\", g[\"environment\"][\"VECTOR_DB_HOST\"])\nfor d in sorted(g[\"depends_on\"]):\n    print(\"waits for:\", d)\nprint(\"volumes:\", \", \".join(sorted(c[\"volumes\"])))\n```\n\nRun it:\n\n```\npython check.py\n```\n\nOutput I got while recording the video:\n\n``` php\nservices: 3\npublished: ai-gateway 80->8000\ndb host: vector-store\nwaits for: inference-engine\nwaits for: vector-store\nvolumes: ollama_data, qdrant_data\n```\n\n**Day 8:** K3d and lightweight Kubernetes: running multi-node clusters on 16 GB of RAM.\n\n*Originally published at [https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose), where this post includes animated diagrams.*", "url": "https://wpnews.pro/news/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack", "canonical_source": "https://dev.to/suresh_kumar_de3920bedd1c/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack-locally-4b6d", "published_at": "2026-11-13 20:38:36+00:00", "updated_at": "2026-11-13 20:50:58.154056+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "mlops", "developer-tools", "ai-agents"], "entities": ["Docker Compose", "Qdrant", "Ollama", "Docker"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack", "markdown": "https://wpnews.pro/news/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack.md", "text": "https://wpnews.pro/news/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack.txt", "jsonld": "https://wpnews.pro/news/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack.jsonld"}}