# Day 7: Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally

> Source: <https://dev.to/suresh_kumar_de3920bedd1c/day-7-docker-compose-architecting-a-multi-container-database-and-inference-stack-locally-4b6d>
> Published: 2026-11-13 20:38:36+00:00

A production-ready RAG platform is not one script. It needs several specialised services on a shared internal network. **Docker Compose** is the declarative, infrastructure-as-code way to assemble them on a single host, and it closes Week 1 by pulling together everything so far.

**Diagram:** FIG 1 · A gateway publishes one port. The vector store and inference engine sit on a private bridge network, each with its own persistent volume. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).

Three services make up the stack:

```
name: ai-stack

networks:
  ai-internal-net:
    driver: bridge

volumes:
  qdrant_storage:
  ollama_storage:

services:
  vector-store:
    image: qdrant/qdrant:v1.9.0
    networks: [ai-internal-net]
    volumes:
      - qdrant_storage:/qdrant/storage
    healthcheck:
      # A TCP probe avoids needing curl in the image. Adjust to your image version.
      test: ["CMD-SHELL", "bash -c ':> /dev/tcp/127.0.0.1/6333' || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 10

  inference-engine:
    image: ollama/ollama:latest          # pin a tested tag in production
    networks: [ai-internal-net]
    volumes:
      - ollama_storage:/root/.ollama
    deploy:
      resources:
        limits:
          memory: 10G                    # leave about 6 GB of a 16 GB host free
    healthcheck:
      test: ["CMD", "ollama", "list"]
      interval: 15s
      timeout: 10s
      retries: 10

  ai-gateway:
    build: ./gateway
    networks: [ai-internal-net]
    ports:
      - "80:8000"                        # the only port published to the host
    environment:
      VECTOR_DB_HOST: vector-store
      VECTOR_DB_PORT: "6333"
      LLM_HOST: inference-engine
      LLM_PORT: "11434"
    depends_on:
      vector-store:
        condition: service_healthy
      inference-engine:
        condition: service_healthy
```

Start it with `docker compose up -d --build`, check it with `docker compose ps`, and stop it with `docker compose down`. Add `-v` to `down` only when you want the volumes deleted too.

**1. Service discovery by name.** Inside a Compose network every service name is a DNS name. Use it, and never hard-code IP addresses or `localhost`: container IPs change on restart, and `localhost` points at the calling container itself.

**Diagram:** FIG 2 · Inside a Compose network, each service name is a DNS name. Hard-coded IPs break on restart, and localhost points at the calling container itself. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).

**2. Healthchecks and dependency ordering.** `depends_on` with `condition: service_healthy` makes the gateway wait until both services actually pass their checks, so it never queries a database or model server that is still starting.

**Diagram:** FIG 3 · The gateway waits until the vector store and the inference engine pass their healthchecks, so it never queries a service that is still starting. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).

**3. Volume decoupling.** Keep persistent data in named volumes, outside the containers. You can then upgrade, restart or rebuild the stack without losing the vector index or re-downloading model weights.

**Diagram:** FIG 4 · Persistent data lives in named volumes, so containers stay disposable. A memory limit on the inference engine protects the host. [See the animated version](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose).

Two practical notes. The memory limit under `deploy.resources` is honoured by current `docker compose`, but check it with `docker stats` on your setup. And the healthcheck commands depend on what each image ships, so confirm they work with `docker compose ps` showing `healthy`.

That completes Week 1: lean images, GPU access, quantized weights, dedicated runtimes, a validated API and a stack that starts in the right order. Next week moves it all onto Kubernetes, starting with a lightweight cluster on a 16 GB machine.

This is the exact example from the video. Docker Compose itself reads this file, and a short check prints what it found. Nothing is started, and the gateway image is a placeholder.

```
name: ai-stack
services:
  vector-store:
    image: qdrant/qdrant:v1.9.0
    volumes: [qdrant_data:/qdrant/storage]
  inference-engine:
    image: ollama/ollama:latest
    volumes: [ollama_data:/root/.ollama]
  ai-gateway:
    image: ai-gateway:dev
    ports: ["80:8000"]
    environment:
      VECTOR_DB_HOST: vector-store
      LLM_HOST: inference-engine
    depends_on: [vector-store, inference-engine]
volumes: {qdrant_data: {}, ollama_data: {}}
```

Helper file `check.py`, used by the run:

``` python
import json, subprocess
raw = subprocess.check_output(["docker", "compose", "config", "--format", "json"])
c = json.loads(raw)
svc = c["services"]
print("services:", len(svc))
for n, s in svc.items():
    for p in s.get("ports", []):
        print("published:", n, p["published"] + "->" + str(p["target"]))
g = svc["ai-gateway"]
print("db host:", g["environment"]["VECTOR_DB_HOST"])
for d in sorted(g["depends_on"]):
    print("waits for:", d)
print("volumes:", ", ".join(sorted(c["volumes"])))
```

Run it:

```
python check.py
```

Output I got while recording the video:

``` php
services: 3
published: ai-gateway 80->8000
db host: vector-store
waits for: inference-engine
waits for: vector-store
volumes: ollama_data, qdrant_data
```

**Day 8:** K3d and lightweight Kubernetes: running multi-node clusters on 16 GB of RAM.

*Originally published at [https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose](https://sureshpallapothu.in/blog/ai-int-day-7-docker-compose), where this post includes animated diagrams.*