# Multi-Agent LLM Orchestration with Docker Compose and MCP

> Source: <https://github.com/PacktPublishing/Operational-AI-with-Docker>
> Published: 2026-05-26 06:07:06+00:00

This is the code repository for **Operational AI with Docker: LLMOps, Agents and Multi-Model Systems with Docker and Kubernetes**, published by Packt.

Build, deploy and scale production-ready AI applications using Docker's integrated AI toolkit.

If you've ever wanted to take an AI app from "works on my laptop" to something you can actually run in production, this book is for you. It walks through the full lifecycle running local LLMs, wiring them into real applications, integrating external tools through MCP, building autonomous agents and finally orchestrating fleets of agents on Kubernetes all using Docker's AI tooling.

You'll work hands-on with Docker Desktop, Docker Model Runner, MCP Gateway, Docker Hardened Images, kagent and you'll see how the same containers you already know can carry AI workloads safely and at scale.

- Run and optimize local LLMs with Docker Model Runner
- Integrate AI applications with external systems using MCP (Model Context Protocol)
- Deploy MCP servers securely with Docker MCP Gateway
- Build autonomous AI agents with multi-agent architectures
- Implement production security with Docker Hardened Images
- Monitor AI workloads with Prometheus and Grafana
- Integrate AI with GitHub, Slack, Kubernetes and databases
- Scale AI applications from development to production
- Implement enterprise security patterns for AI deployments
- Automate AI workflows with Docker Compose and orchestration

Each chapter has its own folder with runnable code and a chapter-specific `README.md`

. Click any chapter title to jump straight to its code.

| # | Chapter | What's inside |
|---|---|---|
| 1 |
|

`tiny-service-container`

, `tiny-training-run`

) get you comfortable with `docker run`

and `docker build`

before things get serious.[Understanding AI Models in Docker](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-02)`models:`

provider syntax for declaring model dependencies alongside your services.[Model Serving with Docker Model Runner](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-03)[Docker Offload](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-04)`export_and_quantize.py`

pipeline.[Running ML Container Models on Kubernetes](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-05)[Protocol-Based AI Integration with MCP](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-06)[Building Autonomous AI Agents](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-07)[Multi-Model and Multi-Agent Architectures](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-08)[Advanced Agent Orchestration](/PacktPublishing/Operational-AI-with-Docker/blob/main/chap-09)`kagent`

. Auto-registration, peer discovery, distributed tracing and sandboxed execution patterns for real workloads.You don't need to be an AI expert, but you should be comfortable on the command line. Specifically:

**Docker Desktop**(4.40+) with** Model Runner**enabled — required for chapters 2 onwards** Docker Compose v2**(ships with Docker Desktop)** Git**to clone the repo**~16 GB RAM** recommended if you want to run local LLMs comfortably; a GPU helps but isn't required**kubectl** and a local Kubernetes cluster (Docker Desktop's built-in k8s,`kind`

, or`minikube`

) — only needed for chapters 5 and 9- A basic grasp of Docker and what an LLM is. That's it.

The examples are tested on macOS, Windows and Linux.

Clone the repo and `cd`

into whichever chapter you want to try. Most examples are a single `docker compose up`

away.

```
git clone https://github.com/PacktPublishing/Operational-AI-with-Docker.git
cd Operational-AI-with-Docker/chap-03/05-chatbot
docker compose up
```

Every chapter folder has its own `README.md`

with the exact commands, expected output and any setup notes specific to that chapter. If something doesn't work, that's the first place to look.

A typical example looks like this:

```
services:
  gateway:
    image: docker/mcp-gateway
    command:
      - --transport=sse
      - --port=8080
```

**Following is what you need for this book:**

This book is for DevOps engineers, platform engineers, AI/ML engineers, solutions architects and developers who want to operationalize AI applications. Whether you're deploying your first LLM or building complex multi-agent systems, this book provides practical guidance for production AI with Docker.

A basic understanding of Docker containers and AI concepts is helpful but not required. The book assumes familiarity with command-line tools and includes hands-on examples that work on macOS, Windows and Linux.