This is the code repository for Operational AI with Docker: LLMOps, Agents and Multi-Model Systems with Docker and Kubernetes, published by Packt.
Build, deploy and scale production-ready AI applications using Docker's integrated AI toolkit.
If you've ever wanted to take an AI app from "works on my laptop" to something you can actually run in production, this book is for you. It walks through the full lifecycle running local LLMs, wiring them into real applications, integrating external tools through MCP, building autonomous agents and finally orchestrating fleets of agents on Kubernetes all using Docker's AI tooling.
You'll work hands-on with Docker Desktop, Docker Model Runner, MCP Gateway, Docker Hardened Images, kagent and you'll see how the same containers you already know can carry AI workloads safely and at scale.
- Run and optimize local LLMs with Docker Model Runner
- Integrate AI applications with external systems using MCP (Model Context Protocol)
- Deploy MCP servers securely with Docker MCP Gateway
- Build autonomous AI agents with multi-agent architectures
- Implement production security with Docker Hardened Images
- Monitor AI workloads with Prometheus and Grafana
- Integrate AI with GitHub, Slack, Kubernetes and databases
- Scale AI applications from development to production
- Implement enterprise security patterns for AI deployments
- Automate AI workflows with Docker Compose and orchestration
Each chapter has its own folder with runnable code and a chapter-specific README.md
. Click any chapter title to jump straight to its code.
| # | Chapter | What's inside |
|---|---|---|
| 1 | ||
tiny-service-container
, tiny-training-run
) get you comfortable with docker run
and docker build
before things get serious.Understanding AI Models in Dockermodels:
provider syntax for declaring model dependencies alongside your services.Model Serving with Docker Model RunnerDocker Offloadexport_and_quantize.py
pipeline.Running ML Container Models on KubernetesProtocol-Based AI Integration with MCPBuilding Autonomous AI AgentsMulti-Model and Multi-Agent ArchitecturesAdvanced Agent Orchestrationkagent
. Auto-registration, peer discovery, distributed tracing and sandboxed execution patterns for real workloads.You don't need to be an AI expert, but you should be comfortable on the command line. Specifically:
Docker Desktop(4.40+) with** Model Runnerenabled — required for chapters 2 onwards Docker Compose v2**(ships with Docker Desktop)** Gitto clone the repo~16 GB RAM** recommended if you want to run local LLMs comfortably; a GPU helps but isn't requiredkubectl and a local Kubernetes cluster (Docker Desktop's built-in k8s,kind
, orminikube
) — only needed for chapters 5 and 9- A basic grasp of Docker and what an LLM is. That's it.
The examples are tested on macOS, Windows and Linux.
Clone the repo and cd
into whichever chapter you want to try. Most examples are a single docker compose up
away.
git clone https://github.com/PacktPublishing/Operational-AI-with-Docker.git
cd Operational-AI-with-Docker/chap-03/05-chatbot
docker compose up
Every chapter folder has its own README.md
with the exact commands, expected output and any setup notes specific to that chapter. If something doesn't work, that's the first place to look.
A typical example looks like this:
services:
gateway:
image: docker/mcp-gateway
command:
- --transport=sse
- --port=8080
Following is what you need for this book:
This book is for DevOps engineers, platform engineers, AI/ML engineers, solutions architects and developers who want to operationalize AI applications. Whether you're deploying your first LLM or building complex multi-agent systems, this book provides practical guidance for production AI with Docker.
A basic understanding of Docker containers and AI concepts is helpful but not required. The book assumes familiarity with command-line tools and includes hands-on examples that work on macOS, Windows and Linux.