Building an AI agent locally is an exciting first step. Running that same agent reliably in production is a different challenge.
Once real users and external services are involved, the application needs more than working code. It needs repeatable deployments, secure configuration, health checks, monitoring, controlled updates, and a clear recovery process.
This article is part of my MCP series. If you are new to the topic, start with my first article:
[Model Context Protocol (MCP) Servers Explained: A Complete Beginner’s Guide].
In this article, I will outline a practical architecture for taking a Model Context Protocol, or MCP-based, AI agent from a local development environment to Kubernetes.
This is a production architecture blueprint. The exact implementation will depend on the AI provider, MCP servers, cloud platform, and security requirements used by the application.
The Model Context Protocol provides a standardized way for AI applications to connect with external tools, services, and data sources.
An MCP-based agent may interact with:
A basic implementation might work well on a developer's machine. In production, however, every dependency introduces operational questions:
These are familiar DevOps and Site Reliability Engineering problems applied to a new type of workload.
A practical delivery flow could look like this:
Developer
↓
GitHub Repository
↓
GitHub Actions
↓
Container Registry
↓
Kubernetes Cluster
↓
MCP Servers and External Services
↓
Logs, Metrics, Traces, and Alerts
Each component has a clear responsibility:
Containerization gives the application a consistent runtime across development, testing, and production environments.
A simple Python-based agent could use the following Dockerfile:
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd --create-home appuser
USER appuser
EXPOSE 8000
CMD ["python", "app.py"]
This example follows several useful practices:
The container image should not contain API keys, access tokens, or environment-specific credentials.
Kubernetes provides a consistent way to deploy, restart, scale, and update the service.
A simplified deployment might look like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: mcp-agent
spec:
replicas: 2
selector:
matchLabels:
app: mcp-agent
template:
metadata:
labels:
app: mcp-agent
spec:
containers:
- name: mcp-agent
image: registry.example.com/mcp-agent:1.0.0
ports:
- containerPort: 8000
envFrom:
- secretRef:
name: mcp-agent-secrets
readinessProbe:
httpGet:
path: /ready
port: 8000
livenessProbe:
httpGet:
path: /health
port: 8000
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
This configuration introduces several production controls:
The values should be adjusted after observing the application's real resource usage.
An AI agent may require credentials for model providers, MCP servers, databases, or external APIs.
These values should never be committed to Git or embedded in a container image.
Kubernetes Secrets provide a basic separation between application code and sensitive configuration. For stronger production security, the cluster can integrate with a dedicated secrets platform such as:
Access should follow the principle of least privilege. The agent should receive only the permissions it needs, and credentials should have a defined rotation process.
A reliable CI/CD pipeline should verify the application before deploying it.
A typical pipeline could include:
A simplified GitHub Actions workflow could begin like this:
name: Build and Deploy
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Run tests
run: |
pip install -r requirements.txt
pytest
- name: Build container image
run: |
docker build -t mcp-agent:${{ github.sha }} .
Production pipelines should use pinned action versions, protected environments, secure authentication, and immutable image tags.
Using the Git commit SHA as an image tag also makes it easier to identify exactly which code version is running.
Traditional infrastructure metrics are important, but they are not enough for an AI agent.
A useful observability strategy should cover both the platform and the application.
Monitor:
Monitor:
Structured logs should include fields such as:
Sensitive prompts, credentials, personal information, and full model responses should not be written to logs without appropriate controls.
Distributed tracing can help follow a request across:
User Request → Agent → Model Provider → MCP Server → External Service
This becomes especially valuable when the total response time depends on several external systems.
An MCP server or external API will eventually become slow, unavailable, or rate limited. The agent should handle these situations without causing a wider service failure.
Useful reliability controls include:
Retries should be used carefully. Repeating an unsafe or non-idempotent action could create duplicate records or trigger the same operation multiple times.
Kubernetes can scale replicas horizontally, but CPU usage may not always reflect the true load of an AI application.
Depending on the architecture, scaling decisions could consider:
Scaling the agent does not automatically scale its dependencies. A larger number of agent replicas can place additional pressure on databases, MCP servers, and third-party APIs.
Capacity planning should therefore consider the complete request path.
Production AI systems introduce risks beyond normal application security.
Important controls include:
An agent should not receive broad infrastructure or business-system access simply because it can use an MCP tool. Every action should still pass through clear authentication and authorization controls.
Before releasing an MCP-based agent, confirm that:
Building an AI agent demonstrates application functionality. Productionizing it demonstrates engineering maturity.
Docker provides a consistent runtime. Kubernetes manages availability and scaling. CI/CD enables controlled releases. Observability shows how the system behaves. Security and reliability controls determine whether the service can be trusted in a real environment.
MCP may introduce a new integration model, but the production principles remain familiar: automate delivery, reduce unnecessary access, observe every dependency, expect failures, and make recovery part of the design.
How would you approach scaling and monitoring an MCP-based agent in your environment?