How teams move from "one API key per runtime" to "one gateway, any runtime"
You've got a squad of coding agents now. Some teams use Claude Managed Agents. Others prefer Cursor. Finance is piloting Bedrock AgentCore. Engineering wants to stay self-hosted.
This sounds flexible. It's actually a disaster.
Here's what happens:
The models are powerful. The infrastructure is a mess. Each runtime has its own authentication, its own API surface, its own session model. The agents themselves are stateless—the infrastructure that manages them is fragmented.
This is the multi-runtime agent problem of July 2026. Not a model problem. An infrastructure problem.
The answer is not "pick one runtime." The answer is: separate the agent control plane from the agent runtime.
The pattern:
The control plane sits between teams and runtimes. It's a unified gateway that:
Let's build it:
Your control plane starts by connecting to each runtime. One YAML config:
runtimes:
claude_managed:
type: anthropic
api_key: env/ANTHROPIC_API_KEY
cursor:
type: cursor
api_key: env/CURSOR_API_KEY
bedrock:
type: aws_bedrock
region: us-west-2
credentials: env/AWS_ROLE
self_hosted:
type: custom
base_url: https://internal-agent-api.company.com
The control plane normalizes these. A single POST request to the control plane invokes any runtime:
curl -X POST https://control-plane.company.com/agents/pr-babysitter/invoke \
-H "Authorization: Bearer $CONTROL_PLANE_KEY" \
-d '{
"runtime_hint": "any", # Fallback: try best available
"prompt": "Review this PR for security issues",
"tools": ["git", "github_api"]
}'
The control plane picks a runtime (round-robin, cost-aware, or explicit), translates the request, executes, and returns a unified response format.
No developer touches provider consoles directly. Instead:
credentials:
anthropic_shared:
provider: anthropic
api_key: vault://secrets/anthropic-prod
rotated: 2026-07-20
last_used: 2026-07-24
cursor_shared:
provider: cursor
api_key: vault://secrets/cursor-prod
last_used: 2026-07-24
Teams request agent invocation. The control plane handles credential binding. No sprawl. Auditable.
Tools, budgets, rate limits, and approvals live in the control plane—not scattered across four provider consoles.
agents:
pr_babysitter:
owner: engineering
runtime: any # Will use cheapest/fastest available
tools:
- github: read
- linear: read
budgets:
daily_usd: 50
monthly_usd: 1000
rate_limits:
calls_per_minute: 10
concurrent_sessions: 5
approval_rules:
- if_tool: github_push
requires: human_approval
- if_cost_exceeds_usd: 100
requires: human_approval
Policy changes propagate instantly. No redeploy. No framework fighting.
Sessions live in the control plane (Postgres-backed), not in the runtime's memory. An agent running on Cursor can crash, and the control plane resumes it on Bedrock without losing state:
session:
id: sess-pr-babysitter-82946
agent: pr_babysitter
runtime: cursor # Started here
created_at: 2026-07-24T14:30:00Z
steps:
- turn: 1
tool_call: github_list_prs
result: ["PR #1024", "PR #1025"]
cost_usd: 0.12
- turn: 2
tool_call: github_get_diff PR#1024
result: "[diff content]"
cost_usd: 0.08
status: executing
last_activity: 2026-07-24T14:32:15Z
If the Cursor runtime crashes at turn 3, the control plane can:
Control plane maintains a registry of all agents. Teams discover them:
curl https://control-plane.company.com/agents \
-H "Authorization: Bearer $KEY"
{
"agents": [
{
"id": "pr_babysitter",
"owner": "engineering",
"runtime": "any",
"description": "Reviews PRs for security issues",
"calls_this_month": 4829,
"avg_cost_per_call": 0.23
},
{
"id": "compliance_check",
"owner": "legal",
"runtime": "any",
"description": "Audits contracts for risk clauses",
"calls_this_month": 312,
"avg_cost_per_call": 1.42
}
]
}
Finance calls the PR babysitter. Marketing uses the compliance checker. All through the same control plane.
Before (fragmented):
After (control plane):
This is not theoretical. LiteLLM Agent Platform is a Rust-based AI Gateway and Agent Control Plane whose goal is to let teams register, invoke, observe, and govern agents across multiple runtimes, with LiteLLM treating this as an experimental project to learn quickly.
LiteLLM Agent Platform manages unified API across runtimes—one API to create and run agents, regardless of the runtime underneath, with developers creating and running agents here without requiring Bedrock or Anthropic console access.
The five-question evaluation framework:
If the answer is "no" to any of these, you're still in the fragmented phase.
July 2026 moment: Teams are shipping agents on multiple runtimes. The ones that don't solve the fragmentation problem hit walls around month 3:
The ones that build a control plane first move faster. Fewer meetings about credential management. Fewer surprises on the bill. Fewer post-mortems about lost work.
The pattern is converging. Agent infrastructure is separating into three layers: models, harnesses, and runtimes, with a fourth layer emerging as the unified agent control plane to allow calling agents living in different agent runtimes all from 1 place, because companies will not run every agent on one runtime.
This is the infrastructure layer that will define who wins in multi-agent 2026. Not the smartest models. Not the flashiest frameworks. The teams with the clearest control plane.
Paul Twist is an AI infrastructure engineer based in Berlin. This post covers the multi-runtime control plane pattern emerging as table-stakes for production agent deployments in July 2026.