Originally published on tamiz.pro.
The naive image of an AI agent is a chatbot that gets smarter each turn. The production reality is far more demanding: agents that coordinate across services, respect security boundaries, follow standardized communication contracts, and survive unbounded request volume. The shift from prompts to infrastructure isn't philosophical — it's a hard engineering transition that separates demos from deployed systems.
Early AI applications leaned on prompt chaining: sequence of LLM calls linked by a human-readable narrative. This works until you need five agents handling 10,000 concurrent requests, each calling multiple downstream tools, with latency budgets measured in seconds and audit trails required for compliance.
Prompt-level systems face three fatal scaling problems:
The alternative is treating agents as infrastructure — services with explicit contracts, bounded execution contexts, and standardized inter-agent communication.
A2A (Agent-to-Agent) refers to the emerging class of protocols that let agents communicate with each other without a central orchestrator making every decision. Think RPC for agents: structured, typed, versioned, and observable.
Without A2A, agent ecosystems look like this:
With A2A, agents speak a common protocol. Each agent publishes a capability contract — a machine-readable description of what inputs it accepts, what outputs it produces, and what side effects it may have. Other agents discover and invoke capabilities through a typed interface, not by guessing JSON shapes.
A well-designed A2A system rests on three primitives:
Cardinality-bounded messages. Every inter-agent message has a defined lifecycle: sent, acknowledged, completed, or failed. Unlike fire-and-forget HTTP, A2A messages carry sequence numbers and correlation IDs so agents can reconstruct conversation history.
Typed capability descriptors. Instead of POST /agent/process
, a capability is declared as:
{
"type": "tool",
"name": "database_query",
"version": "1.2.0",
"input_schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["query", "connection_id"],
"properties": {
"query": { "type": "string", "maxLength": 4096 },
"connection_id": { "type": "string", "pattern": "^conn-[a-f0-9]+$" }
}
},
"output_schema": {
"type": "object",
"properties": {
"rows": { "type": "array", "maxItems": 1000 },
"truncated": { "type": "boolean" }
}
},
"error_schemas": [
{ "code": "QUERy_TOO_LARGE", "message": "Query exceeds 4096 characters" },
{ "code": "CONNECTION_UNAVAILABLE", "message": "Connection pool exhausted" }
]
}
This schema isn't decorative. It enables compile-time validation at the call site, runtime enforcement at the capability boundary, and automatic test generation for every contract change.
Trust-bound invocation context. Every A2A call carries an invocation context that answers: who is calling? what tools may they access? what is the budget? A2A protocols embed this as a signed token or policy bundle rather than a header you trust because you built the client.
A2A and sandboxes are complementary. A2A defines the contract — what can be asked and what must be returned. Sandboxes define the execution boundary — where and how the agent runs its logic. Together they create a system where agents can cooperate without trusting each other's internals.
An agent sandbox is a controlled environment where agent code runs with explicit resource limits, network restrictions, and output filtering. The sandbox is not a luxury — it is the mechanism that makes multi-agent systems safe.
Consider what goes wrong when agents run without sandboxes:
Each of these scenarios is addressable at the infrastructure layer rather than hoping the next prompt improvement catches it.
A production-grade agent sandbox implements three layers:
Compute sandbox. Isolated process execution with CPU, memory, and timeout budgets. Tools run as subprocesses or WebAssembly modules, not arbitrary Python inside the agent process. Tools that exceed their allocation are killed and reported as errors rather than hanging the orchestrator.
Network sandbox. Agents only reach networks they are authorized for. A read-only agent cannot call POST https://webhook.example.com/keys
. Egress is enforced by the host, not by the agent's code. Ingress is filtered against allowlists for inbound tool responses.
Data sandbox. Secrets, credentials, and PII live outside the agent's execution context. Tools receive tokens, not full credentials. Output streams are scanned for sensitive patterns before leaving the sandbox boundary.
Here is a minimal but production-oriented sandbox wrapper around a tool invocation:
import asyncio
import json
import time
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import Any, Optional
@dataclass
class SandboxConfig:
max_memory_mb: int = 256
timeout_seconds: float = 30.0
allowed_networks: list[str] | None = None
max_output_bytes: int = 65536
secret_prefixes: list[str] | None = None
class SandboxError(Exception):
pass
class SandboxTooMuchMemory(SandboxError):
pass
class SandboxTimeoutError(SandboxError):
pass
class SandboxNetworkBlocked(SandboxError):
pass
@asynccontextmanager
async def run_tool_in_sandbox(
tool_code: str,
arguments: dict[str, Any],
config: SandboxConfig,
):
start = time.monotonic()
payload = json.dumps({
"args": arguments,
"limits": {
"memory_mb": config.max_memory_mb,
"timeout_s": config.timeout_seconds,
"max_output_bytes": config.max_output_bytes,
},
"allowed_networks": config.allowed_networks,
"secret_prefixes": config.secret_prefixes or [],
}).encode()
proc = await asyncio.create_subprocess_exec(
"sandbox-runner",
"--tool-code", "-",
"--payload", "-",
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(
proc.communicate(input=payload),
timeout=config.timeout_seconds,
)
except asyncio.TimeoutError:
proc.kill()
raise SandboxTimeoutError(
f"Tool exceeded {config.timeout_seconds}s budget"
)
if proc.returncode != 0:
error = stderr.decode(errors="replace")
if "memory" in error.lower():
raise SandboxTooMuchMemory(error)
if "network" in error.lower():
raise SandboxNetworkBlocked(error)
raise SandboxError(f"Tool exited {proc.returncode}: {error}")
result = json.loads(stdout.decode())
if result.get("output_bytes", 0) > config.max_output_bytes:
raise SandboxError("Output exceeds max_output_bytes")
yield result
Notice what is not in this code: authentication, authorization, logging, or retry logic. Those belong to the orchestration layer. The sandbox only answers one question: did this tool complete within its declared budget, and was the output structurally valid?
The oldest agent architecture is the central orchestrator: one agent reads a prompt, decides which tools to call, calls them, and returns a result. This pattern collapses under two conditions — high concurrency and heterogeneous tool ownership.
Imagine ten teams each maintain a set of tools. No team wants to expose a generic REST endpoint that any other team's agent can call with any payload. They want:
A central orchestrator cannot accommodate this without becoming a bottleneck and a single point of failure.
The replacement is a capability registry — a service that agents publish to and discover from. Each agent registers its capabilities. Other agents query the registry for capabilities matching their needs. When an agent invokes a capability, the registry resolves the target and injects the invocation context.
capabilities:
- agent_id: payments-agent
version: 2.1.0
capabilities:
- name: charge
input: ChargeRequest
output: ChargeResult
error_schemas: [InsufficientFunds, CardDeclined, NetworkTimeout]
rate_limit:
calls_per_minute: 100
burst: 20
trust_policy:
required_roles: [payments-service]
allowed_origins:
- order-agent
- refund-agent
- agent_id: research-agent
version: 1.4.0
capabilities:
- name: search
input: SearchRequest
output: SearchResultList
error_schemas: [RateLimited, QueryTooLong]
rate_limit:
calls_per_minute: 30
burst: 5
trust_policy:
required_roles: [research-service]
allowed_origins: [user-agent]
This file is not a configuration dump. It is a machine-readable contract. The registry enforces it at runtime by checking caller identity, applying rate limits, and short-circuiting invalid invocations before they reach the agent.
A2A messages carry a capability_id
field instead of an address. The registry translates that identifier to the actual invocation target using the trust policy and rate-limit state. Consumers never know — and should not care — which deployment hosts the capability. This decoupling is what allows agents to scale horizontally without fragile routing tables.
The hardest part of building trustworthy agents is controlling state. An agent that mutates shared variables, sends side-channel messages, or silently retries failures will appear correct in tests and fail catastrophically in production.
Every production agent should expose its state as a finite state machine with explicit transitions. Consider a task agent that processes a user request:
┌──────────┐
user_request ──▶ │ PENDING │
└────┬─────┘
│ plan_generated
┌────▼─────┐
│ PLANNING │
└────┬─────┘
│ plan_approved
┌────▼─────┐
┌─────│ EXECUTING│─────┐
│ └────┬─────┘ │
│ │ tool_failed tool_completed
┌─────▼─────┐ ┌─▼────────┐ ┌──▼──────────┐
│ RETRYING │ │COMPLETED │ │FAILED_MAX │
└─────┬─────┘ └──────────┘ └─────────────┘
│
│ retry_budget_exhausted
└───────────────────────▶ FAILED_MAX
This diagram is not decoration. It drives four engineering decisions:
An agent that calls a tool twice with different results is an agent that is lying about its own state. Every tool handler must be idempotent or clearly non-idempotent with compensation logic.
interface ToolHandler {
id: string;
execute(ctx: InvocationContext, params: unknown): Promise<ToolResult>;
/** True if re-invoking with the same params must return the same result */
idempotent: boolean;
/** Optional cancellation hook */
cancel?(ctx: InvocationContext): Promise<void>;
}
When a tool is non-idempotent — writing to a database, posting to an API — the agent must associate a unique invocation ID with each call and check for prior completion before re-invoking after a failure. Without this, retry storms double costs and corrupt state.
Agents are opaque by default. One bad iteration and you cannot tell whether the fault lies in the prompt, the tool, the LLM, or the routing logic. Observability is not an add-on — it is the lens that makes debugging possible.
A production agent system must emit four signals consistently:
Structured invocation logs. Every A2A call logs correlation ID, caller, target, capability, input hash, start timestamp, end timestamp, and outcome. Inputs are hashed for deduplication but never logged in plaintext.
Tool execution traces. For each tool call, record: tool ID, sandbox boundary info, resource usage, network egress, and output size. This lets you detect sandbox escapes and resource abuse.
LLM cost and latency breakdown. Track tokens consumed, latency per model call, and cost per agent. Agents that appear cheap in isolation can be expensive in aggregate when you sum retries, fallback calls, and long context windows.
State transition audit log. Every FSM transition is recorded with the event that triggered it, the actor, and the resulting state. This log is the source of truth for postmortems.
{
"trace_id": "7f3a9b2c-4e1d-4f8b-b5a6-9c8d7e6f5a4b",
"span_id": "01a2b3c4d5e6f7a8",
"timestamp_ms": 1718496234567,
"caller_agent_id": "order-agent",
"target_agent_id": "payments-agent",
"capability_id": "charge",
"capability_version": "2.1.0",
"input_hash": "sha256:e3b0c44298fc1c149afbf4c8996fb924",
"start_ms": 1718496234567,
"end_ms": 1718496234789,
"duration_ms": 222,
"outcome": "success",
"sandbox": {
"cpu_ms": 45,
"memory_peak_mb": 87,
"network_egress_bytes": 1024,
"timeout_budget_ms": 30000,
"timeout_used_ms": 0
},
"llm_usage": {
"model": "claude-sonnet-4",
"input_tokens": 1240,
"output_tokens": 89,
"cache_read_tokens": 320
},
"cost_usd": 0.00142,
"state_transition": {
"from": "EXECUTING",
"to": "EXECUTING",
"event": "tool_completed"
},
"retry_count": 0,
"idempotency_key": "charge-7f3a9b2c-order-88291"
}
This log tells you everything a postmortem needs without requiring access to production memory or replaying conversations.
Architecture patterns for single-agent systems do not transfer cleanly to multi-agent systems. Here are the patterns that actually work.
When a task decomposes into independent subtasks, fan out to multiple agents and aggregate results. The aggregator waits for a quorum or best-response policy:
This pattern reduces latency linearly with agent count and provides natural fault tolerance.
When tasks have sequential dependencies, pipeline agents with explicit backpressure. Each agent holds a bounded queue. When the queue is full, the upstream agent blocks instead of dropping work or spilling to disk. This keeps the system stable under load.
async def run_pipeline(task: Task, agents: list[Agent]) -> Result:
queue = asyncio.Queue(maxsize=100) # bounded
async def producer():
await queue.put(task)
async def consumer(agent: Agent):
while True:
item = await queue.get() # blocks when empty
try:
result = await agent.process(item)
await queue.put(result) # blocks when full
finally:
queue.task_done()
await asyncio.gather(
producer(),
*(consumer(agent) for agent in agents),
)
The maxsize=100
queue is not arbitrary. It is a memory bound that prevents unbounded queue growth when downstream agents slow down.
Agents should declare fallback hierarchies. If the primary agent is unavailable or returns an error, the system tries the next capability version or a simpler agent with fewer features:
primary-agent:1.0 ──failure──▶ primary-agent:1.0-fallback ──failure──▶ read-only-agent:2.3
Each fallback has a documented capability reduction. The orchestrator surfaces this to the caller so the UI can adapt — showing a degraded response instead of a hard error.
Not all invocations deserve the most capable model. Route requests by complexity:
A cost-aware router inspects the invocation context — input size, required reasoning depth, and output structure complexity — and selects the smallest model that meets the SLA. The savings compound across millions of invocations.
Security in agent systems is not about perimeter defense. It is about assuming compromise at every boundary and designing accordingly.
Assume every agent is potentially compromised. Treat every A2A call as if it could be spoofed. Enforce:
Each capability declares the minimum resources it needs. The sandbox enforces this. Capabilities that only read from a database should never be able to write. This is enforced at the capability registration level, not in code.
Treat user input the same way you treat an untrusted network: it is adversarial. Validate and sanitize inputs at the sandbox boundary. Use structural parsing (JSON schema) rather than natural-language filtering. Natural-language filters fail against adversarial prompts; schema validation does not.
Agents must never hold long-lived secrets. Use short-lived tokens issued by a secrets provider. Rotate automatically on each invocation or on a fixed schedule, whichever comes first. Store secrets in a vault, not in environment variables or configuration files.
Prompt engineering metrics — token count, latency per call, success rate per prompt — are necessary but insufficient. Production agent systems need infrastructure metrics:
| Metric | Why It Matters |
|---|---|
| Mean time to detection (MTTD) | How quickly you notice a bad agent iteration |
| Mean time to recovery (MTTR) | How fast you can revert a deployment |
| Sandbox escape rate | Should always be zero; any non-zero value is critical |
| Capability contract drift | Number of callers using deprecated schemas |
| Retry amplification factor | How often retries cascade into additional failures |
| Cost per successful outcome | Not cost per call — cost per useful result |
| State machine consistency errors | Transitions that violate declared rules |
| Agent-to-agent latency p99 | End-to-end routing latency, not just model latency |
Q: Do I need A2A protocols if I only have one agent?
No. A2A pays for itself when you have multiple agents that coordinate, share capabilities, or evolve independently. For a single agent with a fixed toolset, a monolithic orchestrator is simpler and sufficient. Introduce A2A when coordination complexity exceeds what a single orchestrator can manage.
Q: Can I start with a central orchestrator and migrate to A2A later?
Yes, but design your orchestrator to be stateless and your capabilities to be addressable independently. If every tool call is wrapped in a capability contract from day one, migration is a configuration change. If tool calls are hardcoded across the orchestrator, migration is a rewrite.
Q: How do I handle non-determinism if I am building an agent state machine?
Non-determinism lives in the LLM layer, not the agent layer. The LLM returns a probabilistic plan; the agent executes a deterministic state machine based on that plan. Log the plan, validate it against the schema, apply it as a state transition, and treat any deviation as an error. This separation of concerns keeps agents predictable even when their inputs are not.
The transition from prompts to infrastructure is not about abandoning prompt engineering — it is about recognizing that prompts are one component in a larger system. A2A protocols, agent sandboxes, capability registries, and state machine design form the skeleton. Prompts are the nervous system. Build the skeleton well, and the nervous system has something stable to control.