{"slug": "from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of", "title": "From Prompts to Infrastructure: Building Trustworthy, Scalable AI Agents in the Age of A2A and Agent Sandboxes", "summary": "A developer outlines the engineering shift from prompt-based AI agents to infrastructure-grade systems, emphasizing the need for standardized Agent-to-Agent (A2A) protocols and sandboxed execution environments to handle scalability, security, and compliance. The article details how typed capability contracts and trust-bound invocation contexts enable agents to cooperate safely without trusting each other's internals.", "body_md": "*Originally published on tamiz.pro.*\n\nThe naive image of an AI agent is a chatbot that gets smarter each turn. The production reality is far more demanding: agents that coordinate across services, respect security boundaries, follow standardized communication contracts, and survive unbounded request volume. The shift from prompts to infrastructure isn't philosophical — it's a hard engineering transition that separates demos from deployed systems.\n\nEarly AI applications leaned on prompt chaining: sequence of LLM calls linked by a human-readable narrative. This works until you need five agents handling 10,000 concurrent requests, each calling multiple downstream tools, with latency budgets measured in seconds and audit trails required for compliance.\n\nPrompt-level systems face three fatal scaling problems:\n\nThe alternative is treating agents as infrastructure — services with explicit contracts, bounded execution contexts, and standardized inter-agent communication.\n\nA2A (Agent-to-Agent) refers to the emerging class of protocols that let agents communicate with each other without a central orchestrator making every decision. Think RPC for agents: structured, typed, versioned, and observable.\n\nWithout A2A, agent ecosystems look like this:\n\nWith A2A, agents speak a common protocol. Each agent publishes a **capability contract** — a machine-readable description of what inputs it accepts, what outputs it produces, and what side effects it may have. Other agents discover and invoke capabilities through a typed interface, not by guessing JSON shapes.\n\nA well-designed A2A system rests on three primitives:\n\n**Cardinality-bounded messages.** Every inter-agent message has a defined lifecycle: sent, acknowledged, completed, or failed. Unlike fire-and-forget HTTP, A2A messages carry sequence numbers and correlation IDs so agents can reconstruct conversation history.\n\n**Typed capability descriptors.** Instead of `POST /agent/process`\n\n, a capability is declared as:\n\n```\n{\n  \"type\": \"tool\",\n  \"name\": \"database_query\",\n  \"version\": \"1.2.0\",\n  \"input_schema\": {\n    \"$schema\": \"https://json-schema.org/draft/2020-12/schema\",\n    \"type\": \"object\",\n    \"required\": [\"query\", \"connection_id\"],\n    \"properties\": {\n      \"query\": { \"type\": \"string\", \"maxLength\": 4096 },\n      \"connection_id\": { \"type\": \"string\", \"pattern\": \"^conn-[a-f0-9]+$\" }\n    }\n  },\n  \"output_schema\": {\n    \"type\": \"object\",\n    \"properties\": {\n      \"rows\": { \"type\": \"array\", \"maxItems\": 1000 },\n      \"truncated\": { \"type\": \"boolean\" }\n    }\n  },\n  \"error_schemas\": [\n    { \"code\": \"QUERy_TOO_LARGE\", \"message\": \"Query exceeds 4096 characters\" },\n    { \"code\": \"CONNECTION_UNAVAILABLE\", \"message\": \"Connection pool exhausted\" }\n  ]\n}\n```\n\nThis schema isn't decorative. It enables compile-time validation at the call site, runtime enforcement at the capability boundary, and automatic test generation for every contract change.\n\n**Trust-bound invocation context.** Every A2A call carries an invocation context that answers: who is calling? what tools may they access? what is the budget? A2A protocols embed this as a signed token or policy bundle rather than a header you trust because you built the client.\n\nA2A and sandboxes are complementary. A2A defines the *contract* — what can be asked and what must be returned. Sandboxes define the *execution boundary* — where and how the agent runs its logic. Together they create a system where agents can cooperate without trusting each other's internals.\n\nAn agent sandbox is a controlled environment where agent code runs with explicit resource limits, network restrictions, and output filtering. The sandbox is not a luxury — it is the mechanism that makes multi-agent systems safe.\n\nConsider what goes wrong when agents run without sandboxes:\n\nEach of these scenarios is addressable at the infrastructure layer rather than hoping the next prompt improvement catches it.\n\nA production-grade agent sandbox implements three layers:\n\n**Compute sandbox.** Isolated process execution with CPU, memory, and timeout budgets. Tools run as subprocesses or WebAssembly modules, not arbitrary Python inside the agent process. Tools that exceed their allocation are killed and reported as errors rather than hanging the orchestrator.\n\n**Network sandbox.** Agents only reach networks they are authorized for. A read-only agent cannot call `POST https://webhook.example.com/keys`\n\n. Egress is enforced by the host, not by the agent's code. Ingress is filtered against allowlists for inbound tool responses.\n\n**Data sandbox.** Secrets, credentials, and PII live outside the agent's execution context. Tools receive tokens, not full credentials. Output streams are scanned for sensitive patterns before leaving the sandbox boundary.\n\nHere is a minimal but production-oriented sandbox wrapper around a tool invocation:\n\n``` python\nimport asyncio\nimport json\nimport time\nfrom contextlib import asynccontextmanager\nfrom dataclasses import dataclass\nfrom typing import Any, Optional\n\n@dataclass\nclass SandboxConfig:\n    max_memory_mb: int = 256\n    timeout_seconds: float = 30.0\n    allowed_networks: list[str] | None = None\n    max_output_bytes: int = 65536\n    secret_prefixes: list[str] | None = None\n\nclass SandboxError(Exception):\n    pass\n\nclass SandboxTooMuchMemory(SandboxError):\n    pass\n\nclass SandboxTimeoutError(SandboxError):\n    pass\n\nclass SandboxNetworkBlocked(SandboxError):\n    pass\n\n@asynccontextmanager\nasync def run_tool_in_sandbox(\n    tool_code: str,\n    arguments: dict[str, Any],\n    config: SandboxConfig,\n):\n    start = time.monotonic()\n\n    # 1. Encode the call as a self-contained script payload\n    payload = json.dumps({\n        \"args\": arguments,\n        \"limits\": {\n            \"memory_mb\": config.max_memory_mb,\n            \"timeout_s\": config.timeout_seconds,\n            \"max_output_bytes\": config.max_output_bytes,\n        },\n        \"allowed_networks\": config.allowed_networks,\n        \"secret_prefixes\": config.secret_prefixes or [],\n    }).encode()\n\n    # 2. Spawn a sandboxed subprocess. In practice this would use\n    # gVisor, Firecracker, or WASI — here we show the contract.\n    proc = await asyncio.create_subprocess_exec(\n        \"sandbox-runner\",\n        \"--tool-code\", \"-\",\n        \"--payload\", \"-\",\n        stdin=asyncio.subprocess.PIPE,\n        stdout=asyncio.subprocess.PIPE,\n        stderr=asyncio.subprocess.PIPE,\n    )\n\n    try:\n        stdout, stderr = await asyncio.wait_for(\n            proc.communicate(input=payload),\n            timeout=config.timeout_seconds,\n        )\n    except asyncio.TimeoutError:\n        proc.kill()\n        raise SandboxTimeoutError(\n            f\"Tool exceeded {config.timeout_seconds}s budget\"\n        )\n\n    if proc.returncode != 0:\n        error = stderr.decode(errors=\"replace\")\n        if \"memory\" in error.lower():\n            raise SandboxTooMuchMemory(error)\n        if \"network\" in error.lower():\n            raise SandboxNetworkBlocked(error)\n        raise SandboxError(f\"Tool exited {proc.returncode}: {error}\")\n\n    result = json.loads(stdout.decode())\n\n    # 3. Post-execution output filtering\n    if result.get(\"output_bytes\", 0) > config.max_output_bytes:\n        raise SandboxError(\"Output exceeds max_output_bytes\")\n\n    yield result\n```\n\nNotice what is *not* in this code: authentication, authorization, logging, or retry logic. Those belong to the orchestration layer. The sandbox only answers one question: did this tool complete within its declared budget, and was the output structurally valid?\n\nThe oldest agent architecture is the central orchestrator: one agent reads a prompt, decides which tools to call, calls them, and returns a result. This pattern collapses under two conditions — high concurrency and heterogeneous tool ownership.\n\nImagine ten teams each maintain a set of tools. No team wants to expose a generic REST endpoint that any other team's agent can call with any payload. They want:\n\nA central orchestrator cannot accommodate this without becoming a bottleneck and a single point of failure.\n\nThe replacement is a **capability registry** — a service that agents publish to and discover from. Each agent registers its capabilities. Other agents query the registry for capabilities matching their needs. When an agent invokes a capability, the registry resolves the target and injects the invocation context.\n\n```\n# capability-registry.example.yaml\ncapabilities:\n  - agent_id: payments-agent\n    version: 2.1.0\n    capabilities:\n      - name: charge\n        input: ChargeRequest\n        output: ChargeResult\n        error_schemas: [InsufficientFunds, CardDeclined, NetworkTimeout]\n        rate_limit:\n          calls_per_minute: 100\n          burst: 20\n        trust_policy:\n          required_roles: [payments-service]\n          allowed_origins:\n            - order-agent\n            - refund-agent\n\n  - agent_id: research-agent\n    version: 1.4.0\n    capabilities:\n      - name: search\n        input: SearchRequest\n        output: SearchResultList\n        error_schemas: [RateLimited, QueryTooLong]\n        rate_limit:\n          calls_per_minute: 30\n          burst: 5\n        trust_policy:\n          required_roles: [research-service]\n          allowed_origins: [user-agent]\n```\n\nThis file is not a configuration dump. It is a machine-readable contract. The registry enforces it at runtime by checking caller identity, applying rate limits, and short-circuiting invalid invocations before they reach the agent.\n\nA2A messages carry a `capability_id`\n\nfield instead of an address. The registry translates that identifier to the actual invocation target using the trust policy and rate-limit state. Consumers never know — and should not care — which deployment hosts the capability. This decoupling is what allows agents to scale horizontally without fragile routing tables.\n\nThe hardest part of building trustworthy agents is controlling state. An agent that mutates shared variables, sends side-channel messages, or silently retries failures will appear correct in tests and fail catastrophically in production.\n\nEvery production agent should expose its state as a finite state machine with explicit transitions. Consider a task agent that processes a user request:\n\n```\n                    ┌──────────┐\n   user_request ──▶ │  PENDING │\n                    └────┬─────┘\n                         │ plan_generated\n                    ┌────▼─────┐\n                    │  PLANNING │\n                    └────┬─────┘\n                         │ plan_approved\n                    ┌────▼─────┐\n              ┌─────│  EXECUTING│─────┐\n              │     └────┬─────┘     │\n              │          │ tool_failed      tool_completed\n        ┌─────▼─────┐  ┌─▼────────┐  ┌──▼──────────┐\n        │ RETRYING  │  │COMPLETED │  │FAILED_MAX   │\n        └─────┬─────┘  └──────────┘  └─────────────┘\n              │\n              │ retry_budget_exhausted\n              └───────────────────────▶ FAILED_MAX\n```\n\nThis diagram is not decoration. It drives four engineering decisions:\n\nAn agent that calls a tool twice with different results is an agent that is lying about its own state. Every tool handler must be idempotent or clearly non-idempotent with compensation logic.\n\n```\ninterface ToolHandler {\n  id: string;\n  execute(ctx: InvocationContext, params: unknown): Promise<ToolResult>;\n  /** True if re-invoking with the same params must return the same result */\n  idempotent: boolean;\n  /** Optional cancellation hook */\n  cancel?(ctx: InvocationContext): Promise<void>;\n}\n```\n\nWhen a tool is non-idempotent — writing to a database, posting to an API — the agent must associate a unique invocation ID with each call and check for prior completion before re-invoking after a failure. Without this, retry storms double costs and corrupt state.\n\nAgents are opaque by default. One bad iteration and you cannot tell whether the fault lies in the prompt, the tool, the LLM, or the routing logic. Observability is not an add-on — it is the lens that makes debugging possible.\n\nA production agent system must emit four signals consistently:\n\n**Structured invocation logs.** Every A2A call logs correlation ID, caller, target, capability, input hash, start timestamp, end timestamp, and outcome. Inputs are hashed for deduplication but never logged in plaintext.\n\n**Tool execution traces.** For each tool call, record: tool ID, sandbox boundary info, resource usage, network egress, and output size. This lets you detect sandbox escapes and resource abuse.\n\n**LLM cost and latency breakdown.** Track tokens consumed, latency per model call, and cost per agent. Agents that appear cheap in isolation can be expensive in aggregate when you sum retries, fallback calls, and long context windows.\n\n**State transition audit log.** Every FSM transition is recorded with the event that triggered it, the actor, and the resulting state. This log is the source of truth for postmortems.\n\n```\n{\n  \"trace_id\": \"7f3a9b2c-4e1d-4f8b-b5a6-9c8d7e6f5a4b\",\n  \"span_id\": \"01a2b3c4d5e6f7a8\",\n  \"timestamp_ms\": 1718496234567,\n  \"caller_agent_id\": \"order-agent\",\n  \"target_agent_id\": \"payments-agent\",\n  \"capability_id\": \"charge\",\n  \"capability_version\": \"2.1.0\",\n  \"input_hash\": \"sha256:e3b0c44298fc1c149afbf4c8996fb924\",\n  \"start_ms\": 1718496234567,\n  \"end_ms\": 1718496234789,\n  \"duration_ms\": 222,\n  \"outcome\": \"success\",\n  \"sandbox\": {\n    \"cpu_ms\": 45,\n    \"memory_peak_mb\": 87,\n    \"network_egress_bytes\": 1024,\n    \"timeout_budget_ms\": 30000,\n    \"timeout_used_ms\": 0\n  },\n  \"llm_usage\": {\n    \"model\": \"claude-sonnet-4\",\n    \"input_tokens\": 1240,\n    \"output_tokens\": 89,\n    \"cache_read_tokens\": 320\n  },\n  \"cost_usd\": 0.00142,\n  \"state_transition\": {\n    \"from\": \"EXECUTING\",\n    \"to\": \"EXECUTING\",\n    \"event\": \"tool_completed\"\n  },\n  \"retry_count\": 0,\n  \"idempotency_key\": \"charge-7f3a9b2c-order-88291\"\n}\n```\n\nThis log tells you everything a postmortem needs without requiring access to production memory or replaying conversations.\n\nArchitecture patterns for single-agent systems do not transfer cleanly to multi-agent systems. Here are the patterns that actually work.\n\nWhen a task decomposes into independent subtasks, fan out to multiple agents and aggregate results. The aggregator waits for a quorum or best-response policy:\n\nThis pattern reduces latency linearly with agent count and provides natural fault tolerance.\n\nWhen tasks have sequential dependencies, pipeline agents with explicit backpressure. Each agent holds a bounded queue. When the queue is full, the upstream agent blocks instead of dropping work or spilling to disk. This keeps the system stable under load.\n\n``` python\n# High-level backpressure-controlled pipeline\nasync def run_pipeline(task: Task, agents: list[Agent]) -> Result:\n    queue = asyncio.Queue(maxsize=100)  # bounded\n\n    async def producer():\n        await queue.put(task)\n\n    async def consumer(agent: Agent):\n        while True:\n            item = await queue.get()  # blocks when empty\n            try:\n                result = await agent.process(item)\n                await queue.put(result)  # blocks when full\n            finally:\n                queue.task_done()\n\n    await asyncio.gather(\n        producer(),\n        *(consumer(agent) for agent in agents),\n    )\n```\n\nThe `maxsize=100`\n\nqueue is not arbitrary. It is a memory bound that prevents unbounded queue growth when downstream agents slow down.\n\nAgents should declare fallback hierarchies. If the primary agent is unavailable or returns an error, the system tries the next capability version or a simpler agent with fewer features:\n\n```\nprimary-agent:1.0 ──failure──▶ primary-agent:1.0-fallback ──failure──▶ read-only-agent:2.3\n```\n\nEach fallback has a documented capability reduction. The orchestrator surfaces this to the caller so the UI can adapt — showing a degraded response instead of a hard error.\n\nNot all invocations deserve the most capable model. Route requests by complexity:\n\nA cost-aware router inspects the invocation context — input size, required reasoning depth, and output structure complexity — and selects the smallest model that meets the SLA. The savings compound across millions of invocations.\n\nSecurity in agent systems is not about perimeter defense. It is about assuming compromise at every boundary and designing accordingly.\n\nAssume every agent is potentially compromised. Treat every A2A call as if it could be spoofed. Enforce:\n\nEach capability declares the minimum resources it needs. The sandbox enforces this. Capabilities that only read from a database should never be able to write. This is enforced at the capability registration level, not in code.\n\nTreat user input the same way you treat an untrusted network: it is adversarial. Validate and sanitize inputs at the sandbox boundary. Use structural parsing (JSON schema) rather than natural-language filtering. Natural-language filters fail against adversarial prompts; schema validation does not.\n\nAgents must never hold long-lived secrets. Use short-lived tokens issued by a secrets provider. Rotate automatically on each invocation or on a fixed schedule, whichever comes first. Store secrets in a vault, not in environment variables or configuration files.\n\nPrompt engineering metrics — token count, latency per call, success rate per prompt — are necessary but insufficient. Production agent systems need infrastructure metrics:\n\n| Metric | Why It Matters |\n|---|---|\n| Mean time to detection (MTTD) | How quickly you notice a bad agent iteration |\n| Mean time to recovery (MTTR) | How fast you can revert a deployment |\n| Sandbox escape rate | Should always be zero; any non-zero value is critical |\n| Capability contract drift | Number of callers using deprecated schemas |\n| Retry amplification factor | How often retries cascade into additional failures |\n| Cost per successful outcome | Not cost per call — cost per useful result |\n| State machine consistency errors | Transitions that violate declared rules |\n| Agent-to-agent latency p99 | End-to-end routing latency, not just model latency |\n\n**Q: Do I need A2A protocols if I only have one agent?**\n\nNo. A2A pays for itself when you have multiple agents that coordinate, share capabilities, or evolve independently. For a single agent with a fixed toolset, a monolithic orchestrator is simpler and sufficient. Introduce A2A when coordination complexity exceeds what a single orchestrator can manage.\n\n**Q: Can I start with a central orchestrator and migrate to A2A later?**\n\nYes, but design your orchestrator to be stateless and your capabilities to be addressable independently. If every tool call is wrapped in a capability contract from day one, migration is a configuration change. If tool calls are hardcoded across the orchestrator, migration is a rewrite.\n\n**Q: How do I handle non-determinism if I am building an agent state machine?**\n\nNon-determinism lives in the LLM layer, not the agent layer. The LLM returns a probabilistic plan; the agent executes a deterministic state machine based on that plan. Log the plan, validate it against the schema, apply it as a state transition, and treat any deviation as an error. This separation of concerns keeps agents predictable even when their inputs are not.\n\nThe transition from prompts to infrastructure is not about abandoning prompt engineering — it is about recognizing that prompts are one component in a larger system. A2A protocols, agent sandboxes, capability registries, and state machine design form the skeleton. Prompts are the nervous system. Build the skeleton well, and the nervous system has something stable to control.", "url": "https://wpnews.pro/news/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of", "canonical_source": "https://dev.to/tamizuddin/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-age-of-a2a-and-11md", "published_at": "2026-08-13 18:00:57+00:00", "updated_at": "2026-08-13 18:19:12.256082+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-safety", "ai-policy", "developer-tools"], "entities": ["tamiz.pro"], "alternates": {"html": "https://wpnews.pro/news/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of", "markdown": "https://wpnews.pro/news/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of.md", "text": "https://wpnews.pro/news/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of.txt", "jsonld": "https://wpnews.pro/news/from-prompts-to-infrastructure-building-trustworthy-scalable-ai-agents-in-the-of.jsonld"}}