AWS AgentCore's Multi-Account Architecture: How Separate Inference and Tool Gateways Enforce Agent Governance at Scale

AWS published a reference blueprint for AgentCore, a multi-account platform architecture that separates agent inference from tool execution across AWS account boundaries and gates every deployment through a CI/CD evaluation pipeline. The design routes agent requests through a control plane into isolated per-workstream cells, using Organizations service control policies to block lateral movement, and enforces per-cell IAM scoping of Bedrock models, S3 buckets and external API credentials. The evaluation gate blocks deployments that fall below 95% correct tool calls, exceed a 2% invalid-call rate, or produce any unauthorized tool calls.

AWS just published a reference blueprint for AgentCore, a multi-account platform architecture that separates inference from tool execution and gates every agent deployment with CI/CD evaluations. The design addresses the governance gap that appears when you move from a single proof-of-concept agent to a multi-tenant platform serving dozens of workstreams, each with different risk profiles and tool access requirements. The core idea: split reasoning inference from action tool calls across account boundaries, route everything through a control plane, and isolate each workstream in its own cell. This prevents a compromised agent in one workstream from reaching tools or data in another, and it gives platform teams a single point to enforce policy, observability, and cost allocation. Most agent frameworks run inference and tool execution in the same process or account. When an LLM decides to call a tool, the orchestrator invokes it directly. This works for demos but creates three problems at scale: AgentCore splits these concerns across two gateways: The control plane routes agent requests to the correct cell, and the cell's tool gateway enforces per-workstream policies before invoking any external API or database. The control plane is a lightweight routing layer that maps incoming agent requests to the correct workstream cell. It does not execute inference or tools itself. Instead, it: The control plane runs in its own account and uses AWS Organizations service control policies SCPs to prevent cells from calling each other directly. This enforces the hub-and-spoke topology and prevents lateral movement. Each workstream gets its own AWS account containing: The cell account's IAM policies restrict which Bedrock models, S3 buckets, and external APIs the agent can reach. If a workstream needs access to Salesforce, only that cell's tool gateway gets Salesforce credentials. If another workstream needs Snowflake, only its cell gets Snowflake credentials. This design also simplifies cost allocation. Each cell's CloudWatch bill, Bedrock token usage, and Lambda invocations roll up to the workstream's account, so finance teams can charge back accurately without parsing logs. AgentCore gates every agent deployment with a CI/CD pipeline that runs evaluations before promoting code to production. The pipeline has three stages: The evaluation stage is the critical gate. It uses a test harness that: | Metric | Threshold | Failure Action | |---|---|---| | Accuracy | ≥ 95% correct tool calls | Block deployment | | Hallucination rate | ≤ 2% invalid tool calls | Block deployment | | Latency p99 | ≤ 5 seconds | Warn, allow override | | Cost per request | ≤ $0.10 | Warn, allow override | | Policy violations | 0 unauthorized tool calls | Block deployment | The pipeline runs in a dedicated CI/CD account with read-only access to production cells. It cannot modify production state, but it can invoke staging cells to run evaluations. The tool gateway is a Lambda function or Step Functions state machine that: Here's a simplified Python example: python import json import boto3 from aws lambda powertools import Logger, Tracer logger = Logger tracer = Tracer TOOL ALLOWLIST = { "salesforce query": { "handler": "invoke salesforce", "schema": {"type": "object", "properties": {"query": {"type": "string"}}} } } @tracer.capture method def invoke salesforce params : Call Salesforce API with scoped credentials client = boto3.client "secretsmanager" secret = client.get secret value SecretId="salesforce-creds" creds = json.loads secret "SecretString" ... make API call, return result return {"status": "success", "data": ... } @logger.inject lambda context @tracer.capture lambda handler def handler event, context : tool name = event "tool name" params = event "parameters" if tool name not in TOOL ALLOWLIST: logger.error f"Unauthorized tool: {tool name}" return {"error": "Tool not allowed"} tool config = TOOL ALLOWLIST tool name Validate params against schema omitted for brevity handler fn = globals tool config "handler" result = handler fn params logger.info f"Tool call succeeded", extra={"tool": tool name} return result The tool gateway also enforces rate limits using DynamoDB or ElastiCache to track call counts per session or per hour. If a session exceeds the limit, the gateway returns an error and logs a policy violation. Each cell emits structured logs to CloudWatch Logs with a consistent schema: workstream id : Identifies the cell. session id : Correlates requests across inference and tool gateways. tool name : Which tool was called. latency ms : How long the tool call took. error : Any error message or exception. The control plane aggregates these logs into a central CloudWatch Logs Insights query or ships them to a SIEM like Splunk or Datadog. Platform teams can query across all cells to find slow tools, high error rates, or policy violations. X-Ray tracing links inference requests to tool calls, so you can visualize the full agent execution path and identify bottlenecks. If an agent takes 10 seconds to respond, X-Ray shows whether the delay is in Bedrock inference, tool execution, or network hops. A typical AgentCore deployment for a mid-sized enterprise looks like this: The control plane and shared inference account are managed by the platform team. Workstream cells are provisioned via Terraform or CloudFormation StackSets, so adding a new workstream takes minutes. The CI/CD account is locked down with SCPs that prevent it from modifying production state. AgentCore's multi-account design makes sense when: It does not make sense when: AWS AgentCore is a production-grade blueprint for governed agent platforms. The separation of inference from tool execution, per-workstream cell isolation, and CI/CD evaluation gates address the three hardest problems in multi-tenant agent deployments: blast radius, auditability, and cost attribution. The architecture is overkill for single-agent experiments but essential for enterprises moving from pilots to production. The open-source reference implementation gives you a working starting point, but you will need to customize IAM policies, tool schemas, and evaluation metrics for your workstreams. The main trade-off is operational complexity. You are managing N+3 AWS accounts N workstreams plus control plane, inference, and CI/CD . If your team is not already comfortable with AWS Organizations, cross-account IAM, and infrastructure-as-code, the learning curve is steep. But if you are already running multi-account AWS environments, AgentCore slots in cleanly. Use it when you need to scale agents across teams with different trust boundaries. Skip it if you are still figuring out whether agents work for your use case.