Artificial Intelligence Independent Software Vendors (ISVs) are expanding their offerings and adding AI agents. Common considerations for organizations when adding agentic workloads include security, scalability, time to market, and cost tracking. ISVs have another dimension, they provide services to other organizations and need to manage agentic workloads for each customer. As such, ISVs need to manage those common challenges not only broadly but at the tenant level.
Axonius is the asset intelligence platform that helps Security and IT teams prioritize risks and coordinate fixes. By reconciling data from over 1,400 systems into one authoritative source of truth, Axonius enables security teams and the teams they support to collaborate effectively, reducing the manual burden of security, audit, and compliance by up to 50%. Axonius runs its software as a service (SaaS) infrastructure on AWS, managing hundreds of isolated customer environments.
Amazon Bedrock AgentCore is a platform to build, connect, and optimize agents at scale, with any framework or model. In this post, we cover strategies available for SaaS providers to deploy AI agents, describe how Amazon Bedrock AgentCore supports those options, look into Axonius’s considerations, and share Axonius’s chosen architecture. We also describe how Axonius integrated agentic workloads with its existing methodology. This post is for platform engineers and architects building secure, multi-tenant AI agent deployments on AWS.
Multi-tenancy patterns for AI agents #
Independent Software Vendors (ISVs) tend to provide services using the SaaS model. When choosing an architecture, ISVs have three common architectural patterns: silo, bridge, and pool. The following sections elaborate on what each model means when we look into agentic AI with Amazon Bedrock AgentCore.
The silo model refers to an architecture where tenants are provided dedicated resources. Looking into AgentCore runtime, it means using a dedicated agent per tenant. With the pool model, tenants share resources, one agent serves multiple tenants. With AgentCore runtim, you can isolate each user session by allocating a unique session ID for each user session. The third pattern is the bridge model in which some components are in silo mode while others are in pool mode. For example, we can have a dedicated agent deployed into AgentCore runtime using shared Amazon Bedrock Knowledge Bases, an AWS managed Retrieval Augmented Generation (RAG) service.
The challenge #
Axonius was looking to add AI agents to their offering. Their first AI agent interprets the state of large enterprise environments and identifies gaps and risks, using AI to analyze and make sense of millions of data points from dozens of concurrent integration sources. This initiative allows junior analysts to run complex analyses without tying up senior analysts in hours of manual work. Axonius opted to maintain their current tenant management methodology.
Axonius’s SaaS deployment model is silo. Each customer workload resides in a dedicated Amazon Virtual Private Cloud (Amazon VPC). The VPC contains an Application Load Balancer (ALB), Network Load Balancer (NLB), databases, and general compute infrastructure.
Axonius wanted to introduce AI agents while keeping a silo deployment model. Axonius needed to address several key requirements:
Tenant isolation – Axonius handles sensitive customer data. An agent serving one customer must be scoped exclusively to that customer’s data.Identity – The existing service has an authentication and authorization module in place, residing on the Amazon Elastic Compute Cloud (Amazon EC2) allocated to the tenant. Axonius needed to integrate the agent’s identity flow with the existing module without disruption.Cost tracking – Agentic costs can spiral. Most of the cost is in model invocation, thus it’s critical to track model cost per tenant. Knowing your cost is essential when considering pricing options for agentic offerings.Integration with the service– The agent instance associated with a tenant needs to have secure access to that tenant workload’s APIs.** Lifecycle management**– Adding the agentic workload to the current silo Continuous Delivery (CD) workflow.** Observability –**With a silo model, Axonius will have a large number of agents. The DevOps team needed high quality, straightforward to integrate observability that can track large fleets of agents, issue alarms when things go wrong, and provide tracing capabilities to debug issues.
The following sections cover the options Axonius evaluated and what they chose.
Possible solutions #
Looking into tenant deployment modes, Axonius considered the following options.
Option 1: Pool — shared runtime with JWT-based tenant routing
AgentCore runtime enforces structural separation by allocating a dedicated microVM for each session. A single runtime serves tenants while maintaining full tenant separation through application-level controls.
How it works
Tenants authenticate through an OAuth 2.0 identity provider (for example, Amazon Cognito) with JWTs carrying a unique tenant claim (for example, custom:tenant_id). The runtime’s built-in JWT authorizer uses the configured discovery URL to fetch public keys and validate the token’s issuer. The agent code then reads the claim to route tool calls and data access to the correct tenant environment.
Advantages
Operational simplicity: Only one runtime to deploy, monitor, and update.** Rapid onboarding**: New customers onboard immediately without infrastructure provisioning, reducing time-to-value.
Disadvantages
Application-dependent isolation: Tenant separation relies entirely on application code.** Homogeneous deployment**: Per-tenant customization requires additional conditional logic.
Option 2: Bridge — shared runtime plus gateway-enforced tool isolation
A hybrid approach combines the operational simplicity of a single shared Runtime with infrastructure-level tenant enforcement at the tool layer through AgentCore Gateway. Tenants share one Runtime, but every outbound tool call passes through a Gateway that enforces tenant boundaries before the tool code executes.
How it works
Tenants connect to the same AgentCore runtime, which uses dedicated microVMs per session for compute isolation, identical to Option 1. The difference is that every tool invocation routes through a shared AgentCore Gateway with two enforcement mechanisms:
Policy in Amazon Bedrock AgentCore– Deterministic access control. Cedar rules evaluate each tool call against the caller’s identity attributes (for example, Cognito group claims) and produce an allow/deny decision. A forbid rule can block specific tenant groups from invoking restricted tools.AWS Lambda Interceptor (REQUEST)– Dynamic validation and context enrichment. Runs before the tool call reaches the target extracts the JWT, looks up tenant context, and exchanges the token for short-lived, tenant-scoped IAM credentials through STS AssumeRole (“act-on-behalf” pattern). The downstream tool receives these least-privilege credentials instead of the original JWT.
The Gateway evaluates the interceptor before the Cedar policy, enabling the interceptor to enrich the request context that the policy then evaluates.
A RESPONSE interceptor can additionally filter tool discovery based on tenant identity.
This means two independent mechanisms enforce tenant isolation at the infrastructure layer, outside agent code.
Advantages
Layered infrastructure enforcement: Even if the agent code has a routing bug, the Gateway blocks cross-tenant tool calls. Cedar policy (deterministic) and a Lambda interceptor (dynamic) provide defense in depth.Centralized governance: A single Gateway to audit and enforce tenant policies across all tool invocations, with every decision logged to Amazon CloudWatch.Shared efficiency: Runtime, knowledge bases, and observability infrastructure are managed centrally without sharing security risk.
Disadvantages
Setup complexity: Requires configuring a Gateway, REQUEST/RESPONSE interceptor Lambdas, tenant mappings, Cedar policies, and STS role trust relationships. This adds significantly more moving parts than Option 1 or 3.VPC connectivity: Backend tools typically reside in a VPC, requiring VPC endpoints for private Gateway connectivity, which adds networking complexity and potential points of failure.
Option 3: Silo — dedicated runtime per tenant
In the silo model, every tenant operates on a dedicated AgentCore runtime. Access control is enforced entirely through AWS Identity and Access Management (IAM). Resource-based policies on both the Runtime and its endpoint determine which principals, whether same-account roles or cross-account identities, are allowed to invoke the agent. Because each tenant’s workload runs on separate infrastructure, there is no shared compute between tenants.
How it works
Each tenant gets a dedicated AgentCore runtime, provisioned automatically (for example, with AWS CloudFormation or CDK) when a new customer is onboarded. Within that runtime, each user session runs in its own isolated microVM, so no process state is shared even between users of the same tenant. Access control is enforced through IAM resource-based policies attached to both the AgentCore runtime and its endpoint. The caller must hold explicit permission for their tenant’s specific runtime.
Advantages
Maximum isolation: Dedicated compute per tenant with no shared process state. Each session runs in its own microVM, and tenants cannot reach each other’s infrastructure.Simple authorization model: IAM resource-based policies, applied to both the Runtime and its endpoint, are the sole enforcement point. No application-level routing logic required.Independent configuration: Each runtime can run a different agent version, model, or endpoint configuration without affecting other tenants.
Disadvantages
Scale limitations: The default quota is 1,000 agents per AWS account (adjustable through Service Quotas), which requires capacity planning for large customer bases.Provisioning latency: Each new tenant requires creating a dedicated Runtime and endpoint, introducing onboarding delays compared to a shared-runtime model.Operational overhead: Monitoring, updating, and managing hundreds of runtimes adds operational complexity that demands robust automation (for example, CDK/CloudFormation pipelines, centralized observability).
Solution overview #
Axonius operates today in a siloed mode and chooses to continue with that approach when adding agents. Each customer has a dedicated agent. Axonius designed a multi-tenant agentic AI architecture using Amazon Bedrock and AgentCore with the following key components:
AgentCore runtime– Axonius deploys a dedicated agent for each customer, and each user session runs on an isolated microVM, using AgentCore runtime.Amazon Elastic Container Registry (Amazon ECR)– Stores per-tenant agent container images.** Amazon Bedrock**– Powers the underlying foundation models (FMs), with IAM role tagging for cost allocation.** Amazon Bedrock Knowledge Bases (KB)**– Axonius uses Amazon Bedrock Knowledge Bases (KB) with Amazon S3 Vectors because of their cost efficiency and ability to scale. Axonius uses metadata filtering to isolate tenant-specific data.Amazon Bedrock Guardrails– Provides content filtering and topic denial policies. Guardrails are applied to every model response, keeping responses safe and in scope before they are shared with the user.Amazon CloudWatch– Monitors the overall workload and is instrumental in cost control. It tracks token consumption metrics (input/output), issuing alerts and using IAM-deny enforcement for cost governance. If a customer exceeds their token budget, an automated IAM policy blocks further invocations.Amazon VPC Lattice– Allows Axonius to manage cost-efficient private connectivity, connecting AgentCore runtime, customer VPCs, and AWS service endpoints.
The architecture uses AWS CloudFormation for automated per-customer provisioning and teardown, allowing Axonius to scale agent deployments across their customer base.
Why Amazon Bedrock AgentCore runtime? #
After evaluating multiple approaches, including running agents as additional containers within their existing EC2 instances, Axonius chose Amazon Bedrock AgentCore runtime. Its purpose-built capabilities directly addressed their multi-tenant SaaS requirements.
Session isolation with microVMs– AgentCore runtime’s session isolation was the decisive factor. Each user session runs in adedicated microVMwith isolated CPU, memory, and filesystem resources. After session completion, the entire microVM is terminated and memory is sanitized. This deterministic security model was critical for Axonius, where each customer’s data, including sensitive cybersecurity asset inventories, must remain completely isolated.Framework flexibility and VPC integration– AgentCore runtime’s framework-agnostic design allowed Axonius to deploy agents using their preferred tooling while connecting securely to their existing VPC infrastructure. Each agent connects through an ENI to the customer’s VPC, allowing direct interaction with the Axonius application APIs running inside the isolated EC2 instances.Built-in observability– The built-in observability capabilities gave Axonius the visibility they needed without building custom monitoring infrastructure. These capabilities include CloudWatch integration for logging, AWS X-Ray for distributed tracing, and agent-specific tracing that captures reasoning steps and tool invocations.
User session flow #
Each customer runs their own Axonius deployment on a dedicated workload inside their own VPC. Every customer also gets a dedicated AgentCore runtime: a per-customer agent that reasons with Claude, pulls product knowledge from an Amazon Bedrock Knowledge Base, and reaches back into that customer’s Axonius API to answer data questions. Customers have a chat box that allows them to write questions. The following flow traces a customer question, for example, “ Are there any major changes in my asset count compared to last week?”
Authentication: The user is already signed in to Axonius on their own instance. When they send a chat message, the Axonius application authenticates the request and mints a short-lived impersonation JWT carrying the user’s identity, tenant ID, session ID, and actor ID. This token, rather than a static credential, authorizes everything downstream.Runtime invocation: The Axonius application control plane assembles the invocation payload and calls InvokeAgentRuntime on the customer’s dedicated AgentCore runtime. The payload carries only tenant configuration: the AgentCore Memory ID, Knowledge Base and data-source IDs, AWS Region, current Axonius version, and the callback address for the customer’s own instance. The JWT rides in a custom AgentCore header rather than in the body, so authentication material does not land in the agent’s saved state, and the session and actor IDs scope the call to one user’s conversation.Dedicated, isolated runtime: Each customer has its own AgentCore runtime, and that runtime is attached to a dedicated elastic network interface (ENI) placed inside the customer’s own VPC and subnet. AgentCore creates an isolated session, validates the JWT back against the customer’s Axonius application (signature, expiry, revocation, and permissions), and scopes the reasoning to that single tenant. An invalid token is rejected before the model runs.Agent reasoning with Claude: The team’s existing LangGraph supervisor inspects the question and routes it to the right specialist agent. The specialist uses Amazon Bedrock to analyze the question, deciding at each step whether it can respond directly, needs product documentation, or needs live data from the customer’s environment. For model availability by Region, refer toSupported models by AWS Regionin Amazon Bedrock.Knowledge retrieval: When the question is about how Axonius works, the agent retrieves relevant passages from a Bedrock Knowledge Base. That knowledge base is kept current by up the latest Axonius documentation to Amazon Simple Storage Service (Amazon S3) and syncing it, so responses reflect the customer’s running version.Tool access into the customer’s Axonius API: When the question needs live data, such as “Give me a high-level summary of all assets discovered in the last 24 hours,” the agent invokes its query tool, which:- Uses the Knowledge Base and large language model (LLM) to translate the natural-language question into an Axonius Query Language (AQL) expression.
- Reaches the customer’s Axonius application instance through the runtime’s dedicated ENI, so the traffic stays inside the customer’s VPC and doesn’t leave the AWS network. It compiles the query and fetches the matching assets, carrying the same JWT, so the agent can only ever see data the user is already allowed to see.
- Returns the results to the agent, which reasons over them to compose the response.
Guardrails and response: Amazon Bedrock Guardrails are applied server-side to every model response, keeping responses safe and on scope before they leave Amazon Bedrock. The agent’s answer streams back token by token through the runtime to the chat UI, and the exchange is persisted in AgentCore Memory so follow-up questions keep context.
Tenant isolation and private networking #
The whole design rests on strict per-customer isolation and private connectivity:
Dedicated runtime per customer: Every customer gets its own AgentCore runtime with its own ENI inside their VPC and subnet, so one customer’s agent has no network path to another customer’s data.In-network access to customer data: Because the ENI lives in the customer’s VPC, the runtime reaches the internal Axonius EC2 instance over private addressing. That traffic doesn’t traverse the public internet and does not leave the AWS network.Shared AWS services over Amazon VPC Lattice: A dedicated Axonius services VPC publishes the AWS services the runtime depends on (Amazon ECR, Amazon S3, Amazon Bedrock, and others) through a set of private endpoints, shared with each customer VPC through VPC Lattice. This lets the AgentCore runtime pull the agent container image from Amazon ECR and reach Amazon Bedrock and the Knowledge Base privately, while restricting access so that only the customer VPC and its runtime can reach those services.Networking cost efficiency: Amazon VPC Lattice allows Axonius to define one private endpoint per AWS service, reducing cost and management overhead. With AWS PrivateLink, Axonius would have needed to define a private endpoint per VPC per service.
Together, this keeps a clean trust boundary: the reasoning, knowledge retrieval, and guardrails run on Amazon Bedrock. The authoritative data and identity stay on each customer’s own VPC. And every hop between them travels private AWS networking. The agent does not hold long-lived credentials. It borrows the user’s JWT for the life of a request, and every session, memory, tool call, and network path is scoped to a single tenant.
Token governance and cost management #
A notable aspect of Axonius’s implementation is their approach to token governance:
CloudWatch metrics track sum of input and output tokens per agent. In addition, Axonius uses opentelemetry-instrument to get real-time token usage.IAM role tagging(using Amazon Bedrock cost allocation by IAM user/role feature) allows per-tenant cost attribution.** Automated IAM-deny policiescan be triggered by using CloudWatch alarms to block runaway agent usage. Application Inference Profiles**per model allow granular tagging, alerting, and cost control through Amazon EventBridge.
This approach provides cost allocation visibility updated once or twice daily, with real-time alerting through CloudWatch for immediate enforcement.
Conclusion #
ISVs are in the process of developing agentic AI workloads. This post describes possible avenues and Axonius’s chosen solution, using Amazon Bedrock and AgentCore as the foundation for its agentic AI offering.
By using the AgentCore managed runtime, Knowledge Bases for RAG, and built-in session memory, Axonius reduced the development cycle for their multi-tenant AI agent from an estimated eight weeks of custom infrastructure work to only 10 days of production-ready deployment, a 75 percent reduction in time-to-market.
“AgentCore Runtime gave us the multi-tenant isolation and authentication framework we needed to deploy AI agents across large numbers of customer environments without compromising our security-first architecture.”
— Dori Shmuel, SaaS DevOps Lead, Axonius
To learn more about building multi-tenant AI agents with Amazon Bedrock AgentCore runtime, we recommend you review these additional resources:
Amazon Bedrock AgentCore runtime documentationGetting Started with Amazon Bedrock AgentCore WorkshopAgentCore Observability Configuration