Artificial Intelligence #
Cost management for generative AI is a shared responsibility between the teams that build applications and the teams accountable for cloud spend. Many teams start by comparing per-model token rates and asking application teams to shorten prompts. Those steps help, but they don’t answer the questions that finance and engineering leaders ask as usage scales: which team spent this, on which workload, and was that spend inside an approved budget?
Agentic workloads make those questions more pressing. A conversational assistant typically issues one model invocation per user turn. An agentic workflow, such as an AI coding assistant that plans a change, reads files, calls tools, and iterates, issues many invocations per task and carries a larger context on each one. Token consumption therefore scales with the degree of autonomy you grant the workflow and the size of the context it carries, rather than with the number of user interactions. Because consumption is a function of how a workload is designed and operated, cost management for generative AI is an architectural and operational discipline. Published averages are a poor substitute for your own numbers: use the AWS Pricing Calculator and the Amazon Bedrock pricing page to model rates, and use Step 2 of this post to measure what your workloads actually consume.
The fix isn’t a cheaper model or a smaller prompt. It’s an operating model: a repeatable loop that attributes every dollar to an owner, makes every request observable, alerts on and enforces budgets before overspend compounds, optimizes the platform defaults that drive most waste, and reviews the whole system on a cadence.
In this post, you learn how to manage Amazon Bedrock cost in five steps. You attribute every dollar to an owner, make every request observable, control spend with budgets and enforcement, optimize the platform defaults that drive most avoidable spend, and operate a review cadence that turns data into decisions. This approach is built from AWS building blocks, and you can customize it and apply it to your own environment. The post is written for two audiences: FinOps practitioners who own attribution, chargeback, budgets, and anomaly review, and platform engineering teams who own logging, request metadata, enforcement, and routing defaults.
The operating model in one view #
Five stages, run as a loop. Attribute: every Amazon Bedrock dollar lands on an identity, team, and application in the billing pipeline. Observe: every request is logged with enough metadata to explain what generated the spend. Enforce: budgets have teeth, with alerts at minimum and hard blocks where the risk demands them. Optimize: the platform defaults (model routing, caching, token limits) remove waste centrally instead of asking every team to be careful. Operate: a small scorecard and a review cadence turn the data into decisions. FinOps leads stages one and five. Platform engineering leads two through four. The review is joint. The stages are sequenced deliberately. Each one depends on the ones before it, and teams that jump straight to optimization without attribution and observability are guessing.
Stage 1: Attribute every dollar #
You can’t manage what you can’t attribute, and everything downstream (chargeback, budgets, enforcement, anomaly detection) depends on this stage. The foundation is the granular cost attribution of Amazon Bedrock (see Introducing granular cost attribution for Amazon Bedrock): Amazon Bedrock automatically attributes inference costs to the AWS Identity and Access Management (IAM) principal that made the call, across all models, with no resources to manage. The caller’s ARN lands in the line_item_iam_principal column of the AWS Cost and Usage Report (CUR) 2.0, and coverage spans IAM users, roles, federated identities, and API keys.
Setup is four steps:
- Turn on IAM principal data in your CUR 2.0 data export.
- Tag your IAM principals with business dimensions such as team, application, and cost center. For federated access, configure your identity provider to pass session names and tags through OIDC or SAML claims.
- Activate those tags as cost allocation tags in AWS Billing (console or the
UpdateCostAllocationTagsStatusAPI). - Analyze the tags: they surface in CUR 2.0 with an
iamPrincipal/prefix (for example,iamPrincipal/team) and appear inAWS Cost Explorer under the IAM category within 24–48 hours of activation, ready for filtering and grouping by team, application, or cost center.
This is invoice-grade, per-identity attribution with no custom infrastructure.
The shared-role problem
The pattern that deserves the most attention is the shared-role problem, because it is where most platforms silently lose attribution. A large language model (LLM) gateway that calls Amazon Bedrock through a single IAM role attributes every dollar to that one role, eliminating per-user visibility entirely. The fix is per-user session management. The gateway assumes the Amazon Bedrock invocation role once per user, passing the user’s identity as the role session name and their attributes as session tags, then caches those credentials for up to an hour (one AWS Security Token Service (AWS STS) call per user per hour, not per request).
This needs two roles: a gateway execution role holding sts:AssumeRole and sts:TagSession permissions, and the Amazon Bedrock invocation role it assumes. The result is line_item_iam_principal entries like assumed-role/BedrockRuntimeRole/gw-alex, each carrying its user’s tags: per-user and per-tenant attribution restored behind a shared gateway.
A note on application inference profiles and Projects
Amazon Bedrock offers two more attribution mechanisms, each suited to a different pattern. Application inference profiles are pre-created, per-application resources on the bedrock-runtime endpoint that carry cost allocation tags. You create one profile per application per model per AWS Region, and callers reference the profile ARN, which fits teams with a small and stable set of applications and models. Projects and workspaces serve the bedrock-mantle endpoint (Responses API, Chat Completions API). A project isn’t tied to a specific model, so a single project ID spans many models, which fits teams whose model usage changes frequently. You pass the project ID at client initialization, and its tags flow to CUR 2.0 and Cost Explorer (see Manage AI costs with Amazon Bedrock Projects). Projects are the attribution path for OpenAI-compatible workloads on Amazon Bedrock.
For bedrock-runtime workloads, we recommend IAM principal attribution through session tags as the primary mechanism. It scales with your identity system, requires no pre-created resources, and covers every model and API operation. Application inference profiles work well when a small, stable set of business units needs invoice-grade billing separation without changes to identity design. Use projects for bedrock-mantle workloads that use the Responses API or Chat Completions API.
Stage 2: Make every request observable #
CUR tells you who spent. Invocation logs tell you what happened. Amazon Bedrock supports model invocation logging of request data, response data, and metadata to Amazon CloudWatch Logs, Amazon Simple Storage Service (Amazon S3), or both. Use Amazon CloudWatch for fast operational queries and S3 plus Amazon Athena for long-term business intelligence (BI) access. Treat logging as infrastructure, not a manual toggle: provision the log group or bucket through code, define retention and encryption policies, and validate delivery during environment setup. If logging exists only in development or only in one Region, your cost program becomes fragmentary the moment usage grows.
Logging also closes the latency gap: CUR attribution lags roughly 24 hours, but CloudWatch plus OpenTelemetry can power near-real-time dashboards showing per-developer spend within minutes (Analyzing Claude Code usage with CloudWatch and OpenTelemetry demonstrates this end to end).
Request metadata for workload-level attribution
Raw logs answer identity-level questions. To answer workload-level questions (which feature, workflow, or tenant generated the tokens), stamp request metadata on every call. Amazon Bedrock captures this metadata in the invocation logs for later analysis:
The following example uses Anthropic Claude Sonnet 5.5 on Amazon Bedrock through the US cross-Region inference profile. For the full list of models, see Supported foundation models in Amazon Bedrock. For availability by Region, see Model support by AWS Region in Amazon Bedrock.
Amazon Bedrock doesn’t enforce request metadata, and teams drift into inconsistent schemas quickly when every application does its own thing. The platform team should inject mandatory metadata centrally in a shared SDK wrapper or gateway, normalize the key set, and treat metadata coverage as a platform health metric.
Diagnostic queries
With logs and metadata in place, CloudWatch Logs Insights answers the standing questions. Who consumes the most tokens:
The same query grouped by requestMetadata.application and requestMetadata.workflow instead of caller produces the workload-level leaderboard: the view product and engineering teams will ask FinOps for the moment chargeback conversations start.
Note: Field paths shown in the preceding query are illustrative. Verify against your invocation log schema, which may vary by model, API operation, and logging configuration.
Stage 3: Enforce budgets #
With attribution and observability in place, enforcement becomes an architecture choice rather than a wish. There are three viable tiers. The right one depends on your spend velocity, your risk tolerance for overspend, and how your tools connect to Amazon Bedrock.
Tier 1: Monitor and alert (no blocking)
Amazon Bedrock writes principal identity into the billing pipeline automatically. You put AWS Budgets on top for threshold alerts, with the OpenTelemetry dashboards from Stage 2 closing the 24-hour CUR gap for day-to-day visibility. Nothing to deploy, native AWS billing, and no added latency on inference calls. It works with tools that call Amazon Bedrock directly. Nothing can hard-stop spend, so you see the problem after it happens. Right for early adoption, where alerting plus human intervention is sufficient. Two builder guides walk through it: Limiting Bedrock usage and controlling costs, and Deep dive on managing Bedrock in Claude Code with AWS Budgets and Tags.
Tier 2: IAM tag enforcement (deny on breach)
An aggregator AWS Lambda function tallies per-user token and dollar counters in Amazon DynamoDB from invocation log data. When a user breaches their threshold, the function adds their identity to a shared IAM deny policy (denying bedrock:*) until a monthly reset or admin override. This enforcement runs entirely on IAM and Amazon Bedrock logging, with no third-party component in the inference path, and it works with tools that call Amazon Bedrock directly. The Guidance for Claude Code with Amazon Bedrock in the AWS Solutions Library provides a deployable variant. It checks each user’s usage against per-user and per-team limits when it issues credentials, and withholds credentials once a user exceeds their limit. Because that check uses usage telemetry emitted by the client, its figures can differ from CUR. In either form, enforcement is asynchronous. A block takes effect seconds to minutes after the breaching call for an IAM policy update, or at the next credential refresh for the credential check. Your platform team owns the pipeline. This tier is right for teams that need real enforcement, can tolerate that lag, and want to keep third-party software out of the data path.
Tier 3: Gateway enforcement (synchronous block)
A gateway sits in the inference data path and enforces budgets synchronously, returning HTTP 429 the moment a user or team exceeds their limit. You get sub-second blocking, per-user model allowlists, token rate limits, and centrally enforced routing rules. The trade-off: a component in the data path that must be deployed, scaled, and security-reviewed. Clients must be configured to send their requests through the gateway.
For Claude Code and Claude Desktop fleets, the Claude apps gateway for AWS is the official recommendation. It ships inside the CLI binary and runs as a single stateless container backed by PostgreSQL. It provides OIDC identity, centrally managed policy, OpenTelemetry telemetry, multi-Region Amazon Bedrock routing, and per-user/group/org spend caps out of the box. For mixed-model workloads spanning multiple providers, general-purpose gateways remain the right choice: LiteLLM through the Amazon Elastic Container Service (Amazon ECS) on AWS Fargate reference architecture, or a serverless alternative described in Build a proactive AI cost management system for Amazon Bedrock, Part 2.
Choosing a tier
Two factors dominate.
Connectivity: If developers call Amazon Bedrock directly with no proxy, only Tiers 1 and 2 are viable.
Spend velocity: If spend is already in the millions per month, build enforcement now. If alerts can hold the line for another quarter, invest in Stages 1 and 2 first and add enforcement when the data demands it.
Most organizations should phase it: observe at Tier 1, then step up. Treat all three as bridges, not destinations: the Well-Architected Agentic AI Lens already codifies agent cost governance as a first-class architectural concern, and the attribution foundation from Stage 1 is what any managed enforcement will sit on.
Stage 4: Optimize the defaults #
Most Amazon Bedrock waste is not individual carelessness. It is platform defaults nobody set. The two highest-impact levers are both platform-controlled.
Model routing
Not every task needs a frontier model. Routine completions (boilerplate, linting suggestions, small refactors) can run on smaller, cheaper models with quality reserved for complex reasoning, through intelligent prompt routing within a model family or routing rules at the gateway (see Reduce costs and latency with Amazon Bedrock Intelligent Prompt Routing and prompt caching for the full walkthrough). Which workflows can downshift is an empirical question: pull the workload leaderboard from Stage 2, run a quality evaluation on the cheaper model for the top entries, and route what passes. For agentic coding fleets, where boilerplate dominates call volume, more than half of calls landing on cheaper models is an achievable target.
Prompt caching
Prompt caching rewards stable repeated prefixes. As of October 2026, the published Amazon Bedrock pricing for Anthropic Cl