AWS argues that the right unit for AI agent spend is cost per task, not cost per token, and its own worked example has a model that lists at double the price costing about 65% less per task. The argument comes from Part 2 of AWS's Amazon Bedrock spend series, published November 12, 2026 by Majd Arbash, Ajit Mahareddy and Aswath Ram A Srinivasan. Part 1 covered who spent the money and how to cap it. A companion AWS post, Tokenomics at scale, shows the native tag-and-budget route. Part 2 asks two harder questions: why a workflow is expensive, and whether the agent beats the humans it replaces.
The advice is vendor guidance written for Bedrock, and the example prices are illustrative. The method still applies to any model provider, which is why it is worth reading even if you do not use AWS. This post covers the cost-per-task and ROI-method sections that were readable for us.
TL;DR #
| Idea | AWS's point |
|---|---|
| Unit of cost | One complete task, not one token |
| Hidden drivers | Tokenizer, verbosity, agent loop turns, reasoning tokens |
| Example | Model A: $0.0042 per task. Model B: $0.00145, about 65% cheaper despite 2x list price |
| Failure cost | Divide by success rate, or add escalation cost |
| Levers | Prompt caching, prompt routing, schemas and caps, batch inference |
| ROI | Baseline first, experiment from day one, track human minutes |
Why cost per token misleads #
A token price tells you what a call costs, not what a job costs. AWS defines a task as one complete unit of work, such as resolving one support email, and notes that it may involve many model calls. It lists four hidden drivers.
- Tokenizer efficiency. Each model family splits the same text into a different number of tokens. A cheaper model can cost more if it produces many more tokens for your data, especially structured JSON or non-Latin scripts.
- Output alignment and verbosity. Output tokens typically cost three to five times as much as input tokens, per AWS. A model that ignores formatting limits adds billable filler.
- Agentic loop efficiency. Each tool-use turn re-sends the full history as input, so extra turns compound the cost.
- Reasoning overhead. Extended thinking tokens bill as output tokens even when the reasoning is redacted. AWS recommends keeping reasoning off until a benchmark shows a quality gap that justifies it.
That fourth point matters for anyone who turned on a "high" thinking setting by default. We saw the same effect in the Gemini 4 Hydrogen comparison, where an independent cost-per-task figure rises with the thinking level because output tokens nearly double.
The worked example #
AWS compares two models on one workflow: read a 750-word email, extract entities, call an account API and output strict JSON. The prices are illustrative.
| Metric | Model A (lower price) | Model B (higher price) |
|---|---|---|
| List price per 1M tokens, input / output | $0.50 / $1.50 | $1.00 / $3.00 | | Cumulative input tokens | 6,150 | 1,000 | | Cumulative output tokens | 750 | 150 | | Agentic turns | 3, with error correction | 1, zero-shot | | Total task cost | $0.0042 | $0.00145 |
You can check the math. Model A costs 6,150 times $0.50 per million plus 750 times $1.50 per million, which is $0.003075 plus $0.001125, or $0.0042. Model B costs $0.001 plus $0.00045, or $0.00145. That is about 65% less for the model with twice the list price.
AWS gives three reasons. Model A's tokenizer inflates the inputs. Its verbose JSON triples the output tokens. Its schema misalignment causes extra turns, and each turn re-sends a growing history. This is one example built to make a point, not a market survey, so do not read it as a claim about any real pair of models.
Cost per successful task #
Failures add cost, so AWS gives two formulas:
- Retry until success: cost per successful task equals cost per attempt divided by the success rate.
- Escalate on failure: cost per successful task equals cost per attempt plus the failure rate times the escalation cost.
A quick check shows why this matters. If an attempt costs $0.01 and succeeds 50% of the time, a retry loop costs $0.02 per success. A pricier attempt at $0.015 that succeeds 90% of the time costs about $0.0167. The cheaper model loses. These numbers are ours, for illustration, not AWS's.
With prompt caching, AWS says input cost splits into three buckets: uncached input at the base rate, cache reads below the base rate, and cache writes above it. It also warns that cache checkpoints drift as your application and tools change, so you should re-check the cache setup regularly. For a broader view of caching habits, see our guide to token efficiency and prompt cache sessions.
How to measure cost on your own workload #
AWS points to several Bedrock features:
- Model evaluation to score quality on your own prompt datasets.
- CountTokens to compare tokenizer efficiency across candidate models without running inference.
- The response usage block , which reports input, output, cache-read and cache-write tokens. With caching on, total input is the sum of all three, and each is priced differently.
- Model invocation logging to S3 or CloudWatch Logs, so you can measure turns to completion on real traffic.
- Request metadata to tag calls by team, feature or experiment. AWS notes these are log fields, not cost allocation tags, so they do not appear in Cost Explorer or the Cost and Usage Report.
- Application inference profiles , which show in cost reports at daily granularity. That suits chargeback but is too coarse for cost per task.
Multiplying logged tokens by published rates gives a near-real-time estimate. AWS says it ignores commitments, batch pricing and provisioned throughput unless you model them, so reconcile it against the Cost and Usage Report before treating it as the bill. Our post on the Bedrock tokenomics data lake covers the invocation-log side of this in more depth.
Four levers to lower cost per task #
- Prompt caching. It reduces re-sent context, and AWS calls it the largest win on agentic paths.
- Intelligent prompt routing. It sends easier requests to a cheaper model in the same family.
- Tool schemas and maxTokens caps. They control verbosity more reliably than prose instructions.
- Batch inference. It roughly halves the rate for bulk single-turn work, but it does not support tool calling or multi-turn exchanges.
The same logic appears outside AWS. Teams that cut cost by delegating easy steps to smaller models report large savings, as in Nvidia's edge delegation result and the Gartner open-weight analysis, though those are vendor or analyst claims that we flagged as such.
Proving the agent is worth it #
AWS argues that efficiency does not establish value. A cheap agent that does the wrong thing is still waste. The core question is whether the agent beats the human process. Its method has three steps.
1. Document the human process first. Describe triggers, roles, handoffs and completion criteria. That description decides what you measure later.
2. Measure the baseline before launch. Capture time to completion, queue time, active effort, reopen and escalation rates, cost per task and outcome quality. AWS warns that retrospective baselines are "systematically flattering to the agent," because people remember average cases and forget the fast ones.
3. Run the agent as an experiment from day one. Give every run a trace ID and a workflow-run record. AWS's example record covers:
- Identity and cost attribution: principal, team and cost center.
- Experiment block: variant, holdout flag, prompt version and policy version, passed into Bedrock request metadata so model-level and workflow-level records can be joined.
- Operations: latency, retries, tool calls, human approvals, human minutes and escalations.
- Economics: input, output and cache-read tokens, plus estimated cost.
- Outcome: status, acceptance, a reopen flag and a revenue event ID for later revenue attribution.
AWS highlights three fields. Human minutes matter because without them the agent's apparent productivity silently ignores human review time. The revenue event ID costs nothing to record and cannot be rebuilt later. The experiment block lets you join the two record types. The article goes on to describe a scorecard and holdout experiments, which we do not summarize because that part of the page was not readable for us.
What to do with this #
- Pick one workflow and define the task. Count calls per task from invocation logs, not from the price sheet.
- Run CountTokens on two or three candidate models with your real inputs. Compare tokenizers before comparing prices.
- Cap output with a schema and maxTokens, and test with reasoning off first.
- Compute cost per successful task, using your measured success rate.
- Record a baseline for the human process before you ship the agent, and keep a holdout group.
- Log human minutes. It is the field most teams skip and the one that changes the ROI answer.
For the platform side of running agents on AWS, see the AgentCore runtime next-generation post.
Caveats #
This is vendor guidance from AWS, and the worked example uses illustrative prices, not benchmarks of real models. We read the sections through the instrumentation example and not the closing sections. Features named here, such as CountTokens and intelligent prompt routing, are Bedrock features, so check the current docs for your region and models before you build on them.
Related reading #
This post reflects sources available on November 12, 2026.