On November 12, 2026, AWS published Four Amazon Bedrock inference choices that affect cost—a measured follow-up to the same morning's Tokenomics at scale and cost-per-task ROI posts. Authors Sai Pavan Kasani and Aswath Ram A Srinivasan argue that request-level knobs still move bills after you tune prompts: prompt cache TTL, service tier, adaptive thinking effort, and maxTokens.
If you run agents on Bedrock or compare Grok 4.7 long-horizon patterns, these four settings are the difference between a flat forecast and a surprise CUR line.
TL;DR #
| Knob | AWS default or permissive state | What their tests showed | Risk AWS flags |
|---|---|---|---|
| Cache duration | Model-specific TTL | 1-hour cache beats 5-minute when same prefix returns after gaps over 5 minutes | 1-hour writes cost 2× input; low reuse loses money; partition tenant prefixes |
| Service tier | default |
Nova 2 Lite Flex billed at half Standard in us-west-2 at time of writing | Unsupported models error; set deadlines; optional fallback retry |
| Reasoning effort | high on tested Opus 5 |
Low→high +169% median output tokens on agent tasks; code task flat | Quality can drop on hard tasks; track pass rate with tokens |
maxTokens |
Large ceiling (e.g. 2048) | 155-token fence avoided 1,893 output tokens on one verbose prompt | stopReason max_tokens truncates multi-turn chains |
Where inference cost actually lives #
Every Bedrock response includes usage metadata. You pay input and output tokens at the model and tier rate, plus separate cache read and write counters, and reasoning tokens count as output on adaptive models.
That is why IAM principal caps still miss the target when developers change only application code: the same role can double output tokens by leaving effort at default after a model upgrade.
AWS recommends changing one setting at a time, confirming model support in the current Region, then validating in your task distribution—not vendor averages alone.
Prompt cache duration #
Repeated system prompts, tool definitions, and reference context can re-bill when cache entries expire. The prompt caching guide lists 5-minute and 1-hour TTL values for supported Anthropic models and 30-minute minimum TTL for supported GPT-5.6 models on Bedrock.
Explicit caching uses a Converse API checkpoint:
{ "cachePoint": { "type": "default", "ttl": "1h" } }
InvokeModel for Anthropic models places the same TTL inside cache_control.
Published rate shape (Sonnet 5 and Opus 5)
AWS cites Bedrock pricing for Anthropic Claude Sonnet 5 and Opus 5 on Bedrock:
| Event | Multiple of input rate (per AWS post) |
|---|---|
| 5-minute cache write | 1.25× |
| 1-hour cache write | 2× |
| Cache read | 0.1× |
Break-even intuition from AWS: a 5-minute cache pays back on the first reuse within 5 minutes. If the same prefix returns more than 5 minutes apart, you pay 1.25× writes on every call unless you use a longer TTL. A 1-hour cache is cheaper from the second request within the hour. Beating no cache needs three requests per hour with the reuse pattern AWS describes.
Implicit caching is best-effort without cache controls; duration tuning in the post targets explicit checkpoints.
Stability and multi-tenant layouts
AWS reran six calls on Claude Sonnet 5 with a 1-hour checkpoint, changing the system prompt once and the tool definition once. Each change produced zero cache reads and a new write—checkpoints evaluate in order tools, system, messages, so an early edit invalidates later sections.
For SaaS, AWS repeats the pattern in Optimizing cost and latency with Amazon Bedrock prompt caching for Anthropic models: prepend a tenant-specific SHA-256 hash to the cacheable prefix so entries partition inside one account. Enforce authorization before building the prompt; regulated tenants may still need separate AWS accounts for hard isolation.
Constraint: prompt caching does not work with batch inference—on-demand only.
Amazon Bedrock inference cost tuning starts with prompt cache TTL and invalidation timing
Service tier (Flex) #
Background jobs—eval harnesses, nightly summarization, backfills—may not need Standard latency. For models that support Flex, Converse accepts:
{ "serviceTier": { "type": "flex" } }
Use default, not standard, for Standard tier per AWS naming in the post.
Support matrix AWS reports
| Model family | Flex in model card (per AWS Nov 12 post) |
|---|---|
| Amazon Nova 2 Lite | Supported |
| OpenAI gpt-oss-120b | Supported |
| Anthropic Claude Sonnet 5 | Not supported |
| Anthropic Claude Opus 5 | Not supported |
An unsupported Flex request returns ValidationException—not a silent downgrade. AWS suggests a single catch-and-retry to default when policy allows, and warns not to retry every validation error.
Nova 2 Lite interleaved test
AWS sent 200 default and 200 flex requests in interleaved pairs on Nova 2 Lite. Both tiers processed the same input and output token totals; all 400 resolved to the requested tier; median latency difference was noise. At publication time, Flex in us-west-2 was half the Standard rate on the pricing page, and AWS said the bill matched.
Verify with response serviceTier and CloudWatch ResolvedServiceTier. See Service tiers for optimizing performance and cost.
Amazon Bedrock Flex service tier halves Standard rates on supported models like Nova 2 Lite
Adaptive thinking effort #
Reasoning tokens bill as output. Claude Sonnet 5 and Opus 5 default to high adaptive effort even when a request omits thinking—migrating from Sonnet 4.6 without edits can start billing thinking tokens immediately, and maxTokens caps thinking plus final text together.
Lower effort via Converse:
{
"thinking": { "type": "adaptive" },
"output_config": { "effort": "low" }
}
Keep effort inside output_config, not nested under thinking.
Opus 5 agent benchmark (AWS table)
AWS reused tasks from Build long-horizon agents with Grok 4.7 on Amazon Bedrock—each task has one correct answer. Eight runs per effort level:
| Effort | Correct runs | Median output tokens | Change vs low |
|---|---|---|---|
| low | 24 of 24 | 936 | baseline |
| medium | 24 of 24 | 1,484 | +59% |
| high (default) | 24 of 24 | 2,521 | +169% |
| xhigh | 24 of 24 | 2,964 | +217% |
| max | 24 of 24 | 3,762 | +302% |
Accuracy held on these tasks, but AWS stresses your distribution may differ—measure pass rate alongside tokens.
On the code generation sample in Invoke Bedrock model for code generation, median output was 676 at low versus 743 at high—within run-to-run variation, with programs passing at both levels.
Practice: start below high for bounded routines; keep high only where quality tests prove the margin. Cross-read our Grok 4.7 effort cost walkthrough for a parallel agent lens.
Bedrock adaptive thinking effort raises output tokens on agent workloads
Output token limits (maxTokens) #
maxTokens is a ceiling on application output, not a substitute for Bedrock Guardrails. Quota logic reserves input tokens plus maxTokens at request start per How tokens are counted in Amazon Bedrock.
Example Converse fence:
{ "inferenceConfig": { "maxTokens": 155 } }
Illustrative Sonnet 5 test
After five normal outputs peaking at 103 tokens, AWS set maxTokens: 155. The same deliberately verbose prompt with thinking disabled produced 2,048 tokens under a 2,048 ceiling versus 155 under the fence—1,893 tokens avoided, both ending at stopReason max_tokens. The fence limits spend but does not fix a bad prompt; truncated turns can break downstream agent steps.
Calibration: compare P95 and P99 per task type with thinking enabled when adaptive thinking is on; P95 caps exposure more aggressively, P99 preserves long tails at higher cost.
Putting the four knobs together #
AWS's closing table is a deployment checklist:
| Symptom | Verify with | First response |
|---|---|---|
| Cache not helping | Latency and cost telemetry | Prefix stability, reuse interval, model support |
| Wrong tier | serviceTier plusResolvedServiceTier |
Supported tier or move interactive traffic to default |
| Tokens up, quality flat | Invocation logs plus pass rate | Lower effort for that task class |
Frequent max_tokens stops |
Stop reason plus output distribution | Inspect prompt before raising ceiling |
Maintain a capability record per model version and Region: supported cache TTL, tiers, effort levels, and fences—validate each request before send.
If you also operate a LiteLLM gateway on Fargate, encode those records in the router so one misconfigured profile does not inherit another team's cache TTL.
What builders should do this week #
- Inventory defaults on production Bedrock profiles—especiallyeffort after any Sonnet or Opus upgrade.
- Plot reuse intervals for stable system and tool prefixes; pick5m versus 1h with the AWS break-even math, then confirm cache read and write lines in CUR.
- Pilot Flex only on supported models in latency-tolerant queues; log
ValidationExceptionrates. - Set task-scoped
maxTokensfrom measured P95 with headroom; alert onmax_tokensstopReason spikes. - Link spend to principals using the companionTokenomics guide so experiments show up in the right cost center.
What people are asking #
Does this replace model routing or distillation? No—AWS points to Effective cost optimization strategies for Amazon Bedrock for model choice, batch inference, and distillation. These four knobs sit inside a chosen model.
Can I mix Flex and prompt caching? Caching rules are independent, but both need model support checks. Flex failures are immediate; cache misses show up as write spikes.
Does Kimi K3.1 Pro highspeed change the math? Separate model card and pricing—rerun the same telemetry loop when you add Bedrock highspeed profiles.
Related reading #
Inference parameters, pricing multiples, and model support matrices reflect AWS documentation and the November 12, 2026 AWS Machine Learning blog post as we read them; confirm current Region and model card text before production rollout.