# Use prompt caching with open-weight models on Amazon Bedrock

> Source: <https://aws.amazon.com/blogs/machine-learning/use-prompt-caching-with-open-weight-models-on-amazon-bedrock/>
> Published: 2026-11-16 15:42:41+00:00

## [Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/)

# Use prompt caching with open-weight models on Amazon Bedrock

Many generative AI applications send the same long context with every request, and prompt caching avoids processing it again each time. That context can be a system prompt, reference documents, tool definitions, or the conversation so far. Open-weight models on Amazon Bedrock can reuse it across requests, which helps reduce input costs and response latency.

[Prompt caching](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html) reuses the computed state of a matching prompt prefix, so repeated context is read from cache instead of being processed again. Amazon Bedrock manages the cache for you. Depending on the model, you can use implicit caching, which works automatically, explicit caching, where you mark the reusable prefix, or both. The more context you reuse, and the more often you reuse it, the more you can save.

As with other models on Amazon Bedrock, your prompts and outputs aren’t shared with the model provider or used to train models. For more information, see [Data protection](https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html) in the Amazon Bedrock User Guide.

Open-weight models on Amazon Bedrock now support explicit prompt caching. In this post, we use both modes with an open-weight model through the [Chat Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html) and show how to read cache usage from each response. We then apply implicit caching to an OpenCode coding agent session on the same API and close with best practices.

## Prompt caching for open-weight models on Amazon Bedrock

When a request starts with the same content as an earlier one, Amazon Bedrock reads that prefix from cache and processes only the rest. Cache reads are billed at a lower rate than standard input, so the savings can grow each time a prefix is reused. Depending on the model, cache writes can be billed at a higher rate than standard input. See [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) for the rates of each model.

Supported modes depend on the model and the API. For each model’s caching modes, minimum prefix length, and how long entries stay cached, see [Models at a glance](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html). The two modes compare as follows:

| **Feature** | **Implicit caching** | **Explicit caching** | 
| Request changes | None | On the Chat Completions API, `prompt_cache_options` with mode set to explicit, plus one or more`prompt_cache_breakpoint` markers | 
| What is cached | Eligible prefixes, selected automatically on a best-effort basis | Content up to and including each breakpoint | 
| Good fit | Prompts that already start with a stable prefix, such as conversations and agent loops | A fixed shared prefix with a changing question, or layered prefixes that you want to control | 

**Implicit caching**: Amazon Bedrock reuses eligible prefixes automatically, on a best-effort basis. In a conversation, each request resends the full context, so the previous request becomes the cached prefix of the next one (Figure 1). As the [prompt caching guide](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html) recommends, put static content first and dynamic content last. Keeping the start of the prompt stable lets every later request reuse it. If an early part changes, as in turn 4 of Figure 1, the cache is rebuilt from that point.

**Explicit caching**: You mark the end of each reusable prefix with a cache breakpoint. You can add several breakpoints, up to the maximum that the model supports, to cache layers that change at different rates, as Figure 2 shows. Content up to each breakpoint is written to cache once and then read on later requests, and content after the last breakpoint is processed on each request. Check the model card for the minimum token count before a breakpoint, the maximum number of breakpoints, and the time to live (TTL), which resets with each cache hit.

With implicit caching, Amazon Bedrock can also write content to cache that later requests never reuse, such as a question that’s different each time. When the content after a shared prefix changes on every request, use explicit caching to limit cache writes to the shared prefix. Content after the last breakpoint is billed at the standard input rate. On models that charge more for cache writes, you don’t pay the higher write rate for content that’s never read back.

## Solution overview

The following diagram shows how both examples in this post reach Amazon Bedrock through the same Chat Completions API endpoint.

1. The Python application and the OpenCode coding agent send requests to the Amazon Bedrock Runtime endpoint in your Region, authenticated with a short-term Amazon Bedrock API key.
2. Amazon Bedrock checks the prompt cache for a matching prefix. It reads a matching prefix from cache, and writes new cacheable content to cache.
3. Amazon Bedrock sends the request to the selected open-weight model and returns the response, including the cache usage fields that the following sections show you how to read.

## Prerequisites

To run the samples, you need the following:

- An AWS account with access to your selected model on Amazon Bedrock. Model availability and prompt caching support vary by AWS Region, so check the [model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html) for the Regions that your model supports.
- AWS credentials for an AWS Identity and Access Management (IAM) identity that has permission to invoke your selected model on Amazon Bedrock.
- Python 3.12 with the `openai` (3.26.0 or later) and`aws-bedrock-token-generator` (1.1.0 or later) packages.
- For the coding agent example, npm to install OpenCode 1.18.35.

Before you use a model, review its license and service terms, linked from its [model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html). Your use of the model is also subject to the [AWS Responsible AI Policy](https://aws.amazon.com/ai/responsible-ai/policy/).

## Cache a product manual through the Chat Completions API

The example is a support assistant for a fictional water filter, the AnyCompany AquaPure 500. The system prompt holds a one-line instruction and the owner’s manual, which is the reusable prefix. Each request asks a different customer question.

### Set up the client

The samples use the OpenAI Python SDK with the Amazon Bedrock Chat Completions endpoint. For authentication, the `provide_token()` function from the `aws-bedrock-token-generator` package turns your AWS credentials into a [short-term Amazon Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html), valid for up to 12 hours. You don’t need to create or store a long-term key.

1. Create a virtual environment and install the packages:

2. Create the manual. Ours is about 4,000 tokens of Markdown. The following excerpt shows its format:

Save it as `manual.md` and add sections until it’s longer than the minimum prefix length on your model card. Placeholder text works, because the cache only needs an identical prefix. Keep the three sections shown, which answer the sample questions.

3. Save the following setup code as `common.py` in the same directory. The samples that you create next import from it.

If your model returns reasoning tokens, they count toward `max_tokens`. The samples set `max_tokens=1024` so that short answers have room to finish.

## Implicit caching

Implicit caching takes no cache controls. Put the manual in the system message and the question in the user message. Sending both as parts of one user message got no implicit cache hits. Save the following as `implicit.py` and run it with `python implicit.py`:

### Verify implicit caching

Each response reports cache activity in `usage.prompt_tokens_details`. Figure 4 shows how to read these fields and the pattern to expect across the three calls of each sample.

In implicit mode, call 1 writes the prompt to cache. Calls 2 and 3 read the system message, with the instruction and the manual, back from cache. They also write a few tokens each, because implicit mode extends the cached prefix to cover each new question. A few tokens after the last message stay uncached on every call. This is the pattern for the first run. If you run the sample again within the TTL, every call reads from cache, and none writes.

## Explicit caching

Choose explicit caching to control exactly what is cached. As Figure 4 shows, calls 2 and 3 of the implicit run each write a few tokens to cache, whereas the same calls with explicit caching write nothing. With several breakpoints, you can cache layered prefixes, such as shared instructions and tools followed by a per-customer document (Figure 2).

For explicit caching, add the following to the request:

- `prompt_cache_options` with mode set to**explicit** selects explicit mode. It’s an Amazon Bedrock request field, and the OpenAI SDK sends it through`extra_body` .
- `prompt_cache_breakpoint` on a text content part marks the end of the reusable prefix. Content up to and including that part is cached, and content after it’s processed again on every request.

Save the explicit version of the same loop as `explicit.py`:
