Use prompt caching with open-weight models on Amazon Bedrock Amazon Bedrock now supports explicit prompt caching for open-weight models, letting applications mark reusable prompt prefixes with `prompt_cache_breakpoint` markers via the Chat Completions API and read cache usage from each response, AWS announced in a blog post. The service also offers implicit caching, which reuses eligible prefixes automatically on a best-effort basis, and AWS states that cache reads are billed at a lower rate than standard input while cache writes can be billed higher depending on the model. AWS says prompts and outputs are not shared with model providers or used to train models, and that savings grow with how much context is reused and how often. Artificial Intelligence https://aws.amazon.com/blogs/machine-learning/ Use prompt caching with open-weight models on Amazon Bedrock Many generative AI applications send the same long context with every request, and prompt caching avoids processing it again each time. That context can be a system prompt, reference documents, tool definitions, or the conversation so far. Open-weight models on Amazon Bedrock can reuse it across requests, which helps reduce input costs and response latency. Prompt caching https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html reuses the computed state of a matching prompt prefix, so repeated context is read from cache instead of being processed again. Amazon Bedrock manages the cache for you. Depending on the model, you can use implicit caching, which works automatically, explicit caching, where you mark the reusable prefix, or both. The more context you reuse, and the more often you reuse it, the more you can save. As with other models on Amazon Bedrock, your prompts and outputs aren’t shared with the model provider or used to train models. For more information, see Data protection https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html in the Amazon Bedrock User Guide. Open-weight models on Amazon Bedrock now support explicit prompt caching. In this post, we use both modes with an open-weight model through the Chat Completions API https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html and show how to read cache usage from each response. We then apply implicit caching to an OpenCode coding agent session on the same API and close with best practices. Prompt caching for open-weight models on Amazon Bedrock When a request starts with the same content as an earlier one, Amazon Bedrock reads that prefix from cache and processes only the rest. Cache reads are billed at a lower rate than standard input, so the savings can grow each time a prefix is reused. Depending on the model, cache writes can be billed at a higher rate than standard input. See Amazon Bedrock pricing https://aws.amazon.com/bedrock/pricing/ for the rates of each model. Supported modes depend on the model and the API. For each model’s caching modes, minimum prefix length, and how long entries stay cached, see Models at a glance https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html . The two modes compare as follows: | Feature | Implicit caching | Explicit caching | | Request changes | None | On the Chat Completions API, prompt cache options with mode set to explicit, plus one or more prompt cache breakpoint markers | | What is cached | Eligible prefixes, selected automatically on a best-effort basis | Content up to and including each breakpoint | | Good fit | Prompts that already start with a stable prefix, such as conversations and agent loops | A fixed shared prefix with a changing question, or layered prefixes that you want to control | Implicit caching : Amazon Bedrock reuses eligible prefixes automatically, on a best-effort basis. In a conversation, each request resends the full context, so the previous request becomes the cached prefix of the next one Figure 1 . As the prompt caching guide https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html recommends, put static content first and dynamic content last. Keeping the start of the prompt stable lets every later request reuse it. If an early part changes, as in turn 4 of Figure 1, the cache is rebuilt from that point. Explicit caching : You mark the end of each reusable prefix with a cache breakpoint. You can add several breakpoints, up to the maximum that the model supports, to cache layers that change at different rates, as Figure 2 shows. Content up to each breakpoint is written to cache once and then read on later requests, and content after the last breakpoint is processed on each request. Check the model card for the minimum token count before a breakpoint, the maximum number of breakpoints, and the time to live TTL , which resets with each cache hit. With implicit caching, Amazon Bedrock can also write content to cache that later requests never reuse, such as a question that’s different each time. When the content after a shared prefix changes on every request, use explicit caching to limit cache writes to the shared prefix. Content after the last breakpoint is billed at the standard input rate. On models that charge more for cache writes, you don’t pay the higher write rate for content that’s never read back. Solution overview The following diagram shows how both examples in this post reach Amazon Bedrock through the same Chat Completions API endpoint. 1. The Python application and the OpenCode coding agent send requests to the Amazon Bedrock Runtime endpoint in your Region, authenticated with a short-term Amazon Bedrock API key. 2. Amazon Bedrock checks the prompt cache for a matching prefix. It reads a matching prefix from cache, and writes new cacheable content to cache. 3. Amazon Bedrock sends the request to the selected open-weight model and returns the response, including the cache usage fields that the following sections show you how to read. Prerequisites To run the samples, you need the following: - An AWS account with access to your selected model on Amazon Bedrock. Model availability and prompt caching support vary by AWS Region, so check the model card https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html for the Regions that your model supports. - AWS credentials for an AWS Identity and Access Management IAM identity that has permission to invoke your selected model on Amazon Bedrock. - Python 3.12 with the openai 3.26.0 or later and aws-bedrock-token-generator 1.1.0 or later packages. - For the coding agent example, npm to install OpenCode 1.18.35. Before you use a model, review its license and service terms, linked from its model card https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html . Your use of the model is also subject to the AWS Responsible AI Policy https://aws.amazon.com/ai/responsible-ai/policy/ . Cache a product manual through the Chat Completions API The example is a support assistant for a fictional water filter, the AnyCompany AquaPure 500. The system prompt holds a one-line instruction and the owner’s manual, which is the reusable prefix. Each request asks a different customer question. Set up the client The samples use the OpenAI Python SDK with the Amazon Bedrock Chat Completions endpoint. For authentication, the provide token function from the aws-bedrock-token-generator package turns your AWS credentials into a short-term Amazon Bedrock API key https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html , valid for up to 12 hours. You don’t need to create or store a long-term key. 1. Create a virtual environment and install the packages: 2. Create the manual. Ours is about 4,000 tokens of Markdown. The following excerpt shows its format: Save it as manual.md and add sections until it’s longer than the minimum prefix length on your model card. Placeholder text works, because the cache only needs an identical prefix. Keep the three sections shown, which answer the sample questions. 3. Save the following setup code as common.py in the same directory. The samples that you create next import from it. If your model returns reasoning tokens, they count toward max tokens . The samples set max tokens=1024 so that short answers have room to finish. Implicit caching Implicit caching takes no cache controls. Put the manual in the system message and the question in the user message. Sending both as parts of one user message got no implicit cache hits. Save the following as implicit.py and run it with python implicit.py : Verify implicit caching Each response reports cache activity in usage.prompt tokens details . Figure 4 shows how to read these fields and the pattern to expect across the three calls of each sample. In implicit mode, call 1 writes the prompt to cache. Calls 2 and 3 read the system message, with the instruction and the manual, back from cache. They also write a few tokens each, because implicit mode extends the cached prefix to cover each new question. A few tokens after the last message stay uncached on every call. This is the pattern for the first run. If you run the sample again within the TTL, every call reads from cache, and none writes. Explicit caching Choose explicit caching to control exactly what is cached. As Figure 4 shows, calls 2 and 3 of the implicit run each write a few tokens to cache, whereas the same calls with explicit caching write nothing. With several breakpoints, you can cache layered prefixes, such as shared instructions and tools followed by a per-customer document Figure 2 . For explicit caching, add the following to the request: - prompt cache options with mode set to explicit selects explicit mode. It’s an Amazon Bedrock request field, and the OpenAI SDK sends it through extra body . - prompt cache breakpoint on a text content part marks the end of the reusable prefix. Content up to and including that part is cached, and content after it’s processed again on every request. Save the explicit version of the same loop as explicit.py :