Prompt Caching at the Edge: Using CloudFront Functions and Lambda to Speed Up Claude Calls

A developer has shared a technique for reducing latency in LLM API calls by implementing prompt caching at the edge using AWS CloudFront Functions and Lambda. The approach stores prompt-response pairs in a fast lookup table, allowing repeated queries to be answered instantly without contacting the LLM provider. The developer notes that CloudFront Functions can access a built-in KV store but cannot make outbound network requests, so the heavy lifting remains in Lambda.

LLM APIs like Claude feel snappy—until latency spikes hit your users. By caching prompt‑response pairs right at the edge, you can cut round‑trip time to milliseconds. This post shows you how to make that happen with CloudFront Functions and a Lambda origin. When a user types a question, your front‑end sends the text to an LLM large language model API, waits for the model to generate a reply, and then shows the answer. The user experience is dominated by two things: Even if the model itself is fast, the network hop to the provider’s data center can add 100 ms – 300 ms, and sometimes more during traffic spikes. For a chat UI that refreshes every few seconds, those extra milliseconds feel like a noticeable lag. Prompt caching means storing the exact prompt the user’s message together with the response the model’s answer in a fast lookup table. If the same prompt arrives again within a short window, you can return the cached answer instantly, without touching the LLM provider at all. In plain English:Think of the cache as a “sticky note” on the receptionist’s desk. If someone asks the same question twice, the receptionist can hand them the note instead of calling the manager again. LLM responses are not immutable—new data, temperature settings, or model updates can change the answer. A short time‑to‑live TTL of a few minutes gives you a good trade‑off: most users repeat recent prompts, but you still get new answers after a reasonable window. Install the AWS SDK for JavaScript v3 if you need it locally npm install @aws-sdk/client-cloudfront js // kv-setup.ts – run once with node import { CreateKeyGroupCommand, CreateCachePolicyCommand, CreateOriginRequestPolicyCommand, CreateDistributionCommand, CloudFrontClient, } from "@aws-sdk/client-cloudfront"; const client = new CloudFrontClient { region: "us-east-1" } ; async function createDistribution { // The KV store is defined via a Cache Policy that enables “Cache‑Based Origin Request”. // In the console this appears as “Cache key and origin request settings”. const cachePolicy = await client.send new CreateCachePolicyCommand { CachePolicyConfig: { Name: "ClaudePromptCachePolicy", // We want the request body the prompt to be part of the cache key. ParametersInCacheKeyAndForwardedToOrigin: { EnableAcceptEncodingBrotli: false, EnableAcceptEncodingGzip: false, // The request body is not automatically part of the key, // so we forward it to the origin Lambda where we will hash it. // The KV store itself is accessed via the Function, not the cache policy. }, DefaultTTL: 0, // we control TTL manually in the KV store MaxTTL: 0, MinTTL: 0, // Important: Edge Functions can only read from KV if the cache behavior // points to a "function association" with “viewer request”. // The association is set later in the console or via the API. // No extra code needed here. }, } ; // Omitted: create the origin Lambda ARN and the distribution. // The key point for the beginner is that the distribution must have: // - a Viewer Request Function our edge function // - an Origin our Lambda for /chat // - the default behavior points to the Lambda. } createDistribution .catch console.error ; Key takeaway:CloudFront Functions can read/write a built‑in KV store, but they can’t talk to the internet. They’re perfect for a quick “look‑aside” cache; the heavy lifting stays in Lambda. | Item | Why it matters | |---|---| 2 MB code limit for Functions | Keep the script tiny; avoid large libraries. | No outbound network | Trying to fetch Claude from the function returns 502. | Cache invalidation delay up to 60 s | If you delete a KV entry, the edge may still serve it for a minute. | OAI Origin Access Identity is deprecated | If you still use it, CloudFront will log warnings but still work; plan to switch to Origin Access Control. | The Lambda does three things: fetch . @aws-sdk/client-cloudfront CreateKeyValueStore ‑style API . js // lambda-handler.ts import { CloudFrontClient, PutKeyValueStoreCommand } from "@aws-sdk/client-cloudfront"; // Claude endpoint – replace with your actual URL and API key. const CLAUDE ENDPOINT = "https://api.anthropic.com/v1/complete"; const CLAUDE API KEY = process.env.CLAUDE API KEY ; // TTL in seconds for the edge KV entry. const KV TTL = 300; // 5 minutes export const handler = async event: any = { // ------------------------------------------------- // 1️⃣ Extract the prompt from the incoming request. // ------------------------------------------------- const body = JSON.parse event.body ; const prompt = body.prompt?.trim ; if prompt { return { statusCode: 400, body: JSON.stringify { error: "Missing 'prompt' field" } , }; } // ------------------------------------------------- // 2️⃣ Call Claude. fetch is available in Node 20 // ------------------------------------------------- const claudeResponse = await fetch CLAUDE ENDPOINT, { method: "POST", headers: { "Content-Type": "application/json", "x-api-key": CLAUDE API KEY, }, body: JSON.stringify { prompt, max tokens: 256, temperature: 0.7, } , } ; if claudeResponse.ok { const err = await claudeResponse.text ; return { statusCode: claudeResponse.status, body: JSON.stringify { error: err } , }; } const claudeData = await claudeResponse.json ; const answer = claudeData.completion; // ------------------------------------------------- // 3️⃣ Store prompt → answer in the Edge KV store. // ------------------------------------------------- // The KV store lives on CloudFront, not Lambda, but the SDK // lets us write to it from anywhere with the right permissions. const cf = new CloudFrontClient {} ; await cf.send new PutKeyValueStoreCommand { // “ClaudePromptCache” is the name you gave the KV store // when you created the distribution. KeyValueStoreName: "ClaudePromptCache", Items: { Key: prompt, // exact prompt string as the key Value: answer, // TTL tells the edge when the entry expires. // Edge will automatically drop it after 5 minutes. // The attribute name is TTL , expressed in seconds. TTL: KV TTL, }, , } ; // ------------------------------------------------- // 4️⃣ Return the answer to the caller. // ------------------------------------------------- return { statusCode: 200, headers: { "Content-Type": "application/json" }, body: JSON.stringify { answer } , }; }; PutKeyValueStoreCommand The CloudFront SDK treats the edge KV store like any other key‑value database: you give it a key the prompt and a value the answer . The TTL ensures the entry disappears after the freshness window. | Gotcha | Explanation | |---|---| Node 22 require esm breakage | If you upgrade the runtime, any require of an ES module will silently fail. Stick to import syntax or pin the runtime to Node 20. | SnapStart + VPC = no benefit | SnapStart speeds up cold starts only when the function has no VPC attachment. If you need VPC e.g., to reach a private Claude endpoint , the startup time stays the same. | Response streaming needs headers | If you ever switch to streaming Claude’s output, you must set Transfer-Encoding: chunked and a matching Content-Type . Otherwise CloudFront buffers the whole response. | Provisioned Concurrency cost | Enabling it makes the function always warm, but you’ll be billed even when idle. For a low‑traffic chat app, on‑demand is cheaper. | Lambda@Edge size limits | We are NOT using Lambda@Edge here; we use a regular regional Lambda, so the 1 MB response limit does not apply. | Raw prompts can be long, and the KV store limits key length max 256 bytes . A simple way to keep keys short is to hash the prompt with SHA‑256 and store the original prompt as metadata optional . python import crypto from "crypto"; function hashPrompt prompt: string : string { // Create a hex‑encoded SHA‑256 hash of the prompt. // This produces a fixed‑length, URL‑safe string. return crypto.createHash "sha256" .update prompt .digest "hex" ; } You would then use hashPrompt prompt as the KV key, and optionally store the original prompt in the value e.g., as a JSON object {prompt, answer} for debugging. Because the TTL is only five minutes, you rarely need manual invalidation. However, if you release a new model version or change the temperature, you may want to wipe the cache immediately. js // invalidate-cache.ts import { CloudFrontClient, DeleteKeyValueStoreCommand, } from "@aws-sdk/client-cloudfront"; const client = new CloudFrontClient {} ; export async function clearCache { await client.send new DeleteKeyValueStoreCommand { KeyValueStoreName: "ClaudePromptCache", } ; console.log "Edge KV store cleared" ; } Tip:Deleting the whole store is fast, but the edge may still serve stale data for up to 60 seconds due to propagation delay. If you need per‑key removal, you must overwrite the key with a very short TTL e.g., 1 second instead. | Gotcha | Fix | |---|---| KV write latency | Writes are asynchronous; the function returns to the caller before the edge fully propagates the entry. In practice the next request sees the entry within a few hundred milliseconds. | Maximum value size 10 KB | Claude answers are usually small, but if you request large completions, truncate or compress before storing. | Permission errors | The Lambda’s execution role must have cloudfront:UpdateKeyValueStore permission. Add a policy like {"Effect":"Allow","Action":"cloudfront:UpdateKeyValueStore","Resource":" "}. | A CloudFront Function runs on the viewer request event. It can read from the KV store, and if it finds a hit, it can craft a response and stop further processing i.e., skip the Lambda origin . // cf-function.js / CloudFront Function – viewer request. Looks for a cached answer to the incoming prompt. If found, returns it immediately; otherwise lets the request continue to the Lambda origin. / function handler event { var request = event.request; var uri = request.uri; // e.g., "/chat" // We only care about POST /chat if uri == "/chat" || request.method == "POST" { return request; // let everything else pass through } // ------------------------------------------------- // 1️⃣ Extract the request body the prompt JSON . // ------------------------------------------------- // CloudFront Functions can only read the body if the // request is sent with the "application/json" content type. var body = request.body && request.body.text; if body { // No body – forward to Lambda for proper error handling. return request; } var prompt; try { var parsed = JSON.parse body ; prompt = parsed.prompt?.trim ; if prompt throw "missing"; } catch e { // Malformed JSON – forward to Lambda. return request; } // ------------------------------------------------- // 2️⃣ Look up the prompt in the Edge KV store. // ------------------------------------------------- // The KV store is called "ClaudePromptCache". The key is the // raw prompt string or a hash if you used one in Lambda . var kvStore = "ClaudePromptCache"; var cached = KV.get kvStore, prompt ; // KV.get returns null if the key does not exist or has expired. if cached { // ------------------------------------------------- // 3️⃣ Build a synthetic response with the cached answer. // ------------------------------------------------- var response = { statusCode: 200, statusDescription: "OK", headers: { "content-type": { value: "application/json" }, "cache-control": { value: "max-age=0, private" }, }, body: JSON.stringify { answer: cached } , }; // Returning a response short‑circuits the rest of the // CloudFront pipeline – Lambda never runs. return response; } // ------------------------------------------------- // 4️⃣ No cache hit → let the request travel to Lambda. // ------------------------------------------------- return request; } /chat POST requests. KV.get store, key reads from the edge store. No network call, just a local memory read. Analogy:Imagine a library with a “quick‑look” shelf at the front desk. If the book you need is already on that shelf, the librarian hands it to you right away. If not, you have to go to the back rooms the Lambda to fetch it. CloudFront Functions are deployed with the @aws-sdk/client-cloudfront CreateFunctionCommand . Example run locally : npm install @aws-sdk/client-cloudfront js // deploy-function.ts import { CloudFrontClient, CreateFunctionCommand, PublishFunctionCommand, } from "@aws-sdk/client-cloudfront"; import as fs from "fs"; const client = new CloudFrontClient { region: "us-east-1" } ; async function deploy { const code = fs.readFileSync "cf-function.js", "utf8" ; const create = await client.send new CreateFunctionCommand { Name: "ClaudePromptCacheFunction", FunctionConfig: { Comment: "Look up Claude prompt cache at the edge", Runtime: "cloudfront-js-1.0", // the only supported runtime }, FunctionCode: Buffer.from code , } ; console.log "Created:", create.FunctionSummary?.ARN ; // Publishing makes the function live. const publish = await client.send new PublishFunctionCommand { Name: "ClaudePromptCacheFunction", IfMatch: create.ETag, // ensures we publish the right version } ; console.log "Published version:", publish.FunctionSummary?.FunctionMetadata?.FunctionARN ; } deploy .catch console.error ; After publishing, attach the function to the Viewer Request event of the /chat cache behavior in the CloudFront console or via the API. | Gotcha | Fix | |---|---| No outbound HTTP | Never try fetch inside the function; you’ll see a 502 error. | 2 MB size limit | Keep the code tiny, avoid bundlers, use native JSON.parse . | Only simple headers | Functions can set headers but can’t manipulate cookies beyond simple key‑value pairs. | Propagation delay up to 60 s | After a new version is published, give the distribution a minute before testing. | What you now have:a three‑piece pipeline that reduces the perceived latency of Claude calls from hundreds of milliseconds to a few milliseconds for repeat prompts. Now you can add prompt caching to any LLM‑backed feature—chatbots, code assistants, or summarization services—and give your users an experience that feels instant , even when the underlying model lives far away. Happy building Transparency noticeThis article was written with the help of an AI system — Groq GPT OSS 120B . Published:2026-08-27 ·Primary focus:CloudFrontAll code blocks are intended to be correct and runnable, but please verify them against the official docs for the tools mentioned before using in production. Find an error? Drop a comment — corrections are always welcome.