{"slug": "prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up", "title": "Prompt Caching at the Edge: Using CloudFront Functions and Lambda to Speed Up Claude Calls", "summary": "A developer has shared a technique for reducing latency in LLM API calls by implementing prompt caching at the edge using AWS CloudFront Functions and Lambda. The approach stores prompt-response pairs in a fast lookup table, allowing repeated queries to be answered instantly without contacting the LLM provider. The developer notes that CloudFront Functions can access a built-in KV store but cannot make outbound network requests, so the heavy lifting remains in Lambda.", "body_md": "LLM APIs like Claude feel snappy—until latency spikes hit your users. By caching prompt‑response pairs right at the edge, you can cut round‑trip time to milliseconds. This post shows you how to make that happen with CloudFront Functions and a Lambda origin.\n\nWhen a user types a question, your front‑end sends the text to an LLM (large language model) API, waits for the model to generate a reply, and then shows the answer. The user experience is dominated by two things:\n\nEven if the model itself is fast, the network hop to the provider’s data center can add 100 ms – 300 ms, and sometimes more during traffic spikes. For a chat UI that refreshes every few seconds, those extra milliseconds feel like a noticeable lag.\n\n**Prompt caching** means storing the exact *prompt* (the user’s message) together with the *response* (the model’s answer) in a fast lookup table. If the same prompt arrives again within a short window, you can return the cached answer instantly, without touching the LLM provider at all.\n\nIn plain English:Think of the cache as a “sticky note” on the receptionist’s desk. If someone asks the same question twice, the receptionist can hand them the note instead of calling the manager again.\n\nLLM responses are not immutable—new data, temperature settings, or model updates can change the answer. A short time‑to‑live (TTL) of a few minutes gives you a good trade‑off: most users repeat recent prompts, but you still get new answers after a reasonable window.\n\n```\n# Install the AWS SDK for JavaScript (v3) if you need it locally\nnpm install @aws-sdk/client-cloudfront\njs\n// kv-setup.ts – run once with node\nimport {\n  CreateKeyGroupCommand,\n  CreateCachePolicyCommand,\n  CreateOriginRequestPolicyCommand,\n  CreateDistributionCommand,\n  CloudFrontClient,\n} from \"@aws-sdk/client-cloudfront\";\n\nconst client = new CloudFrontClient({ region: \"us-east-1\" });\n\nasync function createDistribution() {\n  // The KV store is defined via a Cache Policy that enables “Cache‑Based Origin Request”.\n  // In the console this appears as “Cache key and origin request settings”.\n  const cachePolicy = await client.send(\n    new CreateCachePolicyCommand({\n      CachePolicyConfig: {\n        Name: \"ClaudePromptCachePolicy\",\n        // We want the request body (the prompt) to be part of the cache key.\n        ParametersInCacheKeyAndForwardedToOrigin: {\n          EnableAcceptEncodingBrotli: false,\n          EnableAcceptEncodingGzip: false,\n          // The request body is not automatically part of the key,\n          // so we forward it to the origin (Lambda) where we will hash it.\n          // The KV store itself is accessed via the Function, not the cache policy.\n        },\n        DefaultTTL: 0, // we control TTL manually in the KV store\n        MaxTTL: 0,\n        MinTTL: 0,\n        // Important: Edge Functions can only read from KV if the cache behavior\n        // points to a \"function association\" with “viewer request”.\n        // The association is set later in the console or via the API.\n        // No extra code needed here.\n      },\n    })\n  );\n\n  // Omitted: create the origin (Lambda ARN) and the distribution.\n  // The key point for the beginner is that the distribution must have:\n  //   - a Viewer Request Function (our edge function)\n  //   - an Origin (our Lambda) for /chat\n  //   - the default behavior points to the Lambda.\n}\ncreateDistribution().catch(console.error);\n```\n\nKey takeaway:CloudFront Functions can read/write a built‑in KV store, but they can’t talk to the internet. They’re perfect for a quick “look‑aside” cache; the heavy lifting stays in Lambda.\n\n| Item | Why it matters |\n|---|---|\n2 MB code limit for Functions |\nKeep the script tiny; avoid large libraries. |\nNo outbound network |\nTrying to fetch Claude from the function returns 502. |\nCache invalidation delay (up to 60 s) |\nIf you delete a KV entry, the edge may still serve it for a minute. |\nOAI (Origin Access Identity) is deprecated |\nIf you still use it, CloudFront will log warnings but still work; plan to switch to Origin Access Control. |\n\nThe Lambda does three things:\n\n`fetch`\n\n.\n`@aws-sdk/client-cloudfront`\n\n`CreateKeyValueStore`\n\n‑style API).\n\n``` js\n// lambda-handler.ts\nimport { CloudFrontClient, PutKeyValueStoreCommand } from \"@aws-sdk/client-cloudfront\";\n\n// Claude endpoint – replace with your actual URL and API key.\nconst CLAUDE_ENDPOINT = \"https://api.anthropic.com/v1/complete\";\nconst CLAUDE_API_KEY = process.env.CLAUDE_API_KEY!;\n\n// TTL in seconds for the edge KV entry.\nconst KV_TTL = 300; // 5 minutes\n\nexport const handler = async (event: any) => {\n  // -------------------------------------------------\n  // 1️⃣ Extract the prompt from the incoming request.\n  // -------------------------------------------------\n  const body = JSON.parse(event.body);\n  const prompt = body.prompt?.trim();\n  if (!prompt) {\n    return {\n      statusCode: 400,\n      body: JSON.stringify({ error: \"Missing 'prompt' field\" }),\n    };\n  }\n\n  // -------------------------------------------------\n  // 2️⃣ Call Claude. (fetch is available in Node 20)\n  // -------------------------------------------------\n  const claudeResponse = await fetch(CLAUDE_ENDPOINT, {\n    method: \"POST\",\n    headers: {\n      \"Content-Type\": \"application/json\",\n      \"x-api-key\": CLAUDE_API_KEY,\n    },\n    body: JSON.stringify({\n      prompt,\n      max_tokens: 256,\n      temperature: 0.7,\n    }),\n  });\n\n  if (!claudeResponse.ok) {\n    const err = await claudeResponse.text();\n    return {\n      statusCode: claudeResponse.status,\n      body: JSON.stringify({ error: err }),\n    };\n  }\n\n  const claudeData = await claudeResponse.json();\n  const answer = claudeData.completion;\n\n  // -------------------------------------------------\n  // 3️⃣ Store prompt → answer in the Edge KV store.\n  // -------------------------------------------------\n  // The KV store lives on CloudFront, not Lambda, but the SDK\n  // lets us write to it from anywhere with the right permissions.\n  const cf = new CloudFrontClient({});\n  await cf.send(\n    new PutKeyValueStoreCommand({\n      // “ClaudePromptCache” is the name you gave the KV store\n      // when you created the distribution.\n      KeyValueStoreName: \"ClaudePromptCache\",\n      Items: [\n        {\n          Key: prompt, // exact prompt string as the key\n          Value: answer,\n          // TTL tells the edge when the entry expires.\n          // Edge will automatically drop it after 5 minutes.\n          // The attribute name is `TTL`, expressed in seconds.\n          TTL: KV_TTL,\n        },\n      ],\n    })\n  );\n\n  // -------------------------------------------------\n  // 4️⃣ Return the answer to the caller.\n  // -------------------------------------------------\n  return {\n    statusCode: 200,\n    headers: { \"Content-Type\": \"application/json\" },\n    body: JSON.stringify({ answer }),\n  };\n};\n```\n\n`PutKeyValueStoreCommand`\n\nThe CloudFront SDK treats the edge KV store like any other key‑value database: you give it a **key** (the prompt) and a **value** (the answer). The TTL ensures the entry disappears after the freshness window.\n\n| Gotcha | Explanation |\n|---|---|\nNode 22 `require(esm)` breakage |\nIf you upgrade the runtime, any `require` of an ES module will silently fail. Stick to `import` syntax or pin the runtime to Node 20. |\nSnapStart + VPC = no benefit |\nSnapStart speeds up cold starts only when the function has no VPC attachment. If you need VPC (e.g., to reach a private Claude endpoint), the startup time stays the same. |\nResponse streaming needs headers |\nIf you ever switch to streaming Claude’s output, you must set `Transfer-Encoding: chunked` and a matching `Content-Type` . Otherwise CloudFront buffers the whole response. |\nProvisioned Concurrency cost |\nEnabling it makes the function always warm, but you’ll be billed even when idle. For a low‑traffic chat app, on‑demand is cheaper. |\nLambda@Edge size limits |\nWe are NOT using Lambda@Edge here; we use a regular regional Lambda, so the 1 MB response limit does not apply. |\n\nRaw prompts can be long, and the KV store limits key length (max 256 bytes). A simple way to keep keys short is to hash the prompt with SHA‑256 and store the original prompt as metadata (optional).\n\n``` python\nimport crypto from \"crypto\";\n\nfunction hashPrompt(prompt: string): string {\n  // Create a hex‑encoded SHA‑256 hash of the prompt.\n  // This produces a fixed‑length, URL‑safe string.\n  return crypto.createHash(\"sha256\").update(prompt).digest(\"hex\");\n}\n```\n\nYou would then use `hashPrompt(prompt)`\n\nas the KV key, and optionally store the original prompt in the value (e.g., as a JSON object `{prompt, answer}`\n\n) for debugging.\n\nBecause the TTL is only five minutes, you rarely need manual invalidation. However, if you release a new model version or change the temperature, you may want to wipe the cache immediately.\n\n``` js\n// invalidate-cache.ts\nimport {\n  CloudFrontClient,\n  DeleteKeyValueStoreCommand,\n} from \"@aws-sdk/client-cloudfront\";\n\nconst client = new CloudFrontClient({});\n\nexport async function clearCache() {\n  await client.send(\n    new DeleteKeyValueStoreCommand({\n      KeyValueStoreName: \"ClaudePromptCache\",\n    })\n  );\n  console.log(\"Edge KV store cleared\");\n}\n```\n\nTip:Deleting the whole store is fast, but the edge may still serve stale data for up to 60 seconds due to propagation delay. If you need per‑key removal, you must overwrite the key with a very short TTL (e.g., 1 second) instead.\n\n| Gotcha | Fix |\n|---|---|\nKV write latency |\nWrites are asynchronous; the function returns to the caller before the edge fully propagates the entry. In practice the next request sees the entry within a few hundred milliseconds. |\nMaximum value size (10 KB) |\nClaude answers are usually small, but if you request large completions, truncate or compress before storing. |\nPermission errors |\nThe Lambda’s execution role must have `cloudfront:UpdateKeyValueStore` permission. Add a policy like `{\"Effect\":\"Allow\",\"Action\":\"cloudfront:UpdateKeyValueStore\",\"Resource\":\"*\"}.`\n|\n\nA CloudFront Function runs on the **viewer request** event. It can read from the KV store, and if it finds a hit, it can craft a response and stop further processing (i.e., skip the Lambda origin).\n\n```\n// cf-function.js\n/**\n * CloudFront Function – viewer request.\n * Looks for a cached answer to the incoming prompt.\n * If found, returns it immediately; otherwise lets the request\n * continue to the Lambda origin.\n */\n\nfunction handler(event) {\n  var request = event.request;\n  var uri = request.uri; // e.g., \"/chat\"\n  // We only care about POST /chat\n  if (uri !== \"/chat\" || request.method !== \"POST\") {\n    return request; // let everything else pass through\n  }\n\n  // -------------------------------------------------\n  // 1️⃣ Extract the request body (the prompt JSON).\n  // -------------------------------------------------\n  // CloudFront Functions can only read the body if the\n  // request is sent with the \"application/json\" content type.\n  var body = request.body && request.body.text;\n  if (!body) {\n    // No body – forward to Lambda for proper error handling.\n    return request;\n  }\n\n  var prompt;\n  try {\n    var parsed = JSON.parse(body);\n    prompt = parsed.prompt?.trim();\n    if (!prompt) throw \"missing\";\n  } catch (e) {\n    // Malformed JSON – forward to Lambda.\n    return request;\n  }\n\n  // -------------------------------------------------\n  // 2️⃣ Look up the prompt in the Edge KV store.\n  // -------------------------------------------------\n  // The KV store is called \"ClaudePromptCache\". The key is the\n  // raw prompt string (or a hash if you used one in Lambda).\n  var kvStore = \"ClaudePromptCache\";\n  var cached = KV.get(kvStore, prompt);\n  // KV.get returns null if the key does not exist or has expired.\n\n  if (cached) {\n    // -------------------------------------------------\n    // 3️⃣ Build a synthetic response with the cached answer.\n    // -------------------------------------------------\n    var response = {\n      statusCode: 200,\n      statusDescription: \"OK\",\n      headers: {\n        \"content-type\": { value: \"application/json\" },\n        \"cache-control\": { value: \"max-age=0, private\" },\n      },\n      body: JSON.stringify({ answer: cached }),\n    };\n    // Returning a response short‑circuits the rest of the\n    // CloudFront pipeline – Lambda never runs.\n    return response;\n  }\n\n  // -------------------------------------------------\n  // 4️⃣ No cache hit → let the request travel to Lambda.\n  // -------------------------------------------------\n  return request;\n}\n```\n\n`/chat`\n\nPOST requests.\n`KV.get(store, key)`\n\nreads from the edge store. No network call, just a local memory read.\n\nAnalogy:Imagine a library with a “quick‑look” shelf at the front desk. If the book you need is already on that shelf, the librarian hands it to you right away. If not, you have to go to the back rooms (the Lambda) to fetch it.\n\nCloudFront Functions are deployed with the `@aws-sdk/client-cloudfront`\n\n`CreateFunctionCommand`\n\n. Example (run locally):\n\n```\nnpm install @aws-sdk/client-cloudfront\njs\n// deploy-function.ts\nimport {\n  CloudFrontClient,\n  CreateFunctionCommand,\n  PublishFunctionCommand,\n} from \"@aws-sdk/client-cloudfront\";\nimport * as fs from \"fs\";\n\nconst client = new CloudFrontClient({ region: \"us-east-1\" });\n\nasync function deploy() {\n  const code = fs.readFileSync(\"cf-function.js\", \"utf8\");\n  const create = await client.send(\n    new CreateFunctionCommand({\n      Name: \"ClaudePromptCacheFunction\",\n      FunctionConfig: {\n        Comment: \"Look up Claude prompt cache at the edge\",\n        Runtime: \"cloudfront-js-1.0\", // the only supported runtime\n      },\n      FunctionCode: Buffer.from(code),\n    })\n  );\n  console.log(\"Created:\", create.FunctionSummary?.ARN);\n\n  // Publishing makes the function live.\n  const publish = await client.send(\n    new PublishFunctionCommand({\n      Name: \"ClaudePromptCacheFunction\",\n      IfMatch: create.ETag, // ensures we publish the right version\n    })\n  );\n  console.log(\"Published version:\", publish.FunctionSummary?.FunctionMetadata?.FunctionARN);\n}\ndeploy().catch(console.error);\n```\n\nAfter publishing, attach the function to the **Viewer Request** event of the `/chat`\n\ncache behavior in the CloudFront console or via the API.\n\n| Gotcha | Fix |\n|---|---|\nNo outbound HTTP |\nNever try `fetch` inside the function; you’ll see a 502 error. |\n2 MB size limit |\nKeep the code tiny, avoid bundlers, use native `JSON.parse` . |\nOnly simple headers |\nFunctions can set headers but can’t manipulate cookies beyond simple key‑value pairs. |\nPropagation delay (up to 60 s) |\nAfter a new version is published, give the distribution a minute before testing. |\n\nWhat you now have:a three‑piece pipeline that reduces the perceived latency of Claude calls from hundreds of milliseconds to a few milliseconds for repeat prompts.\n\nNow you can add prompt caching to any LLM‑backed feature—chatbots, code assistants, or summarization services—and give your users an experience that feels *instant*, even when the underlying model lives far away. Happy building!\n\nTransparency noticeThis article was written with the help of an AI system —\n\n[Groq](GPT OSS 120B).\n\nPublished:2026-08-27 ·Primary focus:CloudFrontAll code blocks are intended to be correct and runnable, but please verify them\n\nagainst the official docs for the tools mentioned before using in production.\n\nFind an error? Drop a comment — corrections are always welcome.", "url": "https://wpnews.pro/news/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up", "canonical_source": "https://dev.to/dineshgowtham/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up-claude-calls-cbl", "published_at": "2026-08-27 21:06:37+00:00", "updated_at": "2026-08-27 21:49:29.522588+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "large-language-models"], "entities": ["CloudFront", "Lambda", "Claude", "AWS"], "alternates": {"html": "https://wpnews.pro/news/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up", "markdown": "https://wpnews.pro/news/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up.md", "text": "https://wpnews.pro/news/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up.txt", "jsonld": "https://wpnews.pro/news/prompt-caching-at-the-edge-using-cloudfront-functions-and-lambda-to-speed-up.jsonld"}}