{"slug": "streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained", "title": "Streaming Claude Tokens with Fastify on Lambda: A Step‑By‑Step Guide Explained Simply", "summary": "A developer published a step-by-step guide showing how to stream Claude tokens to the browser using a minimal Fastify service on AWS Lambda, calling Bedrock through Node.js 22's native fetch and forwarding tokens as Server-Sent Events. The approach avoids Express middleware overhead and heavy HTTP client dependencies to cut per-token latency and Lambda cold-start time.", "body_md": "LLM developers assume they need a heavyweight framework to handle streaming responses, but Fastify’s tiny core can proxy Claude’s token stream with almost zero overhead. In this post you’ll see how to turn a Fastify service into a real‑time chat endpoint that streams tokens directly to the browser.\n\nWhen you ask a language model (LLM) like Claude to generate text, it often returns **tokens** (the smallest pieces of language) one by one. To give a user a “live‑typing” feel you need to forward each token to the browser as soon as it arrives.  \n\n**Express**—the long‑time favorite of Node developers—adds a lot of middleware layers. Those layers are great for flexibility, but each extra function call adds a tiny amount of latency. When you are forwarding dozens of tokens per second, that latency compounds and you can notice a pause between the model’s output and what the user sees.\n\n**Fastify** is built around *performance first* principles:\n\nThink of Fastify as a **high‑speed conveyor belt** that moves a package (the token) from the model to the client with almost no friction, whereas Express is more like a **busy hallway** where the same package has to stop at several doors before continuing. For a streaming chat endpoint, that difference is noticeable.\n\n**In plain English** – Fastify’s thin design means less time waiting inside the server, so the user sees tokens almost as soon as Claude produces them.\n\n``` python\n// src/server.ts\nimport Fastify from \"fastify\";\n\n// Create a Fastify instance with the smallest possible options\nconst fastify = Fastify({\n  logger: true,               // helpful logs, can be turned off later\n  // No plugins loaded – we keep the core tiny\n});\n\nfastify.get(\"/\", async (request, reply) => {\n  // A simple health‑check endpoint\n  return { status: \"ok\" };\n});\n\n// Export the server so Lambda can import it later\nexport default fastify;\n```\n\n`Fastify()` creates the server.\n`logger: true` prints request details – useful while developing.\nNode.js 22 ships with a **native `fetch` implementation** that works like the browser’s `fetch`. That means we can call AWS Bedrock (the service that hosts Claude) without pulling in a heavy HTTP client library. Using native `fetch` keeps the bundle size tiny, which is important for Lambda where every kilobyte counts toward cold‑start time.\n\n**Why avoid extra libraries?**\n\nEvery additional dependency must be zipped, uploaded, and unpacked when Lambda starts. The more code you ship, the longer the *cold start* (the first time your function runs after being idle). Native `fetch` eliminates that extra work.\n\n``` js\n// src/routes/chat.ts\nimport { FastifyInstance, FastifyReply, FastifyRequest } from \"fastify\";\n\nexport async function chatRoute(fastify: FastifyInstance) {\n  // Register a POST /chat endpoint\n  fastify.post(\"/chat\", async (request: FastifyRequest, reply: FastifyReply) => {\n    // Parse the incoming JSON body { prompt: \"...\" }\n    const { prompt } = request.body as { prompt: string };\n    if (!prompt) {\n      reply.status(400).send({ error: \"Prompt is required\" });\n      return;\n    }\n\n    // Call the helper that talks to Bedrock (defined later)\n    const stream = await invokeClaudeStream(prompt);\n\n    // Set up Server‑Sent Events (SSE) headers – see next section for details\n    reply.raw.writeHead(200, {\n      \"Content-Type\": \"text/event-stream\", // tells browser to treat data as a stream\n      \"Cache-Control\": \"no-cache\",\n      Connection: \"keep-alive\",\n    });\n\n    // Pipe each token from Bedrock to the client as an SSE line\n    for await (const token of stream) {\n      // SSE format: each message starts with \"data:\" and ends with two newlines\n      reply.raw.write(`data: ${JSON.stringify({ token })}\\n\\n`);\n    }\n\n    // Close the SSE stream when Bedrock finishes\n    reply.raw.end();\n  });\n}\n```\n\n`request.body` holds the JSON payload the browser sends.\n`invokeClaudeStream` (implemented later) returns an `reply.raw` gives us access to the underlying Node `http.ServerResponse` so we can write raw SSE data.\n**Tip** – Keep `fetch` calls and response handling inside the route *as short as possible*. Anything that can be moved to a helper function reduces the amount of code that runs during each request, making the Lambda cold start faster.\n\n**Bedrock** is Amazon’s managed service that hosts LLMs like Claude. The SDK package we use is `@aws-sdk/client-bedrock-runtime`. Inside the SDK there is a method called `InvokeModelCommand` that can be asked to **stream** the model’s output instead of waiting for the whole answer.\n\nImagine Claude is a storyteller reading a book aloud. With normal (non‑streaming) calls, you wait until the storyteller finishes the whole chapter before you hear anything. With streaming, you hear each sentence as soon as it is spoken. In technical terms, the service sends a **chunked HTTP response** where each chunk contains the next token.\n\n``` js\n// src/bedrock.ts\nimport {\n  BedrockRuntimeClient,\n  InvokeModelCommand,\n} from \"@aws-sdk/client-bedrock-runtime\";\nimport { Readable } from \"stream\";\n\n// Create a client that uses the default Lambda credentials\nconst bedrock = new BedrockRuntimeClient({});\n\n// Helper that returns an async iterable of tokens\nexport async function* invokeClaudeStream(prompt: string): AsyncGenerator<string> {\n  // Build the request payload according to Claude's API spec\n  const body = JSON.stringify({\n    prompt,\n    // Tell Claude we want a streaming response\n    stream: true,\n    // Optional: max tokens, temperature, etc.\n  });\n\n  // The command carries the model ID and the payload\n  const command = new InvokeModelCommand({\n    modelId: \"anthropic.claude-v2\", // replace with the exact model you use\n    contentType: \"application/json\",\n    accept: \"application/json\",\n    body,\n  });\n\n  // Invoke the model – Bedrock returns a Readable stream of bytes\n  const response = await bedrock.send(command);\n\n  // The response.body is a Node.js Readable stream; we convert it to async iterator\n  const stream = response.body as Readable;\n\n  // The raw bytes are newline‑delimited JSON fragments; each contains a token\n  const decoder = new TextDecoder(\"utf-8\");\n  let buffer = \"\";\n\n  for await (const chunk of stream) {\n    // Convert Uint8Array chunk to string and accumulate\n    buffer += decoder.decode(chunk, { stream: true });\n\n    // Split on newline – each line is a complete JSON message\n    const lines = buffer.split(\"\\n\");\n    // Keep the last partial line in buffer for the next iteration\n    buffer = lines.pop() ?? \"\";\n\n    for (const line of lines) {\n      if (!line.trim()) continue; // skip empty lines\n      const data = JSON.parse(line);\n      // The token appears under `completion` in Claude's streaming format\n      yield data.completion;\n    }\n  }\n\n  // After the stream ends, flush any remaining buffered data\n  if (buffer) {\n    const data = JSON.parse(buffer);\n    yield data.completion;\n  }\n}\n```\n\n`BedrockRuntimeClient` reads credentials from the Lambda execution role, so we don’t embed keys.\n`InvokeModelCommand` tells Bedrock which model to run and that we want a streaming response (`stream: true`).\n`for await` over it.\n`completion` field containing the next token.\n**In plain English** – We ask Bedrock to talk to us one word at a time, turn each word into a JavaScript value, and hand it over to Fastify so the browser can see it instantly.\n\nBackpressure is a fancy way of saying “don’t overwhelm the client”. When the browser can’t read data quickly enough, Node’s stream will automatically pause reading from Bedrock until the socket drains. Because we use `for await` on the async generator, the `await` implicitly respects that pause, so we don’t need extra flow‑control code.\n\nAWS Lambda runs your code inside a sandboxed container. By default, Lambda **buffers the whole HTTP response** before sending it to the caller. That behavior defeats streaming – the client will only get the final payload after the function finishes. To make streaming work we must:\n\n`Content-Type` to `text/event-stream` and add the header `X-Amz-Content-Type` with the same value.\n`reply.send()` after writing data; instead write directly to `callbackWaitsForEmptyEventLoop = false`\n\n``` python\n// src/lambda.ts\nimport fastify from \"./server\";\nimport { chatRoute } from \"./routes/chat\";\n\n// Register the chat route on the shared Fastify instance\nawait chatRoute(fastify);\n\n// Export the handler that Lambda will call\nexport const handler = async (event: any, context: any) => {\n  // Prevent Lambda from waiting for the event loop to be empty\n  context.callbackWaitsForEmptyEventLoop = false;\n\n  // Fastify can handle raw API Gateway events via the \"aws-lambda-fastify\" shim.\n  // We avoid an extra package by using Fastify’s built‑in `server.inject`.\n  const result = await fastify.inject({\n    method: event.httpMethod,\n    url: event.path,\n    headers: event.headers,\n    payload: event.body,\n    // The `raw` field gives us direct access to the Node response object\n    // which Fastify already wrote to in the /chat route.\n  });\n\n  // Return the shape API Gateway expects\n  return {\n    statusCode: result.statusCode,\n    headers: result.headers,\n    // For streaming we must return the raw body stream; API Gateway v2 supports this.\n    // If you use the older REST API, you’ll need to enable “Lambda proxy integration”.\n    body: result.body,\n    isBase64Encoded: false,\n  };\n};\n```\n\n`context.callbackWaitsForEmptyEventLoop = false;` disables the default buffering behavior.\n`fastify.inject` simulates an HTTP request without opening a network socket, which works nicely inside Lambda because we already have the request data from API Gateway.\n\n```\n// infra/cdk-stack.ts\nimport * as cdk from \"aws-cdk-lib\";\nimport * as lambda from \"aws-cdk-lib/aws-lambda\";\nimport * as apigateway from \"aws-cdk-lib/aws-apigatewayv2\";\n\nexport class ClaudeStreamingStack extends cdk.Stack {\n  constructor(scope: cdk.App, id: string) {\n    super(scope, id);\n\n    const fn = new lambda.Function(this, \"FastifyClaudeFn\", {\n      runtime: lambda.Runtime.NODEJS_22_X,\n      handler: \"src/lambda.handler\",\n      code: lambda.Code.fromAsset(\"dist\"), // compiled TypeScript output\n      memorySize: 1024, // give enough RAM for Bedrock client\n      timeout: cdk.Duration.seconds(30),\n      environment: {\n        // Bedrock region, etc.\n        AWS_REGION: \"us-east-1\",\n      },\n    });\n\n    // API Gateway HTTP API (v2) with Lambda integration\n    const api = new apigateway.HttpApi(this, \"ClaudeApi\", {\n      createDefaultStage: true,\n    });\n\n    api.addRoutes({\n      path: \"/chat\",\n      methods: [apigateway.HttpMethod.POST],\n      integration: new apigateway.LambdaProxyIntegration({ handler: fn }),\n    });\n  }\n}\n```\n\n**Gotcha** – Lambda’s **response streaming** works only with the **HTTP API (v2)**, not the older REST API. Forgetting this will make the browser receive a single JSON blob instead of a live token stream.\n\nIf you decide to add a custom layer (e.g., for shared utilities) note that **Node 22’s ESM loader (`require(esm)`) can silently break** when the layer is compiled for CommonJS. The safest route is to keep everything in native ESM (`.mjs` or `\"type\":\"module\"` in `package.json`) or avoid the `esm` shim entirely.\n\nTesting streaming is a little different from a normal API call because you need to read data as it arrives. The easiest way is to use **curl** with the `--no-buffer` flag or a small Node script that consumes the SSE stream.\n\n```\ncurl -N -X POST https://<api-id>.execute-api.<region>.amazonaws.com/chat \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"prompt\":\"Tell me a short story about a robot.\"}'\n```\n\n`-N` disables curl’s internal buffering, letting you see each `data:` line instantly.\n\n```\ndata: {\"token\":\"Once\"}\ndata: {\"token\":\" upon\"}\ndata: {\"token\":\" a\"}\n...\npython\n// test/stream-client.js\nimport fetch from \"node-fetch\";\n\nasync function run() {\n  const response = await fetch(\"https://<api-id>.execute-api.<region>.amazonaws.com/chat\", {\n    method: \"POST\",\n    headers: { \"Content-Type\": \"application/json\" },\n    body: JSON.stringify({ prompt: \"Explain quantum computing in one sentence.\" }),\n  });\n\n  // Ensure we get an SSE stream\n  if (!response.ok || !response.headers.get(\"content-type\")?.includes(\"text/event-stream\")) {\n    console.error(\"Not a streaming response\");\n    return;\n  }\n\n  // Read the stream line‑by‑line\n  const reader = response.body.getReader();\n  const decoder = new TextDecoder();\n  let buffer = \"\";\n\n  while (true) {\n    const { done, value } = await reader.read();\n    if (done) break;\n    buffer += decoder.decode(value, { stream: true });\n    const lines = buffer.split(\"\\n\");\n    buffer = lines.pop() ?? \"\";\n\n    for (const line of lines) {\n      if (line.startsWith(\"data:\")) {\n        const payload = JSON.parse(line.slice(5).trim());\n        process.stdout.write(payload.token);\n      }\n    }\n  }\n\n  console.log(\"\\n--- stream finished ---\");\n}\n\nrun();\n```\n\n`/chat` endpoint, checks that the `process.stdout.write` prints tokens without a newline, mimicking the “live typing” effect in a terminal.\n**Tip** – When you test locally (e.g., `npm run dev` with Fastify listening on `localhost`), you can skip the Lambda layer and call the route directly. The same SSE headers work, so the client code stays identical.\n\nIf you forget to set `callbackWaitsForEmptyEventLoop = false`, the client will receive **nothing until the model finishes**. You’ll see a single huge JSON payload instead of many `data:` lines. Open the CloudWatch logs for your Lambda – you’ll notice a line like `REPORT RequestId: … Duration: … Billed Duration: …` **after** the entire response is sent. Once the flag is set, the log will show the function ending **while the connection remains open**, confirming streaming works.\n\n**What you now know** \n\n`fetch` in Node 22 removes the need for extra HTTP libraries, shrinking Lambda bundle size and cold‑start time.\n`await` each token; you don’t need manual flow‑control code.\n`esm` shim that breaks under Node 22.\nArmed with these pieces, you can ship a production‑grade, real‑time chat endpoint that feels as responsive as typing in a messenger, all while staying within the modest cost envelope of a single Lambda function. Happy streaming!\n\n**Transparency notice**\n\nThis article was written with the help of an AI system — [Groq](https://groq.com) (GPT OSS 120B).\n\n**Published:** 2026-11-04 · **Primary focus:** Fastify\n\nAll code blocks are intended to be correct and runnable, but please verify them\n\nagainst the official docs for the tools mentioned before using in production.\n\n*Find an error? Drop a comment — corrections are always welcome.*", "url": "https://wpnews.pro/news/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained", "canonical_source": "https://dev.to/dineshgowtham/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained-simply-4ini", "published_at": "2026-11-04 09:40:12+00:00", "updated_at": "2026-11-04 09:52:25.872704+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "developer-tools", "ai-tools"], "entities": ["Fastify", "Claude", "AWS Lambda", "AWS Bedrock", "Express", "Node.js"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained", "markdown": "https://wpnews.pro/news/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained.md", "text": "https://wpnews.pro/news/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained.txt", "jsonld": "https://wpnews.pro/news/streaming-claude-tokens-with-fastify-on-lambda-a-step-by-step-guide-explained.jsonld"}}