{"slug": "streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end", "title": "Streaming LLM output through a Next.js proxy: why you only get stream_end", "summary": "Ali Ahmed documented the specific configuration needed to stream LLM Server-Sent Events through a Next.js route handler without buffering, drawn from building Turbo Alan Refiner, a multi-pass LLM text-refinement app. The setup requires the Node.js runtime, force-dynamic rendering, no-cache/no-transform and X-Accel-Buffering: no headers, byte-by-byte forwarding via a ReadableStream reader, and a client that buffers incomplete lines and treats only terminal events as the end. Ahmed notes that gzip-compressing proxies reintroduce buffering and recommends DevTools' EventStream view to distinguish client-side from upstream buffering.", "body_md": "When you proxy an LLM backend's **Server-Sent Events (SSE)** through a Next.js route handler, the usual failure is that nothing arrives until the end, and then everything arrives at once, or the client only ever sees the final `stream_end`. Something between the backend and the browser is **buffering**. The fix is a handful of specific things: the Node.js runtime, `dynamic = 'force-dynamic'`, the right headers (` Cache-Control: no-cache, no-transform` and `X-Accel-Buffering: no`), passing bytes through as they arrive, and a client that parses SSE frames properly. I learned these building **Turbo Alan Refiner**, a multi-pass LLM text-refinement app ([frontend source](https://github.com/QuantumBreakz/TurboAlanRefinerFrontend)).\n\nRouting every backend call through `/api/*` route handlers avoids CORS, keeps tokens server-side, and lets you normalise the request. The cost is that your route is now a hop in a streaming pipeline, and every hop is a chance to buffer.\n\n``` ts title=\"Route segment config\"\n\nexport const runtime = \"nodejs\";\n\nexport const dynamic = \"force-dynamic\";\n\nexport const maxDuration = 300;\n\n```\n* `runtime = \"nodejs\"` gives full streaming support without edge-runtime limits.\n* `dynamic = \"force-dynamic\"` stops Next.js caching or pre-rendering the route.\n* `maxDuration = 300` lets long refinements outlive the default function timeout.\n\nThen return a `ReadableStream` that forwards the upstream body as it arrives, with these headers:\n\n``` ts title=\"Streaming response headers\"\nreturn new Response(stream, {\n  headers: {\n    \"Content-Type\": \"text/event-stream\",\n    \"Cache-Control\": \"no-cache, no-transform\",\n    Connection: \"keep-alive\",\n    \"X-Accel-Buffering\": \"no\",\n  },\n});\n```\n\n`X-Accel-Buffering: no` tells nginx-style proxies not to buffer.\n\n`no-transform` matters more than it looks. A proxy that gzips the response has to collect bytes before it can compress them, which is exactly the buffering you're trying to avoid.\n\n`getReader()` and decode with a streaming `TextDecoder`.` await upstream.text()`, which waits for the whole response.` data: {...}` and your proxy prefixes `data:` again, the client receives `data: data: {...}`.` data: {\"type\":\"error\",...}` frame so the UI can show it, instead of the stream just stopping.`AbortController` so closing the tab cancels the upstream request, rather than leaving an LLM job running for nobody.\nUsually one of three things:\n\n**Something buffered the stream** (missing headers, the wrong runtime, a proxy in front), so all the events arrived together and the UI only rendered the last one.\n\n**The client parsed chunks, not lines.** A network chunk can end in the middle of an event. Keep the incomplete tail in a buffer:\n\n``` ts title=\"Client: keep the incomplete last line\"\n\n   buffer += decoder.decode(chunk, { stream: true });\n\n   const lines = buffer.split(\"\\n\");\n\n   buffer = lines.pop() ?? \"\";\n\n```\n3. **The client stopped too early.** Treat only true terminal events (`complete`, `error`) as the end. Keep processing everything already buffered before you close.\n\nBe lenient about input, too: strip a doubled `data: data: ` prefix, and accept raw JSON lines as well as SSE-formatted ones.\n\n## How do you debug it?\n\nOpen DevTools → Network → the streaming request → **EventStream**. If events show up one by one there but the UI updates all at once, the bug is in your client. If they all show up together, something upstream is buffering: check the headers, the runtime, and any proxy or CDN in front.\n\nRelated: [what a serverless cold start really is](https://aliahmed.si/blog/serverless-cold-starts-explained/) (long-lived streams and function timeouts run into the same platform limits).\n\n---\n\nWritten by Ali Ahmed · [Original article](https://aliahmed.si/blog/nextjs-sse-proxy-streaming/)\n```\n\n", "url": "https://wpnews.pro/news/streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end", "canonical_source": "https://dev.to/ali_ahmed_44ee99051e327ce/streaming-llm-output-through-a-nextjs-proxy-why-you-only-get-streamend-23j3", "published_at": "2026-11-05 22:31:47+00:00", "updated_at": "2026-11-05 22:53:04.490507+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-products"], "entities": ["Ali Ahmed", "Next.js", "Turbo Alan Refiner", "Node.js", "nginx"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end", "markdown": "https://wpnews.pro/news/streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end.md", "text": "https://wpnews.pro/news/streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end.txt", "jsonld": "https://wpnews.pro/news/streaming-llm-output-through-a-next-js-proxy-why-you-only-get-stream-end.jsonld"}}