{"slug": "the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories", "title": "The SSE Fragmentation Catastrophe That Took Down CareerPilot AI (Smash Stories)", "summary": "A developer's CareerPilot AI app crashed in production due to Server-Sent Events fragmentation on Google Cloud Run. The SSE stream broke large JSON payloads across TCP packets, causing client-side parse errors. Forcing flush on each chunk worsened the issue by triggering packet fragmentation.", "body_md": "*This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.*\n\nIt was 11:14 PM.\n\nMy friend DM'd me on Twitter: *\"Your app just hung for 30 seconds, spun indefinitely, and then completely died.\"*\n\nI opened the browser DevTools console pointed at production and saw it — the screen flooded in red:\n\n```\nGET https://careerpilot-ai.run.app/api/analyze-career\nnet::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK)\n```\n\nThe Server-Sent Events stream powering [CareerPilot AI](https://github.com/aashitanegii/-CareerPilotAI) was systematically collapsing on Google Cloud Run. And I had no idea why.\n\nLocally on `localhost:3000`\n\n, the agentic pipeline was a masterpiece. The multi-stage reasoning logs streamed gracefully — Step 1 flowed into Step 6, the final structured JSON payload arrived within seconds, the UI lit up with a complete personalized career roadmap. Beautiful.\n\nBut once deployed behind Google's Front End (GFE) proxy, the pipeline was a graveyard of broken sockets.\n\nCareerPilot AI runs a six-stage agentic pipeline on every career analysis request. Instead of firing a single long-running prompt to Gemini and making the user stare at a blank screen for 20+ seconds, we designed a Server-Sent Events logging stream to broadcast real-time reasoning steps directly to the browser — giving the interface the feel of a live, active mentor thinking out loud.\n\nOnce the final stage (Self-Evaluation & Constraint Validation) completed, the backend constructed a massive, nested 15KB JSON payload containing the personalized roadmap: skill weightings, role benchmarks, resource links, and a 30-day milestone calendar.\n\nHere was the delivery mechanism — and the landmine hiding inside it:\n\n```\n// server.ts — The vulnerable streaming channel\napp.get(\"/api/analyze-career\", async (req, res) => {\n  res.setHeader(\"Content-Type\", \"text/event-stream\");\n  res.setHeader(\"Cache-Control\", \"no-cache\");\n  res.setHeader(\"Connection\", \"keep-alive\");\n\n  // Stream intermediate reasoning logs per step\n  for (let step = 1; step <= 6; step++) {\n    const log = await executeAgentStep(step);\n    res.write(`data: ${JSON.stringify({ step, text: log })}\\n\\n`);\n  }\n\n  // The fatal moment: sending a 15KB structured JSON payload\n  res.write(`data: ${JSON.stringify({ finalResult })}\\n\\n`);\n  res.end();\n});\n```\n\nEverything looked fine. Until it wasn't.\n\nOur first instinct was reasonable: the Express response buffer or the compression middleware was throttling chunks and causing a timeout. The fix seemed obvious — force-flush every payload as soon as it was written.\n\nWe imported the compression middleware and started manually calling `res.flush()`\n\nafter every single log line:\n\n```\n// The \"intuitive\" (but catastrophic) mistake\nres.write(`data: ${JSON.stringify({ step, text: log })}\\n\\n`);\nres.flush(); // Force-flushed chunks into arbitrary TCP packet boundaries!\n```\n\nIt made things infinitely worse.\n\nInstead of hanging for 30 seconds and then failing, the client now crashed immediately on almost every single attempt. By aggressively forcing flushes on every minor string, we triggered something far uglier: violent TCP packet fragmentation.\n\nThe large final JSON payload was being split mid-byte across arbitrary network packet boundaries. Our client-side reader, which naively assumed that each incoming SSE chunk was a complete, fully-formed JSON object, was receiving this:\n\n```\n// Chunk 1 — Mutilated\n{\"finalResult\":{\"skills\":[{\"name\":\"Python\",\"category\":\"foun\n\n// Chunk 2 — The rest, arriving milliseconds later\ndational\",\"matched\":true}],\"weeklyPlan\":[]}}\n```\n\n`JSON.parse()`\n\nthrew a `SyntaxError: Unexpected end of JSON input`\n\non Chunk 1. The entire React UI state machine shattered. The app froze. The user saw a broken loading screen.\n\nWe had fixed a timeout by turning it into an instant crash. Brilliant.\n\nThat's when it clicked — we weren't fighting one bug. We were fighting two completely separate infrastructure forces compounding each other in production:\n\n**1. Proxy Buffering**\n\n**Google Cloud Run's GFE proxy holds small SSE packets in a buffer waiting for more data before flushing downstream.** Unless explicitly instructed otherwise, it optimizes throughput by batching chunks — not passing them through immediately. This explained the initial 30-second hang: the proxy was collecting all six of our log lines and sitting on them before sending anything to the client.\n\n**2. TCP Stream Fragmentation**\n\n**An SSE stream is not a guaranteed series of atomic messages — it is a continuous raw byte stream.** When sending a 15KB payload, the network transport layer is fully entitled — and will — split that across multiple TCP packets. The frontend must treat incoming data as a byte accumulator, not a ready-to-parse event queue.\n\nWe had fixed neither. Our flush attempt had only worsened the second while accidentally bypassing the first.\n\nThe solution required changes on both sides of the wire simultaneously.\n\nWe added the HTTP header that explicitly instructs modern reverse proxies — including Nginx and GFE — to pass bytes through immediately without buffering:\n\n```\n// server.ts — Patched SSE initialization\nres.writeHead(200, {\n  \"Content-Type\": \"text/event-stream\",\n  \"Cache-Control\": \"no-cache, no-transform\", // 'no-transform' blocks compression proxies from buffering\n  \"Connection\": \"keep-alive\",\n  \"X-Accel-Buffering\": \"no\"                  // Explicit bypass for Nginx/GFE\n});\n```\n\nWe also disabled global Gzip compression specifically for the `/api/analyze-career`\n\nroute, ensuring the compression layer wasn't holding bytes in its own internal buffer before the proxy even saw them.\n\n`onmessage`\n\n, Build a Byte Accumulator\nWe abandoned the naive `EventSource.onmessage`\n\nlistener entirely. Instead, we wrote a low-level stream reader that accumulates incoming bytes in a staging buffer and only processes complete SSE frames — identified by double-newline (`\\n\\n`\n\n) boundaries — never attempting to parse a fragment:\n\n``` js\n// src/App.tsx — Boundary-safe chunk reassembly\nconst reader = response.body.getReader();\nconst decoder = new TextDecoder();\nlet buffer = \"\";\n\nwhile (true) {\n  const { value, done } = await reader.read();\n  if (done) break;\n\n  buffer += decoder.decode(value, { stream: true });\n  const lines = buffer.split(\"\\n\\n\");\n\n  // Keep the last, potentially incomplete fragment staged in the buffer\n  buffer = lines.pop() || \"\";\n\n  for (const line of lines) {\n    if (!line.startsWith(\"data: \")) continue;\n    const jsonStr = line.replace(\"data: \", \"\").trim();\n    try {\n      const parsed = JSON.parse(jsonStr);\n      // Safe: only complete, boundary-verified JSON ever reaches state\n    } catch (e) {\n      // Fragment guard: push back and wait for remaining bytes\n      console.warn(\"Partial chunk detected — staging for next read cycle.\");\n    }\n  }\n}\n```\n\nThe key insight: `lines.pop()`\n\npulls the trailing incomplete fragment out of the processing loop and holds it in `buffer`\n\nuntil the next network read brings the rest of the bytes. No more mid-payload parsing. No more crashes.\n\nAfter pushing both fixes to production, we ran a load test simulating 100 concurrent users hitting the full six-stage agentic pipeline simultaneously.\n\nThe numbers were stark:\n\n| Metric | Before | After |\n|---|---|---|\n| Pipeline Completion Rate | 67% | 99.2% |\n| Time-to-First Log (FCP) | 28 seconds | 1.1 seconds |\n| JSON Syntax Crashes | Constant | Zero |\n\nI DM'd the user back at 1:30 AM. They tested it again and replied: *\"okay yeah this is actually really cool now.\"*\n\nThe real lesson here wasn't about SSE or TCP fragmentation specifically. It was about the gap between local development and containerized production.\n\nLocally, you are your own proxy. There is no GFE, no Nginx, no buffering layer between your Node process and the browser. Every byte arrives instantly. Every test passes. Everything feels perfect.\n\nThe moment you deploy behind a reverse proxy, your assumptions about when data arrives collapse. Streaming architectures that rely on chunk atomicity — SSE, WebSockets, chunked transfer encoding — need to be designed defensively from the first line of code, not patched reactively at 11 PM when a user DM slides in.\n\nBuild your readers as byte accumulators. Treat every incoming chunk as potentially incomplete. Use `X-Accel-Buffering: no`\n\nfrom day one on any SSE endpoint that goes near a proxy. And test in an environment that actually mirrors production — because localhost will lie to you every time.\n\n*CareerPilot AI is an open-source AI-powered career roadmap agent. You can explore the full codebase at github.com/aashitanegii/-CareerPilotAI and try the live app at careerpilot-ai-667889155113.asia-southeast1.run.app.*", "url": "https://wpnews.pro/news/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories", "canonical_source": "https://dev.to/aashitanegii/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories-16f2", "published_at": "2026-07-15 09:51:27+00:00", "updated_at": "2026-07-15 09:58:28.716715+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "artificial-intelligence"], "entities": ["CareerPilot AI", "Google Cloud Run", "Gemini", "Express", "React", "Sentry", "aashitanegii"], "alternates": {"html": "https://wpnews.pro/news/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories", "markdown": "https://wpnews.pro/news/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories.md", "text": "https://wpnews.pro/news/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories.txt", "jsonld": "https://wpnews.pro/news/the-sse-fragmentation-catastrophe-that-took-down-careerpilot-ai-smash-stories.jsonld"}}