The SSE Fragmentation Catastrophe That Took Down CareerPilot AI (Smash Stories)

A developer's CareerPilot AI app crashed in production due to Server-Sent Events fragmentation on Google Cloud Run. The SSE stream broke large JSON payloads across TCP packets, causing client-side parse errors. Forcing flush on each chunk worsened the issue by triggering packet fragmentation.

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry. It was 11:14 PM. My friend DM'd me on Twitter: "Your app just hung for 30 seconds, spun indefinitely, and then completely died." I opened the browser DevTools console pointed at production and saw it — the screen flooded in red: GET https://careerpilot-ai.run.app/api/analyze-career net::ERR INCOMPLETE CHUNKED ENCODING 200 OK The Server-Sent Events stream powering CareerPilot AI https://github.com/aashitanegii/-CareerPilotAI was systematically collapsing on Google Cloud Run. And I had no idea why. Locally on localhost:3000 , the agentic pipeline was a masterpiece. The multi-stage reasoning logs streamed gracefully — Step 1 flowed into Step 6, the final structured JSON payload arrived within seconds, the UI lit up with a complete personalized career roadmap. Beautiful. But once deployed behind Google's Front End GFE proxy, the pipeline was a graveyard of broken sockets. CareerPilot AI runs a six-stage agentic pipeline on every career analysis request. Instead of firing a single long-running prompt to Gemini and making the user stare at a blank screen for 20+ seconds, we designed a Server-Sent Events logging stream to broadcast real-time reasoning steps directly to the browser — giving the interface the feel of a live, active mentor thinking out loud. Once the final stage Self-Evaluation & Constraint Validation completed, the backend constructed a massive, nested 15KB JSON payload containing the personalized roadmap: skill weightings, role benchmarks, resource links, and a 30-day milestone calendar. Here was the delivery mechanism — and the landmine hiding inside it: // server.ts — The vulnerable streaming channel app.get "/api/analyze-career", async req, res = { res.setHeader "Content-Type", "text/event-stream" ; res.setHeader "Cache-Control", "no-cache" ; res.setHeader "Connection", "keep-alive" ; // Stream intermediate reasoning logs per step for let step = 1; step <= 6; step++ { const log = await executeAgentStep step ; res.write data: ${JSON.stringify { step, text: log } }\n\n ; } // The fatal moment: sending a 15KB structured JSON payload res.write data: ${JSON.stringify { finalResult } }\n\n ; res.end ; } ; Everything looked fine. Until it wasn't. Our first instinct was reasonable: the Express response buffer or the compression middleware was throttling chunks and causing a timeout. The fix seemed obvious — force-flush every payload as soon as it was written. We imported the compression middleware and started manually calling res.flush after every single log line: // The "intuitive" but catastrophic mistake res.write data: ${JSON.stringify { step, text: log } }\n\n ; res.flush ; // Force-flushed chunks into arbitrary TCP packet boundaries It made things infinitely worse. Instead of hanging for 30 seconds and then failing, the client now crashed immediately on almost every single attempt. By aggressively forcing flushes on every minor string, we triggered something far uglier: violent TCP packet fragmentation. The large final JSON payload was being split mid-byte across arbitrary network packet boundaries. Our client-side reader, which naively assumed that each incoming SSE chunk was a complete, fully-formed JSON object, was receiving this: // Chunk 1 — Mutilated {"finalResult":{"skills": {"name":"Python","category":"foun // Chunk 2 — The rest, arriving milliseconds later dational","matched":true} ,"weeklyPlan": }} JSON.parse threw a SyntaxError: Unexpected end of JSON input on Chunk 1. The entire React UI state machine shattered. The app froze. The user saw a broken loading screen. We had fixed a timeout by turning it into an instant crash. Brilliant. That's when it clicked — we weren't fighting one bug. We were fighting two completely separate infrastructure forces compounding each other in production: 1. Proxy Buffering Google Cloud Run's GFE proxy holds small SSE packets in a buffer waiting for more data before flushing downstream. Unless explicitly instructed otherwise, it optimizes throughput by batching chunks — not passing them through immediately. This explained the initial 30-second hang: the proxy was collecting all six of our log lines and sitting on them before sending anything to the client. 2. TCP Stream Fragmentation An SSE stream is not a guaranteed series of atomic messages — it is a continuous raw byte stream. When sending a 15KB payload, the network transport layer is fully entitled — and will — split that across multiple TCP packets. The frontend must treat incoming data as a byte accumulator, not a ready-to-parse event queue. We had fixed neither. Our flush attempt had only worsened the second while accidentally bypassing the first. The solution required changes on both sides of the wire simultaneously. We added the HTTP header that explicitly instructs modern reverse proxies — including Nginx and GFE — to pass bytes through immediately without buffering: // server.ts — Patched SSE initialization res.writeHead 200, { "Content-Type": "text/event-stream", "Cache-Control": "no-cache, no-transform", // 'no-transform' blocks compression proxies from buffering "Connection": "keep-alive", "X-Accel-Buffering": "no" // Explicit bypass for Nginx/GFE } ; We also disabled global Gzip compression specifically for the /api/analyze-career route, ensuring the compression layer wasn't holding bytes in its own internal buffer before the proxy even saw them. onmessage , Build a Byte Accumulator We abandoned the naive EventSource.onmessage listener entirely. Instead, we wrote a low-level stream reader that accumulates incoming bytes in a staging buffer and only processes complete SSE frames — identified by double-newline \n\n boundaries — never attempting to parse a fragment: js // src/App.tsx — Boundary-safe chunk reassembly const reader = response.body.getReader ; const decoder = new TextDecoder ; let buffer = ""; while true { const { value, done } = await reader.read ; if done break; buffer += decoder.decode value, { stream: true } ; const lines = buffer.split "\n\n" ; // Keep the last, potentially incomplete fragment staged in the buffer buffer = lines.pop || ""; for const line of lines { if line.startsWith "data: " continue; const jsonStr = line.replace "data: ", "" .trim ; try { const parsed = JSON.parse jsonStr ; // Safe: only complete, boundary-verified JSON ever reaches state } catch e { // Fragment guard: push back and wait for remaining bytes console.warn "Partial chunk detected — staging for next read cycle." ; } } } The key insight: lines.pop pulls the trailing incomplete fragment out of the processing loop and holds it in buffer until the next network read brings the rest of the bytes. No more mid-payload parsing. No more crashes. After pushing both fixes to production, we ran a load test simulating 100 concurrent users hitting the full six-stage agentic pipeline simultaneously. The numbers were stark: | Metric | Before | After | |---|---|---| | Pipeline Completion Rate | 67% | 99.2% | | Time-to-First Log FCP | 28 seconds | 1.1 seconds | | JSON Syntax Crashes | Constant | Zero | I DM'd the user back at 1:30 AM. They tested it again and replied: "okay yeah this is actually really cool now." The real lesson here wasn't about SSE or TCP fragmentation specifically. It was about the gap between local development and containerized production. Locally, you are your own proxy. There is no GFE, no Nginx, no buffering layer between your Node process and the browser. Every byte arrives instantly. Every test passes. Everything feels perfect. The moment you deploy behind a reverse proxy, your assumptions about when data arrives collapse. Streaming architectures that rely on chunk atomicity — SSE, WebSockets, chunked transfer encoding — need to be designed defensively from the first line of code, not patched reactively at 11 PM when a user DM slides in. Build your readers as byte accumulators. Treat every incoming chunk as potentially incomplete. Use X-Accel-Buffering: no from day one on any SSE endpoint that goes near a proxy. And test in an environment that actually mirrors production — because localhost will lie to you every time. CareerPilot AI is an open-source AI-powered career roadmap agent. You can explore the full codebase at github.com/aashitanegii/-CareerPilotAI and try the live app at careerpilot-ai-667889155113.asia-southeast1.run.app.