{"slug": "the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real", "title": "The Disconnect Test: A Take-Home Where Cancelling Upstream Is the Real Requirement", "summary": "A developer published a take-home interview exercise that tests whether candidates building a streaming proxy in front of an OpenAI-compatible /v1/chat/completions endpoint correctly abort the upstream request when the client disconnects. The reference solution uses Node's built-in fetch with one AbortController per upstream attempt and a deterministic fake upstream exposing started/completed/aborted counters, so graders can verify cancellation at the process boundary without network access. The writeup argues that leaked upstream calls keep sockets open, hold shared endpoint capacity, and generate tokens nobody reads, making cancellation the difference between a proxy that scales and one that accumulates debt.", "body_md": "A streaming proxy take-home rarely fails on the happy path, because almost every candidate can pipe bytes from an upstream endpoint into an HTTP response. The graded behaviour appears when the client disappears halfway through the stream and the upstream request has to die with it. Most submissions leak that upstream call, keep generating tokens that nobody will read, and never notice, because their own test suite exercises only the successful case. This article gives the prompt, a reference solution, a weighted rubric, and the failure modes worth grading.\n\nThe exercise stays intentionally small: one route, one upstream dependency, one process. Candidates receive the following brief and roughly two hours to return code plus a short written note.\n\nImplement `POST /chat` as a streaming proxy in front of an OpenAI-compatible `/v1/chat/completions` endpoint.\n\n- Stream the upstream body to the client without buffering the whole response.\n- Abort the upstream request when the client disconnects.\n- Retry only before the first byte reaches the client, never afterwards.\n- Use an idle timeout rather than a total-duration timeout.\n- Emit one structured log line per request with status, bytes, and outcome.\n- Ship a test that proves the cancellation path without network access.\n\nThe third and fourth bullets carry most of the signal, because they encode a specific mental model of stream lifetime. A candidate who treats the upstream call as a fire-and-forget side effect will pass everything else and fail those two. The deliverable is deliberately narrow so that graders compare designs rather than feature counts.\n\nEvery leaked upstream request keeps a socket open, holds a slot on a shared endpoint, and continues generating tokens that no user will read. On a metered endpoint that leak shows up directly on the invoice, and on a shared free endpoint it degrades the experience for everyone else using the same capacity. Cancellation is therefore not a nicety but the difference between a proxy that scales and one that quietly accumulates debt.\n\nThe same reasoning explains the no-retry-after-first-byte rule. Once a token has reached the client, a retry produces a second stream spliced into the first, which surfaces as duplicated or interleaved text. Graders look for that duplication because it is one of the few failures a reviewer can spot from outside the process.\n\nTwo counters on a fake upstream settle the question faster than any code review. Start a stream, abort the client after a couple hundred milliseconds, then read the counters back.\n\n```\ncurl -s localhost:8787/__stats   # {\"started\":2,\"completed\":1,\"aborted\":1}\n```\n\nAn `aborted` count that matches the number of killed clients means the design is sound at the process boundary. A zero there means the upstream call outlived its consumer, and the rest of the submission is mostly commentary.\n\nThe reference uses the built-in `fetch`, one `AbortController` per upstream attempt, and a deterministic fake upstream so the tests never touch a real endpoint. It is a spec-complete sketch rather than a polished service, so run it locally before you grade anyone with it.\n\n``` python\n// fake-upstream.mjs\nimport http from \"node:http\";\n\nconst state = { started: 0, completed: 0, aborted: 0 };\n\nhttp.createServer((req, res) => {\n  if (req.url === \"/__stats\") {\n    res.writeHead(200, { \"content-type\": \"application/json\" });\n    res.end(JSON.stringify(state));\n    return;\n  }\n  if (req.url !== \"/v1/chat/completions\") {\n    res.writeHead(404).end();\n    return;\n  }\n\n  state.started++;\n  let finished = false;\n  res.writeHead(200, { \"content-type\": \"text/event-stream\", \"cache-control\": \"no-cache\" });\n\n  let i = 0;\n  const timer = setInterval(() => {\n    if (i === 40) {                       // 40 chunks every 25 ms => ~1 s stream\n      clearInterval(timer);\n      finished = true;\n      state.completed++;\n      res.write(\"data: [DONE]\\n\\n\");\n      res.end();\n      return;\n    }\n    res.write(`data: {\"choices\":[{\"delta\":{\"content\":\"tok${i} \"}}]}\\n\\n`);\n    i++;\n  }, 25);\n\n  res.on(\"close\", () => {                 // fires for a clean end and for a client abort\n    if (finished) return;\n    clearInterval(timer);\n    state.aborted++;\n  });\n}).listen(8787, () => console.log(\"fake upstream on :8787\"));\npython\n// proxy.mjs\nimport http from \"node:http\";\n\nconst UPSTREAM = process.env.UPSTREAM ?? \"http://127.0.0.1:8787\";\nconst IDLE_TIMEOUT_MS = 2000;\nconst MAX_RETRIES = 2;\n\nconst readBody = (req) => new Promise((resolve, reject) => {\n  const chunks = [];\n  req.on(\"data\", (c) => chunks.push(c));\n  req.on(\"end\", () => resolve(Buffer.concat(chunks)));\n  req.on(\"error\", reject);\n});\nconst once = (emitter, event) => new Promise((r) => emitter.once(event, r));\nconst backoff = (n) => new Promise((r) => setTimeout(r, 50 * 2 ** n));\n\nhttp.createServer(async (req, res) => {\n  if (req.method !== \"POST\" || req.url !== \"/chat\") return void res.writeHead(404).end();\n\n  const body = await readBody(req);\n  const clientGone = new AbortController();\n  // Bind the upstream lifetime to the response socket, not to the request body.\n  res.on(\"close\", () => { if (!res.writableEnded) clientGone.abort(new Error(\"client_gone\")); });\n\n  for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {\n    const upstream = new AbortController();\n    const onClientGone = () => upstream.abort(new Error(\"client_gone\"));\n    clientGone.signal.addEventListener(\"abort\", onClientGone, { once: true });\n\n    const idle = setTimeout(() => upstream.abort(new Error(\"idle_timeout\")), IDLE_TIMEOUT_MS);\n    let firstByteSent = false;\n\n    try {\n      const up = await fetch(`${UPSTREAM}/v1/chat/completions`, {\n        method: \"POST\",\n        headers: { \"content-type\": \"application/json\", accept: \"text/event-stream\" },\n        body,\n        signal: upstream.signal,\n      });\n\n      if (up.status >= 500 && attempt < MAX_RETRIES) {\n        clearTimeout(idle);\n        await backoff(attempt);\n        continue;\n      }\n\n      res.writeHead(up.status, {\n        \"content-type\": up.headers.get(\"content-type\") ?? \"text/event-stream\",\n        \"cache-control\": \"no-cache\",\n      });\n\n      for await (const chunk of up.body) {\n        idle.refresh();                                   // idle, not wall-clock\n        firstByteSent = true;\n        if (!res.write(chunk)) await once(res, \"drain\");   // respect backpressure\n      }\n      clearTimeout(idle);\n      res.end();\n      return;\n    } catch (err) {\n      clearTimeout(idle);\n      if (firstByteSent || clientGone.signal.aborted) return void res.end();\n      if (attempt === MAX_RETRIES) return void res.writeHead(502).end(\"upstream_unavailable\");\n      await backoff(attempt);\n    } finally {\n      clientGone.signal.removeEventListener(\"abort\", onClientGone);\n    }\n  }\n}).listen(3000, () => console.log(\"proxy on :3000\"));\n```\n\nTwo details in that file deserve explicit attention. The cancellation listener attaches to `res`, because `req` emits `close` when the request body finishes uploading and would abort every healthy stream. The retry guard checks `firstByteSent`, because a retry after the first chunk is a correctness bug rather than a resilience feature.\n\n```\nnode fake-upstream.mjs & UP=$!\nnode proxy.mjs &        PX=$!\nsleep 0.3\n\n# Happy path: the stream must reach [DONE] and count as completed.\ncurl -sN -X POST localhost:3000/chat -H 'content-type: application/json' \\\n  -d '{\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}' -o /dev/null\nsleep 0.15\n\n# Disconnect test: abort 200 ms into a roughly one-second stream.\ncurl -sN -X POST localhost:3000/chat -H 'content-type: application/json' \\\n  -d '{\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}' -o /dev/null --max-time 0.2 || true\nsleep 0.15\n\ncurl -s localhost:8787/__stats\n# expected: {\"started\":2,\"completed\":1,\"aborted\":1}\nkill $UP $PX\n```\n\n| # | Criterion | Weight | Pass condition | \n|---|---|---|---|\n| 1 | Upstream cancelled on client disconnect | 25 | `aborted` increments within a few hundred milliseconds of the client abort | \n| 2 | No retry after the first byte | 20 | no duplicated `tok` fragments appear in the client output | \n| 3 | Backpressure respected | 15 | the proxy awaits `drain` instead of buffering the whole stream | \n| 4 | Idle timeout, not a total timeout | 10 | a slow but live stream survives longer than the timeout window | \n| 5 | Bounded retries before the first byte | 10 | repeated 5xx leads to at most two retries, then a 502 | \n| 6 | Deterministic test included | 10 | the suite passes with the fake upstream and no network access | \n| 7 | Written trade-off note | 10 | the candidate names one real limitation of their own design | \n\nCriterion one carries the largest weight by design, and criterion seven breaks the tie between two submissions that both pass the harness. Neither is a style preference, since both map onto production incidents that a streaming service will eventually experience.\n\n`req.on(\"close\")` fires when the request body has been fully read, so every healthy stream looks abandoned and real users get dropped mid-answer.`fetch` promise keeps its connection open until the upstream finishes on its own schedule.`res.write()`.` drain`, a slow consumer causes unbounded buffering inside the process, which looks like a memory leak rather than a backpressure bug.\nThe harness above is deliberately offline, but reviewers want a URL they can open and candidates need an endpoint that costs nothing to experiment against. The MonkeyCode project, which the operator of this account describes as open source, is presented as offering free model access and a free server option for exactly this kind of small deployment. The operator states an allowance on the order of ten million tokens together with a free server, so pointing `UPSTREAM` at that endpoint and running `proxy.mjs` on the free instance makes the demo reachable without a card on file.\n\nDisclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nTwo honest caveats belong next to that recommendation. Free tiers change over time, so confirm the current allowance and terms on the project's own page instead of trusting a figure quoted in any blog post, including this one. A free allowance is also sized for harnesses and demos rather than load testing, so keep the fake upstream as the assertion target and treat the live endpoint as a smoke test.\n\n| Situation | Fake upstream | Free endpoint plus free server | \n|---|---|---|\n| Contract tests for cancellation and backpressure | Required | Optional | \n| A demo URL a reviewer can open | Not possible | Useful | \n| Comparing model output quality across weeks | Not applicable | Unsuitable, outputs drift | \n| Sustained production traffic | Not applicable | Unsuitable | \n| Measuring your own container's cold-start latency | Not applicable | Workable | \n| An interview take-home that must cost nothing to grade | Required | Useful for the demo step | \n\nIf you want a live endpoint behind the proxy during review, the free access described above covers a harness of this size, while the fake upstream remains the source of truth for pass and fail.", "url": "https://wpnews.pro/news/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real", "canonical_source": "https://dev.to/appjs_3979/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real-requirement-3knh", "published_at": "2026-10-16 05:29:50+00:00", "updated_at": "2026-10-16 05:35:02.021779+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-agents", "developer-tools"], "entities": ["OpenAI", "Node.js", "AbortController"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real", "markdown": "https://wpnews.pro/news/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real.md", "text": "https://wpnews.pro/news/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real.txt", "jsonld": "https://wpnews.pro/news/the-disconnect-test-a-take-home-where-cancelling-upstream-is-the-real.jsonld"}}