The Disconnect Test: A Take-Home Where Cancelling Upstream Is the Real Requirement

A streaming proxy take-home rarely fails on the happy path, because almost every candidate can pipe bytes from an upstream endpoint into an HTTP response. The graded behaviour appears when the client disappears halfway through the stream and the upstream request has to die with it. Most submissions leak that upstream call, keep generating tokens that nobody will read, and never notice, because their own test suite exercises only the successful case. This article gives the prompt, a reference solution, a weighted rubric, and the failure modes worth grading.

The exercise stays intentionally small: one route, one upstream dependency, one process. Candidates receive the following brief and roughly two hours to return code plus a short written note.

Implement POST /chat as a streaming proxy in front of an OpenAI-compatible /v1/chat/completions endpoint.

• Stream the upstream body to the client without buffering the whole response.
• Abort the upstream request when the client disconnects.
• Retry only before the first byte reaches the client, never afterwards.
• Use an idle timeout rather than a total-duration timeout.
• Emit one structured log line per request with status, bytes, and outcome.
• Ship a test that proves the cancellation path without network access.

The third and fourth bullets carry most of the signal, because they encode a specific mental model of stream lifetime. A candidate who treats the upstream call as a fire-and-forget side effect will pass everything else and fail those two. The deliverable is deliberately narrow so that graders compare designs rather than feature counts.

Every leaked upstream request keeps a socket open, holds a slot on a shared endpoint, and continues generating tokens that no user will read. On a metered endpoint that leak shows up directly on the invoice, and on a shared free endpoint it degrades the experience for everyone else using the same capacity. Cancellation is therefore not a nicety but the difference between a proxy that scales and one that quietly accumulates debt.

The same reasoning explains the no-retry-after-first-byte rule. Once a token has reached the client, a retry produces a second stream spliced into the first, which surfaces as duplicated or interleaved text. Graders look for that duplication because it is one of the few failures a reviewer can spot from outside the process.

Two counters on a fake upstream settle the question faster than any code review. Start a stream, abort the client after a couple hundred milliseconds, then read the counters back.

curl -s localhost:8787/__stats   # {"started":2,"completed":1,"aborted":1}

An aborted count that matches the number of killed clients means the design is sound at the process boundary. A zero there means the upstream call outlived its consumer, and the rest of the submission is mostly commentary.

The reference uses the built-in fetch, one AbortController per upstream attempt, and a deterministic fake upstream so the tests never touch a real endpoint. It is a spec-complete sketch rather than a polished service, so run it locally before you grade anyone with it.

// fake-upstream.mjs
import http from "node:http";

const state = { started: 0, completed: 0, aborted: 0 };

http.createServer((req, res) => {
  if (req.url === "/__stats") {
    res.writeHead(200, { "content-type": "application/json" });
    res.end(JSON.stringify(state));
    return;
  }
  if (req.url !== "/v1/chat/completions") {
    res.writeHead(404).end();
    return;
  }

  state.started++;
  let finished = false;
  res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });

  let i = 0;
  const timer = setInterval(() => {
    if (i === 40) {                       // 40 chunks every 25 ms => ~1 s stream
      clearInterval(timer);
      finished = true;
      state.completed++;
      res.write("data: [DONE]\n\n");
      res.end();
      return;
    }
    res.write(`data: {"choices":[{"delta":{"content":"tok${i} "}}]}\n\n`);
    i++;
  }, 25);

  res.on("close", () => {                 // fires for a clean end and for a client abort
    if (finished) return;
    clearInterval(timer);
    state.aborted++;
  });
}).listen(8787, () => console.log("fake upstream on :8787"));
python
// proxy.mjs
import http from "node:http";

const UPSTREAM = process.env.UPSTREAM ?? "http://127.0.0.1:8787";
const IDLE_TIMEOUT_MS = 2000;
const MAX_RETRIES = 2;

const readBody = (req) => new Promise((resolve, reject) => {
  const chunks = [];
  req.on("data", (c) => chunks.push(c));
  req.on("end", () => resolve(Buffer.concat(chunks)));
  req.on("error", reject);
});
const once = (emitter, event) => new Promise((r) => emitter.once(event, r));
const backoff = (n) => new Promise((r) => setTimeout(r, 50 * 2 ** n));

http.createServer(async (req, res) => {
  if (req.method !== "POST" || req.url !== "/chat") return void res.writeHead(404).end();

  const body = await readBody(req);
  const clientGone = new AbortController();
  // Bind the upstream lifetime to the response socket, not to the request body.
  res.on("close", () => { if (!res.writableEnded) clientGone.abort(new Error("client_gone")); });

  for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
    const upstream = new AbortController();
    const onClientGone = () => upstream.abort(new Error("client_gone"));
    clientGone.signal.addEventListener("abort", onClientGone, { once: true });

    const idle = setTimeout(() => upstream.abort(new Error("idle_timeout")), IDLE_TIMEOUT_MS);
    let firstByteSent = false;

    try {
      const up = await fetch(`${UPSTREAM}/v1/chat/completions`, {
        method: "POST",
        headers: { "content-type": "application/json", accept: "text/event-stream" },
        body,
        signal: upstream.signal,
      });

      if (up.status >= 500 && attempt < MAX_RETRIES) {
        clearTimeout(idle);
        await backoff(attempt);
        continue;
      }

      res.writeHead(up.status, {
        "content-type": up.headers.get("content-type") ?? "text/event-stream",
        "cache-control": "no-cache",
      });

      for await (const chunk of up.body) {
        idle.refresh();                                   // idle, not wall-clock
        firstByteSent = true;
        if (!res.write(chunk)) await once(res, "drain");   // respect backpressure
      }
      clearTimeout(idle);
      res.end();
      return;
    } catch (err) {
      clearTimeout(idle);
      if (firstByteSent || clientGone.signal.aborted) return void res.end();
      if (attempt === MAX_RETRIES) return void res.writeHead(502).end("upstream_unavailable");
      await backoff(attempt);
    } finally {
      clientGone.signal.removeEventListener("abort", onClientGone);
    }
  }
}).listen(3000, () => console.log("proxy on :3000"));

Two details in that file deserve explicit attention. The cancellation listener attaches to res, because req emits close when the request body finishes up and would abort every healthy stream. The retry guard checks firstByteSent, because a retry after the first chunk is a correctness bug rather than a resilience feature.

node fake-upstream.mjs & UP=$!
node proxy.mjs &        PX=$!
sleep 0.3

curl -sN -X POST localhost:3000/chat -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}]}' -o /dev/null
sleep 0.15

curl -sN -X POST localhost:3000/chat -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}]}' -o /dev/null --max-time 0.2 || true
sleep 0.15

curl -s localhost:8787/__stats
kill $UP $PX

Criterion one carries the largest weight by design, and criterion seven breaks the tie between two submissions that both pass the harness. Neither is a style preference, since both map onto production incidents that a streaming service will eventually experience.

req.on("close") fires when the request body has been fully read, so every healthy stream looks abandoned and real users get dropped mid-answer.fetch promise keeps its connection open until the upstream finishes on its own schedule.res.write(). drain, a slow consumer causes unbounded buffering inside the process, which looks like a memory leak rather than a backpressure bug. The harness above is deliberately offline, but reviewers want a URL they can open and candidates need an endpoint that costs nothing to experiment against. The MonkeyCode project, which the operator of this account describes as open source, is presented as offering free model access and a free server option for exactly this kind of small deployment. The operator states an allowance on the order of ten million tokens together with a free server, so pointing UPSTREAM at that endpoint and running proxy.mjs on the free instance makes the demo reachable without a card on file.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two honest caveats belong next to that recommendation. Free tiers change over time, so confirm the current allowance and terms on the project's own page instead of trusting a figure quoted in any blog post, including this one. A free allowance is also sized for harnesses and demos rather than load testing, so keep the fake upstream as the assertion target and treat the live endpoint as a smoke test.

If you want a live endpoint behind the proxy during review, the free access described above covers a harness of this size, while the fake upstream remains the source of truth for pass and fail.