{"slug": "cancelled-is-a-terminal-state-not-an-exception", "title": "Cancelled Is a Terminal State, Not an Exception", "summary": "A developer argues that cancellation in full-stack AI generation features should be modeled as a first-class terminal state in a distributed transaction rather than an exception path, citing a support ticket where a user who closed a tab mid-summary was billed twice. The post outlines a state machine with a `cancelled` state and required ledger side effects, and provides a FastAPI/httpx skeleton that explicitly closes the upstream provider socket on client disconnect and records partial spend. It notes caveats including per-chunk disconnect polling cost and proxy buffering that can leave zombie streams alive.", "body_md": "A support ticket landed on a Tuesday: a user closed the tab forty seconds into a long summary, reopened the app, and found two generations billed against her workspace. Nothing in the logs was red. The job row said `running`, the ledger said `charged`, and the provider dashboard said the call had finished normally — because from the provider's point of view, nobody ever hung up.\n\nThat ticket is the whole article. We keep treating \"cancel\" as a piece of UI candy: a stop button that flips local state and lets the expensive part keep going. I think that is backwards. Cancellation is a distributed transaction, and it only counts when it reaches every party that can spend your money — the model provider, your persistence layer, and whoever reads the ledger afterward.\n\nTrace a single cancel through a normal full-stack AI feature and you will find the same seam. The browser aborts the `fetch`, your API sees the socket drop, and then your handler does the one thing that guarantees a leak: it raises, unwinds, and never writes a terminal row. The upstream `httpx` call keeps reading tokens into a buffer nobody will ever consume.\n\nWhy does it survive code review? Because it looks like an exception path, and exceptions feel handled. If your generation state machine has no `cancelled` state, the only honest thing your code can do with a cancel is throw — and throwing is exactly what skips your `finally` bookkeeping in the sloppy version everyone writes first.\n\nSo the fix is not defensive try/except everywhere. The fix is to declare the state machine, then make the code obey it.\n\n| From | Event | To | Required side effect | \n|---|---|---|---|\n| requested | first token | streaming | open the spend row | \n| streaming | cancel received | cancelled | close upstream socket, record partial spend | \n| streaming | model finishes | completed | record final spend | \n| streaming | upstream error | failed | record what was spent, keep the error code | \n\nWhen `cancelled` is a first-class state, the ledger question answers itself: you spent tokens up to the moment you hung up, and you say so.\n\nHere is the shape I ship, written against FastAPI and `httpx`. Treat it as a skeleton: the timeouts are placeholders you should measure, and the ledger functions are your own repository calls.\n\n``` python\n# app/api/generate.py - skeleton, not a drop-in file\nimport httpx\nfrom fastapi import APIRouter, Request\n\nrouter = APIRouter()\n\n@router.post(\"/generate/{gen_id}\")\nasync def generate(gen_id: str, payload: Prompt, request: Request):\n    await ledger.open(gen_id, prompt=payload.text)  # idempotent on gen_id\n    client = httpx.AsyncClient(timeout=httpx.Timeout(60.0, read=15.0))\n    final = \"failed\"\n    try:\n        async with client.stream(\"POST\", PROVIDER_URL, json=payload.body()) as upstream:\n            async for chunk in upstream.aiter_text():\n                if await request.is_disconnected():\n                    await client.aclose()  # hang up on the provider\n                    final = \"cancelled\"\n                    break\n                await sink.write(gen_id, chunk)\n            else:\n                final = \"completed\"\n    except httpx.ReadTimeout:\n        final = \"failed\"\n    finally:\n        await client.aclose()\n        await ledger.close(gen_id, state=final, spent=usage_for(gen_id))\n```\n\nThe one line people skip is `await client.aclose()` inside the loop. Breaking out of `aiter_text` closes the stream in some client versions and not others, and \"in some versions\" is how you get a monthly bill you cannot explain. Closing twice is harmless; not closing is not.\n\nTwo honest caveats on this snippet. Polling `request.is_disconnected()` per chunk is cheap but not free, so at high chunk rates you want an `asyncio.Event` set by the disconnect watcher and an `anyio` race instead. And if you sit behind a proxy that buffers responses, the disconnect may reach the proxy before it reaches you, which means your handler keeps a zombie stream alive. The mitigation is a hard per-generation deadline, not hope.\n\nAborting a `fetch` is easy. Making the abort arrive is the hard part, because a closing tab does not always flush a normal request.\n\n``` js\nconst ctrl = new AbortController();\nconst timer = setTimeout(() => ctrl.abort(), 45_000);\n\ntry {\n  const res = await fetch(`/generate/${genId}`, {\n    method: \"POST\",\n    body: JSON.stringify(payload),\n    signal: ctrl.signal,\n  });\n  await readStream(res.body);\n} catch (err) {\n  if (err.name === \"AbortError\") await reportCancel(genId, \"user_stopped\");\n  throw err;\n} finally {\n  clearTimeout(timer);\n}\n\nwindow.addEventListener(\"pagehide\", () => {\n  navigator.sendBeacon(`/generate/${genId}/cancel`, JSON.stringify({ reason: \"tab_closed\" }));\n});\n```\n\nThe explicit `cancel` endpoint matters more than the `AbortController`. A dropped socket is a hint; a POST to a cancel route is a fact you can act on idempotently, and `sendBeacon` fires on `pagehide` when a normal `fetch` would be cancelled by the browser itself. Your cancel route should be safe to call twice, five times, or after completion, and it should return `409` or a no-op `200` depending on your taste — but it must never create a second generation.\n\nHere is the part I care about most, because a cancel path you have not killed is fiction. Run this as a test, not as a manual click-through.\n\n```\n# tests/test_cancel_path.py - one failure mode, end to end\n@pytest.mark.asyncio\nasync def test_stream_killed_midflight_lands_as_cancelled(app, mock_provider, ledger):\n    mock_provider.emit_tokens(count=50, delay=0.05)  # ~2.5s of fake stream\n\n    async with httpx.AsyncClient(app=app, base_url=\"http://test\") as client:\n        task = asyncio.create_task(client.post(\"/generate/gen_1\", json={\"text\": \"hi\"}))\n        await asyncio.sleep(0.4)\n        task.cancel()  # the client walks away\n        with pytest.raises(asyncio.CancelledError):\n            await task\n\n    row = await wait_for_terminal(ledger, \"gen_1\", timeout=2.0)\n    assert row.state == \"cancelled\"\n    assert row.spent_tokens < 50\n    assert mock_provider.last_request_aborted is True\n    assert mock_provider.open_streams == 0\n```\n\nThat last assertion is the one that catches real bugs. `state == cancelled` proves your bookkeeping heard you; `open_streams == 0` proves you actually hung up. I have seen the first pass and the second fail, which is the worst kind of green build: clean audit trail, leaking socket.\n\nAdd a second test that replays the cancel endpoint three times after completion and asserts no duplicate ledger row. Then add a chaos case where the provider ignores your abort — a mock that keeps emitting after `aclose()` — and confirm your sink stops writing anyway, because you cannot control a vendor, only your own side.\n\nA drill like this needs traffic, and traffic is what free tiers are for. I run the kill-the-stream suite against MonkeyCode's free model access — the operator advertises a ten-million-token allowance — because burning a few hundred aborted completions on a paid key to prove a `finally` block is an absurd way to spend a Tuesday. The free server option matters for a subtler reason: running the same test against a deployed instance instead of `localhost` puts a real network hop between client and handler, and that hop is where disconnect detection gets honest about itself.\n\nDisclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nIf you want to try the drill, the open-source project is a reasonable place to start, and I would rather you bring the test than the tutorial. What I will not claim is that any free allowance makes poor cancellation hygiene acceptable — the allowance is finite and your users are not.\n\nNot every generation needs a cancel path, and pretending otherwise adds states you will never exercise. Batch jobs where nobody is watching, internal tools with three users, and anything wrapped in a queue worker that owns its own lifecycle can get away with a deadline and a dead-letter state instead of a live cancel protocol.\n\nYou should also skip the interactive cancel state machine if your provider offers no way to signal an abort and your client library cannot close the underlying socket — in that case your only real control is a server-side deadline, and you should say so in the UI rather than shipping a stop button that stops nothing.\n\nOn the other end, if you are shipping anything where a user can watch tokens arrive, the drill above is table stakes, and it is cheaper than the refund email.\n\nWhich handoff in your stack is least stable — the browser abort, the server's disconnect detection, or the provider's stream close? Give me the concrete failure state or response code you see, and I will tell you which assertion to add first.\n\n*If this was useful, the one thing I would ask: run the kill-the-stream test against your own endpoint before you deploy the next change to it, and paste the response body you get back when a stream dies mid-flight.*", "url": "https://wpnews.pro/news/cancelled-is-a-terminal-state-not-an-exception", "canonical_source": "https://dev.to/kongkong1/cancelled-is-a-terminal-state-not-an-exception-4ifo", "published_at": "2026-10-26 11:09:13+00:00", "updated_at": "2026-10-26 11:12:58.261529+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-agents", "developer-tools", "mlops"], "entities": ["FastAPI", "httpx", "anyio"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cancelled-is-a-terminal-state-not-an-exception", "markdown": "https://wpnews.pro/news/cancelled-is-a-terminal-state-not-an-exception.md", "text": "https://wpnews.pro/news/cancelled-is-a-terminal-state-not-an-exception.txt", "jsonld": "https://wpnews.pro/news/cancelled-is-a-terminal-state-not-an-exception.jsonld"}}