Cancelled Is a Terminal State, Not an Exception

A developer argues that cancellation in full-stack AI generation features should be modeled as a first-class terminal state in a distributed transaction rather than an exception path, citing a support ticket where a user who closed a tab mid-summary was billed twice. The post outlines a state machine with a `cancelled` state and required ledger side effects, and provides a FastAPI/httpx skeleton that explicitly closes the upstream provider socket on client disconnect and records partial spend. It notes caveats including per-chunk disconnect polling cost and proxy buffering that can leave zombie streams alive.

A support ticket landed on a Tuesday: a user closed the tab forty seconds into a long summary, reopened the app, and found two generations billed against her workspace. Nothing in the logs was red. The job row said running , the ledger said charged , and the provider dashboard said the call had finished normally — because from the provider's point of view, nobody ever hung up. That ticket is the whole article. We keep treating "cancel" as a piece of UI candy: a stop button that flips local state and lets the expensive part keep going. I think that is backwards. Cancellation is a distributed transaction, and it only counts when it reaches every party that can spend your money — the model provider, your persistence layer, and whoever reads the ledger afterward. Trace a single cancel through a normal full-stack AI feature and you will find the same seam. The browser aborts the fetch , your API sees the socket drop, and then your handler does the one thing that guarantees a leak: it raises, unwinds, and never writes a terminal row. The upstream httpx call keeps reading tokens into a buffer nobody will ever consume. Why does it survive code review? Because it looks like an exception path, and exceptions feel handled. If your generation state machine has no cancelled state, the only honest thing your code can do with a cancel is throw — and throwing is exactly what skips your finally bookkeeping in the sloppy version everyone writes first. So the fix is not defensive try/except everywhere. The fix is to declare the state machine, then make the code obey it. | From | Event | To | Required side effect | |---|---|---|---| | requested | first token | streaming | open the spend row | | streaming | cancel received | cancelled | close upstream socket, record partial spend | | streaming | model finishes | completed | record final spend | | streaming | upstream error | failed | record what was spent, keep the error code | When cancelled is a first-class state, the ledger question answers itself: you spent tokens up to the moment you hung up, and you say so. Here is the shape I ship, written against FastAPI and httpx . Treat it as a skeleton: the timeouts are placeholders you should measure, and the ledger functions are your own repository calls. python app/api/generate.py - skeleton, not a drop-in file import httpx from fastapi import APIRouter, Request router = APIRouter @router.post "/generate/{gen id}" async def generate gen id: str, payload: Prompt, request: Request : await ledger.open gen id, prompt=payload.text idempotent on gen id client = httpx.AsyncClient timeout=httpx.Timeout 60.0, read=15.0 final = "failed" try: async with client.stream "POST", PROVIDER URL, json=payload.body as upstream: async for chunk in upstream.aiter text : if await request.is disconnected : await client.aclose hang up on the provider final = "cancelled" break await sink.write gen id, chunk else: final = "completed" except httpx.ReadTimeout: final = "failed" finally: await client.aclose await ledger.close gen id, state=final, spent=usage for gen id The one line people skip is await client.aclose inside the loop. Breaking out of aiter text closes the stream in some client versions and not others, and "in some versions" is how you get a monthly bill you cannot explain. Closing twice is harmless; not closing is not. Two honest caveats on this snippet. Polling request.is disconnected per chunk is cheap but not free, so at high chunk rates you want an asyncio.Event set by the disconnect watcher and an anyio race instead. And if you sit behind a proxy that buffers responses, the disconnect may reach the proxy before it reaches you, which means your handler keeps a zombie stream alive. The mitigation is a hard per-generation deadline, not hope. Aborting a fetch is easy. Making the abort arrive is the hard part, because a closing tab does not always flush a normal request. js const ctrl = new AbortController ; const timer = setTimeout = ctrl.abort , 45 000 ; try { const res = await fetch /generate/${genId} , { method: "POST", body: JSON.stringify payload , signal: ctrl.signal, } ; await readStream res.body ; } catch err { if err.name === "AbortError" await reportCancel genId, "user stopped" ; throw err; } finally { clearTimeout timer ; } window.addEventListener "pagehide", = { navigator.sendBeacon /generate/${genId}/cancel , JSON.stringify { reason: "tab closed" } ; } ; The explicit cancel endpoint matters more than the AbortController . A dropped socket is a hint; a POST to a cancel route is a fact you can act on idempotently, and sendBeacon fires on pagehide when a normal fetch would be cancelled by the browser itself. Your cancel route should be safe to call twice, five times, or after completion, and it should return 409 or a no-op 200 depending on your taste — but it must never create a second generation. Here is the part I care about most, because a cancel path you have not killed is fiction. Run this as a test, not as a manual click-through. tests/test cancel path.py - one failure mode, end to end @pytest.mark.asyncio async def test stream killed midflight lands as cancelled app, mock provider, ledger : mock provider.emit tokens count=50, delay=0.05 ~2.5s of fake stream async with httpx.AsyncClient app=app, base url="http://test" as client: task = asyncio.create task client.post "/generate/gen 1", json={"text": "hi"} await asyncio.sleep 0.4 task.cancel the client walks away with pytest.raises asyncio.CancelledError : await task row = await wait for terminal ledger, "gen 1", timeout=2.0 assert row.state == "cancelled" assert row.spent tokens < 50 assert mock provider.last request aborted is True assert mock provider.open streams == 0 That last assertion is the one that catches real bugs. state == cancelled proves your bookkeeping heard you; open streams == 0 proves you actually hung up. I have seen the first pass and the second fail, which is the worst kind of green build: clean audit trail, leaking socket. Add a second test that replays the cancel endpoint three times after completion and asserts no duplicate ledger row. Then add a chaos case where the provider ignores your abort — a mock that keeps emitting after aclose — and confirm your sink stops writing anyway, because you cannot control a vendor, only your own side. A drill like this needs traffic, and traffic is what free tiers are for. I run the kill-the-stream suite against MonkeyCode's free model access — the operator advertises a ten-million-token allowance — because burning a few hundred aborted completions on a paid key to prove a finally block is an absurd way to spend a Tuesday. The free server option matters for a subtler reason: running the same test against a deployed instance instead of localhost puts a real network hop between client and handler, and that hop is where disconnect detection gets honest about itself. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you want to try the drill, the open-source project is a reasonable place to start, and I would rather you bring the test than the tutorial. What I will not claim is that any free allowance makes poor cancellation hygiene acceptable — the allowance is finite and your users are not. Not every generation needs a cancel path, and pretending otherwise adds states you will never exercise. Batch jobs where nobody is watching, internal tools with three users, and anything wrapped in a queue worker that owns its own lifecycle can get away with a deadline and a dead-letter state instead of a live cancel protocol. You should also skip the interactive cancel state machine if your provider offers no way to signal an abort and your client library cannot close the underlying socket — in that case your only real control is a server-side deadline, and you should say so in the UI rather than shipping a stop button that stops nothing. On the other end, if you are shipping anything where a user can watch tokens arrive, the drill above is table stakes, and it is cheaper than the refund email. Which handoff in your stack is least stable — the browser abort, the server's disconnect detection, or the provider's stream close? Give me the concrete failure state or response code you see, and I will tell you which assertion to add first. If this was useful, the one thing I would ask: run the kill-the-stream test against your own endpoint before you deploy the next change to it, and paste the response body you get back when a stream dies mid-flight.