Freeze the Error Contract Before One except Change

A developer argues that mixed None-and-raise APIs are the real product surface and should be pinned with characterization tests before any refactor, since AI coding tools tend to unify error handlers and drop edge cases like a send TypeError. The post presents a worked example of a dispatch_event function with three distinct outcomes — raising ValueError or RuntimeError, returning None, or returning a dict with a status code — and a fixture table specifying escapes, return shape, status, and warning counts for each case.

Mixed None-and-raise APIs fail under model rewrites. The error taxonomy is the real product surface. Pin that taxonomy with characterization tests first, always. Change one except clause only after the pin stays green. Happy-path unit tests will not save this refactor. Callers already branch on None , dict keys, and types. Collapse any one of those shapes and production breaks. The defect is an unrecorded taxonomy, not missing types. AI coding threads keep offering full-file cleanups. Those cleanups prefer one error type. They also prefer raising over returning None . That preference is style, not evidence. A messy dispatcher often has three outcomes. Some inputs raise ValueError for bad payloads. Some inputs return None after a send failure. Some inputs return {"ok": False, "status": 429} . Downstream code already checks all three shapes. Loop-style agent edits make the same cut. They unify handlers because duplication looks sloppy. Duplication here is the published contract. Record these four fields for every fixture. Skip prose messages on the first pass. Message strings drift across harmless edits. None , mapping keys, or other. Types and keys usually stay stable. Status integers also stay stable. Log counts need a named logger, not the root. The table below is the spec. It is a labeled example. It is not a production trace. | Fixture | Escapes | Return | Status | WARN+ | |---|---|---|---|---| | empty body | RuntimeError | n/a | n/a | 0 | | invalid JSON | ValueError | n/a | n/a | 0 | | JSON list | ValueError | n/a | n/a | 0 | | missing id | none | None | n/a | 1 | | send TypeError | none | None | n/a | 1 | | send TimeoutError | none | None | n/a | 1 | | downstream 429 | none | dict | 429 | 1 | | downstream 200 | none | dict | 200 | 0 | Keep the send TypeError row. That row is the trap. A cleaner except often drops it. The code is a worked example. Run it locally before any extract. Do not treat it as measured field data. python events.py — labeled example, not a live service from future import annotations import json import logging from typing import Any, Callable log = logging.getLogger "events" SendFn = Callable dict , tuple int, Any def dispatch event raw: str | None, send: SendFn : """Messy contract: None, dict, or raise.""" if raw is None or raw == "": raise RuntimeError "empty body" try: payload = json.loads raw except Exception: raise ValueError "bad json" if not isinstance payload, dict : raise ValueError "bad json" if "id" not in payload: log.warning "missing id" return None try: status, body = send payload except Exception: log.warning "send failed" return None if status = 400: log.warning "downstream %s", status return {"ok": False, "status": status, "body": body} return {"ok": True, "status": status, "body": body} python test events contract.py — worked example import json import logging import pytest from events import dispatch event def records caplog : return r for r in caplog.records if r.name == "events" and r.levelno = logging.WARNING def run raw, send, caplog : caplog.set level logging.WARNING, logger="events" try: value = dispatch event raw, send return None, value, records caplog except Exception as exc: return type exc , None, records caplog def test empty body raises runtime error caplog : exc, value, recs = run "", lambda p: 200, "ok" , caplog assert exc is RuntimeError assert value is None assert len recs == 0 @pytest.mark.parametrize "raw", "{", " ", "null" def test bad json raises value error raw, caplog : exc, value, recs = run raw, lambda p: 200, "ok" , caplog assert exc is ValueError assert value is None assert len recs == 0 def test missing id returns none caplog : exc, value, recs = run '{"name": "x"}', lambda p: 200, "ok" , caplog assert exc is None assert value is None assert len recs == 1 def test send type error returns none caplog : def send payload : raise TypeError "broken client" exc, value, recs = run '{"id": 1}', send, caplog assert exc is None assert value is None assert len recs == 1 def test send timeout returns none caplog : def send payload : raise TimeoutError "late" exc, value, recs = run '{"id": 1}', send, caplog assert exc is None assert value is None assert len recs == 1 def test downstream 429 is dict caplog : exc, value, recs = run '{"id": 1}', lambda p: 429, "slow" , caplog assert exc is None assert value == {"ok": False, "status": 429, "body": "slow"} assert len recs == 1 def test downstream 200 is dict caplog : exc, value, recs = run '{"id": 1}', lambda p: 200, {"n": 1} , caplog assert exc is None assert value == {"ok": True, "status": 200, "body": {"n": 1}} assert len recs == 0 Run the file before editing handlers. Use a short traceback during the first pin. python -m pytest test events contract.py -q --tb=short A green suite with no mutation check is weak. You must see a failed unified-error fork. Keep that fork out of the merge. python test events mutation.py — must fail against a unified raise rewrite import events def test unified raise would break send type error monkeypatch, caplog : def rewritten raw, send : payload = import "json" .loads raw status, body = send payload TypeError now escapes return {"ok": status < 400, "status": status, "body": body} monkeypatch.setattr events, "dispatch event", rewritten with pytest.raises TypeError : events.dispatch event '{"id": 1}', lambda p: for in .throw TypeError "x" Expect this mutation test to fail on the rewrite. Restore the messy handler after that check. The failure is the evidence, not a vibe. is None or catch types. except edit. Step two is not optional. Tests invented from the callee miss caller branches. Caller branches are the contract. rg -n "dispatch event\ " -g " .py" rg -n "is None|except ValueError|except RuntimeError" -g " .py" Record each hit as a fixture name. Missing hits become missing rows. Missing rows make unsafe extracts look safe. Do not narrow except Exception on send first. The table says TypeError becomes None . Narrowing that clause raises TypeError instead. That raise is a contract break. Extract the block without narrowing. Keep the same log line and None return. python def send or none payload, send : try: return send payload except Exception: log.warning "send failed" return None def dispatch event raw, send : if raw is None or raw == "": raise RuntimeError "empty body" payload = parse object raw if "id" not in payload: log.warning "missing id" return None result = send or none payload, send if result is None: return None status, body = result if status = 400: log.warning "downstream %s", status return {"ok": False, "status": status, "body": body} return {"ok": True, "status": status, "body": body} That extract is boring. Boring is the point. The taxonomy stays in the table. A later change can narrow exceptions. Do that only with a version note. Add a row that expects TypeError to escape. Tell callers before you merge. The JSON branch is different. json.loads should raise json.JSONDecodeError . Re-raising ValueError is the published type. Narrowing except Exception to except json.JSONDecodeError can be safe. Prove it with the invalid JSON row. Prove it with the JSON list row too. python def parse object raw : try: payload = json.loads raw except json.JSONDecodeError: raise ValueError "bad json" from None if not isinstance payload, dict : raise ValueError "bad json" return payload Both bad-JSON fixtures must still raise ValueError . A leaked JSONDecodeError is a taxonomy change. Do not ship that leak without a caller audit. from None also drops cause . Add a cause fixture if any caller reads it. Skip that fixture when no caller inspects causes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A model can draft the extract. It must not invent a new taxonomy. MonkeyCode provides free model access and a free server option. Use either only after the characterization file is green. Feed the model the table and the test file. Ask for one extract, not a rewrite. Reject a patch that removes a row. Reject a patch that changes None into a raise. This is not an agent loop. One prompt. One diff. Same tests. The harness does not prove semantic equality. It pins types, keys, status, and log counts. It misses timing, retry storms, and byte identity. Log counts break if a library logs extra warnings. Pin the logger name events . Cap propagation on that logger. Do not pin the root logger. Root pins go red on unrelated imports. That noise hides a real taxonomy drift. The table is only as good as the fixtures. One unlisted caller path is an untested shape. Untested shapes are how "safe" extracts land in incident channels. Do not use this flow for greenfield APIs. Design one error shape there. Do not preserve None plus raises on purpose. Do not use this flow for security boundaries. Characterization will pin insecure behavior. Pinning is not hardening. Do not use this flow without tests you can run offline. A model server cannot replace the table. If pytest cannot collect locally, stop. Teams with a published OpenAPI error schema may skip dict-key rows. Use the schema as the table instead. Still pin process-local exception escapes. HTTP schemas often omit those local raises. Omitting them is how ValueError turns into a 500. Keep the escape column anyway. If a model patch violates any line, drop the patch. Rewrite pressure is not evidence. The table is. If the characterization file is already green, one extract prompt on the free server is enough. Skip that prompt when the table still has empty cells.