Mixed None-and-raise APIs fail under model rewrites. The error taxonomy is the real product surface. Pin that taxonomy with characterization tests first, always.
Change one except clause only after the pin stays green. Happy-path unit tests will not save this refactor. Callers already branch on None, dict keys, and types.
Collapse any one of those shapes and production breaks. The defect is an unrecorded taxonomy, not missing types.
AI coding threads keep offering full-file cleanups. Those cleanups prefer one error type. They also prefer raising over returning None.
That preference is style, not evidence. A messy dispatcher often has three outcomes. Some inputs raise ValueError for bad payloads.
Some inputs return None after a send failure. Some inputs return {"ok": False, "status": 429}. Downstream code already checks all three shapes.
Loop-style agent edits make the same cut. They unify handlers because duplication looks sloppy. Duplication here is the published contract.
Record these four fields for every fixture. Skip prose messages on the first pass. Message strings drift across harmless edits.
None, mapping keys, or other.
Types and keys usually stay stable. Status integers also stay stable. Log counts need a named logger, not the root.
The table below is the spec. It is a labeled example. It is not a production trace.
| Fixture | Escapes | Return | Status | WARN+ |
|---|---|---|---|---|
| empty body | RuntimeError | n/a | n/a | 0 |
| invalid JSON | ValueError | n/a | n/a | 0 |
| JSON list | ValueError | n/a | n/a | 0 |
| missing id | none | None | n/a | 1 |
| send TypeError | none | None | n/a | 1 |
| send TimeoutError | none | None | n/a | 1 |
| downstream 429 | none | dict | 429 | 1 |
| downstream 200 | none | dict | 200 | 0 |
Keep the send TypeError row. That row is the trap. A cleaner except often drops it.
The code is a worked example. Run it locally before any extract. Do not treat it as measured field data.
from __future__ import annotations
import json
import logging
from typing import Any, Callable
log = logging.getLogger("events")
SendFn = Callable[[dict], tuple[int, Any]]
def dispatch_event(raw: str | None, send: SendFn):
"""Messy contract: None, dict, or raise."""
if raw is None or raw == "":
raise RuntimeError("empty body")
try:
payload = json.loads(raw)
except Exception:
raise ValueError("bad json")
if not isinstance(payload, dict):
raise ValueError("bad json")
if "id" not in payload:
log.warning("missing id")
return None
try:
status, body = send(payload)
except Exception:
log.warning("send failed")
return None
if status >= 400:
log.warning("downstream %s", status)
return {"ok": False, "status": status, "body": body}
return {"ok": True, "status": status, "body": body}
python
import json
import logging
import pytest
from events import dispatch_event
def _records(caplog):
return [r for r in caplog.records if r.name == "events" and r.levelno >= logging.WARNING]
def _run(raw, send, caplog):
caplog.set_level(logging.WARNING, logger="events")
try:
value = dispatch_event(raw, send)
return None, value, _records(caplog)
except Exception as exc:
return type(exc), None, _records(caplog)
def test_empty_body_raises_runtime_error(caplog):
exc, value, recs = _run("", lambda p: (200, "ok"), caplog)
assert exc is RuntimeError
assert value is None
assert len(recs) == 0
@pytest.mark.parametrize("raw", ["{", "[]", "null"])
def test_bad_json_raises_value_error(raw, caplog):
exc, value, recs = _run(raw, lambda p: (200, "ok"), caplog)
assert exc is ValueError
assert value is None
assert len(recs) == 0
def test_missing_id_returns_none(caplog):
exc, value, recs = _run('{"name": "x"}', lambda p: (200, "ok"), caplog)
assert exc is None
assert value is None
assert len(recs) == 1
def test_send_type_error_returns_none(caplog):
def send(_payload):
raise TypeError("broken client")
exc, value, recs = _run('{"id": 1}', send, caplog)
assert exc is None
assert value is None
assert len(recs) == 1
def test_send_timeout_returns_none(caplog):
def send(_payload):
raise TimeoutError("late")
exc, value, recs = _run('{"id": 1}', send, caplog)
assert exc is None
assert value is None
assert len(recs) == 1
def test_downstream_429_is_dict(caplog):
exc, value, recs = _run('{"id": 1}', lambda p: (429, "slow"), caplog)
assert exc is None
assert value == {"ok": False, "status": 429, "body": "slow"}
assert len(recs) == 1
def test_downstream_200_is_dict(caplog):
exc, value, recs = _run('{"id": 1}', lambda p: (200, {"n": 1}), caplog)
assert exc is None
assert value == {"ok": True, "status": 200, "body": {"n": 1}}
assert len(recs) == 0
Run the file before editing handlers. Use a short traceback during the first pin.
python -m pytest test_events_contract.py -q --tb=short
A green suite with no mutation check is weak. You must see a failed unified-error fork. Keep that fork out of the merge.
import events
def test_unified_raise_would_break_send_type_error(monkeypatch, caplog):
def rewritten(raw, send):
payload = __import__("json").loads(raw)
status, body = send(payload) # TypeError now escapes
return {"ok": status < 400, "status": status, "body": body}
monkeypatch.setattr(events, "dispatch_event", rewritten)
with pytest.raises(TypeError):
events.dispatch_event('{"id": 1}', lambda p: (_ for _ in ()).throw(TypeError("x")))
Expect this mutation test to fail on the rewrite. Restore the messy handler after that check. The failure is the evidence, not a vibe.
is None or catch types.except edit.
Step two is not optional. Tests invented from the callee miss caller branches. Caller branches are the contract.
rg -n "dispatch_event\(" -g "*.py"
rg -n "is None|except ValueError|except RuntimeError" -g "*.py"
Record each hit as a fixture name. Missing hits become missing rows. Missing rows make unsafe extracts look safe.
Do not narrow except Exception on send first. The table says TypeError becomes None. Narrowing that clause raises TypeError instead.
That raise is a contract break. Extract the block without narrowing. Keep the same log line and None return.
def _send_or_none(payload, send):
try:
return send(payload)
except Exception:
log.warning("send failed")
return None
def dispatch_event(raw, send):
if raw is None or raw == "":
raise RuntimeError("empty body")
payload = _parse_object(raw)
if "id" not in payload:
log.warning("missing id")
return None
result = _send_or_none(payload, send)
if result is None:
return None
status, body = result
if status >= 400:
log.warning("downstream %s", status)
return {"ok": False, "status": status, "body": body}
return {"ok": True, "status": status, "body": body}
That extract is boring. Boring is the point. The taxonomy stays in the table.
A later change can narrow exceptions. Do that only with a version note. Add a row that expects TypeError to escape. Tell callers before you merge.
The JSON branch is different. json.loads should raise json.JSONDecodeError. Re-raising ValueError is the published type.
Narrowing except Exception to except json.JSONDecodeError can be safe. Prove it with the invalid JSON row. Prove it with the JSON list row too.
def _parse_object(raw):
try:
payload = json.loads(raw)
except json.JSONDecodeError:
raise ValueError("bad json") from None
if not isinstance(payload, dict):
raise ValueError("bad json")
return payload
Both bad-JSON fixtures must still raise ValueError. A leaked JSONDecodeError is a taxonomy change. Do not ship that leak without a caller audit.
from None also drops __cause__. Add a cause fixture if any caller reads it. Skip that fixture when no caller inspects causes.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
A model can draft the extract. It must not invent a new taxonomy. MonkeyCode provides free model access and a free server option.
Use either only after the characterization file is green. Feed the model the table and the test file. Ask for one extract, not a rewrite.
Reject a patch that removes a row. Reject a patch that changes None into a raise. This is not an agent loop.
One prompt. One diff. Same tests.
The harness does not prove semantic equality. It pins types, keys, status, and log counts. It misses timing, retry storms, and byte identity.
Log counts break if a library logs extra warnings. Pin the logger name events. Cap propagation on that logger.
Do not pin the root logger. Root pins go red on unrelated imports. That noise hides a real taxonomy drift.
The table is only as good as the fixtures. One unlisted caller path is an untested shape. Untested shapes are how "safe" extracts land in incident channels.
Do not use this flow for greenfield APIs. Design one error shape there. Do not preserve None plus raises on purpose.
Do not use this flow for security boundaries. Characterization will pin insecure behavior. Pinning is not hardening.
Do not use this flow without tests you can run offline. A model server cannot replace the table. If pytest cannot collect locally, stop.
Teams with a published OpenAPI error schema may skip dict-key rows. Use the schema as the table instead. Still pin process-local exception escapes.
HTTP schemas often omit those local raises. Omitting them is how ValueError turns into a 500. Keep the escape column anyway.
If a model patch violates any line, drop the patch. Rewrite pressure is not evidence. The table is.
If the characterization file is already green, one extract prompt on the free server is enough. Skip that prompt when the table still has empty cells.