I built a deterministic OpenAPI diff tool because I was tired of LLM-written release notes that hallucinated breaking changes. Feed it a base spec and a new spec, get back a structured changelog. No model in the loop, no vibes — just a tree walk and a diff.
The first version was naive. I compared paths, operations, schemas. If a property was added to a response schema, I flagged it as non-breaking. If a required request field appeared, I flagged it as breaking. Clean, defensible, wrong.
Then a user wired it into their CI and their release pipeline started crying wolf. Turns out "added a required field with a default value" is breaking in spec-land but non-breaking in reality-land — most HTTP clients tolerate defaults fine. And "renamed an enum value" is technically the same type but every downstream consumer that string-matches is now on fire.
So I had to encode judgment. Not AI judgment — deterministic judgment. The tool now classifies changes into tiers: definitely breaking (removed path, changed type, new required field without default), probably breaking (renamed enum value, tightened validation pattern, removed response property), additive (new optional field, new path, new operation). Each tier carries a human-readable explanation of why it's classified that way, so a human can override without reading the diff.
The lesson: a spec diff isn't a git diff. Git diffs bytes. OpenAPI diffs contracts, and contracts have semantics that live in the minds of the humans who wrote the clients. My tool can't read minds. But it can be honest about its uncertainty — and that turns out to be the feature people actually want. "Here's what I'm sure about, here's what I'm guessing about, here's the command to tell me I'm wrong."
If you're versioning an API and writing changelogs by hand, the deterministic part is freeing. You stop arguing about whether something is breaking and start arguing about whether your classification rules match your actual consumers. That's a much better argument. I packaged it as https://x402.freeq.one/tools/changelog_openapi.html — give it two spec URLs and see what it thinks changed.