{"slug": "how-i-keep-an-llm-from-inventing-breaking-changes", "title": "How I keep an LLM from inventing breaking changes", "summary": "A developer building a release-notes tracker for React, TypeScript, Vite and similar projects described a pipeline that prevents an LLM from inventing breaking changes, including a system prompt restricting the model to supplied release notes, a schema-constrained output with an explicit \"breakingChanges\" list, and verbatim-quote citations resolved by string search rather than model-generated line numbers. The developer reported validating the approach against an evaluation set of 14 real releases from 8 projects, five of them outside the tool's catalogue to avoid tuning the product to the test.", "body_md": "I'm building a tool that reads every new release of React, TypeScript, Vite and friends, and tells you two things: what changed, and whether you need to touch your code before upgrading.\n\nThe summarising part is easy. Any decent model will write you a tidy paragraph about a release.\n\nThe part that decides whether the tool is worth anything is different: **when it says \"action required\", is that true?** A tracker that invents a breaking change is worse than no tracker, because you'll go looking for a migration that doesn't exist. And a tracker that misses one is worse still.\n\nHere is everything I do about that, in the order I had to learn it.\n\nThe system prompt says it in the second line:\n\nUse only the release notes in the user message. Do not use what you know about the project from elsewhere.\n\nThis matters more than it sounds. A model asked about \"React 19.2.0\" has opinions about React 19 from training. Those opinions are often right, which is exactly what makes them dangerous — they're right until the release where they aren't, and you have no way to tell which one you're looking at.\n\nSo the notes go into the user message, wrapped in a tag, and nothing else about the project does.\n\nTwo consequences follow immediately:\n\n```\nif (notes.trim() === \"\") return { ok: false, reason: \"release notes are empty\" };\nif (notes.length > MAX_NOTES_LENGTH) {\n  return { ok: false, reason: `release notes are ${notes.length} characters, over the ${MAX_NOTES_LENGTH} limit` };\n}\n```\n\nBoth are failures I'd rather see than paper over.\n\nThe prompt doesn't ask the model whether you should act. It asks for a list:\n\n\"breakingChanges\": every change the notes state or clearly describe as breaking, removed or incompatible, so that users must change code, configuration or runtime to upgrade.\n\n\"actionRequired\": true exactly when \"breakingChanges\" is not empty.\n\nAnd then, flatly:\n\nNever infer a breaking change the notes do not mention, even if the version number suggests one.\n\nThat last line earns its place. Without it, a `x.0.0` invites the model to reason \"this is a major, majors break things\" and produce something. Version numbers are a *classification* input for me, never evidence about content.\n\nThe output shape is constrained with a schema, so a reply that isn't that shape never reaches the database. But keep reading — schemas do less than you think.\n\nEach key point and each breaking change links back into the official notes, to the line it came from. On the page it looks like `release-notes:20`, and clicking it opens the notes, scrolls to line 20 and puts a caret at the right column.\n\nThe obvious implementation is to ask the model for the line number. Don't. Models count badly, and — worse — they count badly *invisibly*: you get `42`, it looks like a line number, and it's off by nine.\n\nSo the model returns a **verbatim quote** instead:\n\nA \"quote\" is the stretch of the release notes the item comes from, copied character for character [...] Never adjust, shorten with an ellipsis or rewrite a quote to fit: a quote that does not appear verbatim in the notes is dropped.\n\nAnd I compute the position myself, by searching my stored copy of the notes for that string:\n\n```\nlocateCitation(notes, quote) // → { fromLine, toLine, column, length } | null\n```\n\n`null` means the quote isn't in the notes. Which means the model made it up. Which means no pointer is rendered.\n\nThis is the part I'm most pleased with, because of what it collapses. \"The model must not hallucinate citations\" and \"references must stay inside the document\" sound like two separate things to test. They're the same thing, and it's a string search. The column offset for the caret falls out for free.\n\nEverything above is reasoning about a prompt. Reasoning about a prompt is how you end up with a prompt that feels great and is wrong.\n\nSo: an evaluation set. 14 real releases, from 8 projects — 5 of them not even in my catalogue, so I couldn't accidentally tune the product to the test. The notes are **copied into the repo**, not read from the database. The catalogue changes; the set must not, or two reports a month apart mean nothing.\n\nThe human judgement lives in a `set.json` next to them, and it's more nuanced than a pass/fail:\n\n`actionRequired`, which can be `true`, `false`, or `\"either\"`\nThe comparison itself is a pure function with its own unit tests. The script around it only does I/O. Two prompt versions, one batch each:\n\n|  | v3 | v4 | \n|---|---|---|\n| cases evaluated | 13/14 | **14/14** | \n| \"action required\" correct | 13/13 | **14/14** | \n| missing breaking changes | 0 | **0** | \n| invented citations | 1 | **0** | \n| verbatim quotes | 66 of 72 | **77 of 81** | \n\nSix of the fourteen are deliberate duds: patch releases whose notes say nothing but \"see the changelog\". All six came back \"no action needed\". That was the point of including them — I wanted evidence that the model shuts up when there's nothing to say, not just that it's eloquent when there is.\n\nTwo full runs cost about 50 cents through the Batch API. Roughly $0.02 a summary.\n\nBoth of these survived a fake adapter, unit tests and integration tests. Both died within minutes of the first real batch.\n\n**Line endings.** Release notes fetched from GitHub contain `CRLF`. The model quotes them back with `LF`. The page searches the normalised text. So any quote spanning more than one line silently failed to locate — no crash, no log, just a pointer that quietly wasn't there. In production. `locateCitation` now normalises both sides.\n\n**A sixth key point.** The prompt asks for 3 to 5 key points, and the schema says so too. A reply came back with six, failed validation, and threw away a summary I had already paid for.\n\nThe lesson is worth stating plainly: **structured output constrains the shape of a reply, never its counts.** You can guarantee \"an array of objects with these fields\". You cannot guarantee \"at most five\". So now the parser keeps the first five — they arrive in order of importance — and the prompt states the limit as a hard rule with the consequence spelled out. That change is what took the prompt to v4, and it's why the \"cases evaluated\" row went from 13 to 14.\n\nA smaller thing, but it's the same instinct. When the Anthropic account runs out of credit, the affected releases go back to `pending`, not to `failed`. `failed` is final — those summaries would never have regenerated, even after topping up.\n\nAnd the case is **typed**, not sniffed out of an error message: `submit` throws a `CreditExhaustedError`, `collect` returns `credit_exhausted`. \"Other errors must not be mistaken for exhausted credit\" is then something the compiler enforces, rather than something I promise.\n\nIf I did this again from scratch, three things carry over:\n\nThe tracker is live and collecting releases while I build the rest of it. The summaries aren't switched on in production yet — that's the next thing, and it's the first part of this project that costs money to leave running.", "url": "https://wpnews.pro/news/how-i-keep-an-llm-from-inventing-breaking-changes", "canonical_source": "https://dev.to/robzepdev/how-i-keep-an-llm-from-inventing-breaking-changes-5625", "published_at": "2026-09-24 14:13:05+00:00", "updated_at": "2026-09-24 14:32:31.260239+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools", "ai-agents"], "entities": ["React", "TypeScript", "Vite"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-i-keep-an-llm-from-inventing-breaking-changes", "markdown": "https://wpnews.pro/news/how-i-keep-an-llm-from-inventing-breaking-changes.md", "text": "https://wpnews.pro/news/how-i-keep-an-llm-from-inventing-breaking-changes.txt", "jsonld": "https://wpnews.pro/news/how-i-keep-an-llm-from-inventing-breaking-changes.jsonld"}}