{"slug": "acceptance-criteria-must-be-provable", "title": "Acceptance Criteria Must Be Provable", "summary": "A blog post argues that acceptance criteria for software tickets, including work delegated to AI coding agents, are only useful when each one can be proven true or false, and recommends rewriting vague criteria into runnable checks. The post's stated rule is \"If you can't prove a criterion false, it isn't an acceptance criterion,\" and it illustrates the point by replacing \"cancellation works correctly\" with two testable criteria: stop future renewal charges, and keep access until the end of the paid period. It also advises giving the implementing agent read-only access to criteria and requiring it to report the exact command, test name, measurement, or screenshot behind each criterion before claiming the work done.", "body_md": "# Acceptance Criteria Must Be Provable\n\n## tl;dr\n\nAcceptance criteria only help if each one can be proven true or false. “Works well” and “is fast” can’t. Rewrite them into a check you can run: a test, a command, a measurement, or a screenshot. Don’t let an agent claim done on a criterion it hasn’t proven. Ask for the evidence, one criterion at a time.\n\n**Key takeaway:** If you can’t prove a criterion false, it isn’t an acceptance criterion.\n\n## A criterion you can’t check is a wish\n\nIn [Let Agents Review Behavior and Intent Too](https://matthewboston.com/blog/review-behavior-and-intent.html), I said to give a reviewing agent the issue and its acceptance criteria, then have it report which requirements the change satisfies and which it misses.\n\nThat only works if the criteria can be checked. “The cancellation flow works well” has no yes or no. A reviewer who tries to verify it ends up guessing what “well” meant. An agent will guess too, and it will sound sure about it.\n\n[Whether You Think AI Can or Can’t, You’re Right](https://matthewboston.com/blog/whether-you-think-ai-can-or-cant-youre-right.html) says a missing definition of done is a setup failure. A vague definition is the same failure with more words. You gave the agent a target it can’t miss, because it can’t be measured.\n\n## Rewrite until a check can fail\n\nTake the cancellation example from [Behavior Tests Tell You Why the Code Exists](https://matthewboston.com/blog/behavior-tests-tell-you-why-the-code-exists.html). A ticket that says “cancellation works correctly” can’t be proven. These can:\n\n- Stop future renewal charges.\n- Keep access until the end of the paid period.\n\nEach one has a check. Create a subscription that would renew on October 1, cancel it, and assert that no charge is due that day. Create one paid through September 30, cancel it, and assert that access still holds on September 15. If those fail, the criterion is false. If you can’t write a check like that, the criterion is still a wish.\n\nVague words on tickets need the same treatment. Rewrite “The checkout is fast” to “A checkout of 10 line items finishes in under 200ms when you run the repo’s checkout benchmark.” Someone can run the command and get a yes or no. Rewrite “The settings page looks good” to “The save button stays disabled until every required field is valid.” A screenshot of the form with the email field empty and the button disabled backs it up.\n\nIf a criterion still says “well,” “fast,” “intuitive,” or “polished,” it isn’t done being written.\n\n## Don’t claim done without evidence\n\nAn agent will often call the work finished as soon as the file is written. A green test suite isn’t enough either, because green tests don’t prove the agent built what you asked for. The tests can encode the same misunderstanding as the code. That’s why [Review the Outcome, Not the Output](https://matthewboston.com/blog/review-the-outcome-not-the-output.html) asks whether the tests check what matters, not whether the agent wrote tests.\n\nSo write the criteria before the work starts, and keep them out of the worker’s write path. That’s the same rule as [Describe the What, Delegate the How](https://matthewboston.com/blog/describe-the-what-delegate-the-how.html). The agent doing the work shouldn’t get to edit the definition of done. It gets read access only. The check has to live outside the agent that did the work.\n\nWhen you ask an agent to implement a ticket, ask it to prove each criterion. This is an example to try, not a report of a workflow I’ve already run.\n\n```\nImplement this ticket against the acceptance criteria below.\nDo not edit the criteria or the tests that encode them.\n\nFor each criterion, report:\n- The check you ran: the exact command, test name, measurement, or screenshot\n- The result\n- If you could not prove it, say so and stop short of claiming done\n\nA passing suite is not evidence for a criterion that has no check.\nReading a test is not a passing test run.\nDo not claim done until every criterion has evidence or an explicit gap.\n```\n\nA human still has to look at that evidence. If the criterion was “keep access until the end of the paid period” and the only test asserts that a `cancel` method was called, that isn’t proof. Send it back.\n\nAcceptance criteria are how you and the agent agree what done means. If a line can’t be proven, it isn’t a criterion yet. Rewrite it before the work starts.", "url": "https://wpnews.pro/news/acceptance-criteria-must-be-provable", "canonical_source": "https://matthewboston.com/blog/acceptance-criteria-must-be-provable.html", "published_at": "2026-11-12 15:40:41+00:00", "updated_at": "2026-11-12 15:51:13.854332+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools"], "entities": [], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/acceptance-criteria-must-be-provable", "markdown": "https://wpnews.pro/news/acceptance-criteria-must-be-provable.md", "text": "https://wpnews.pro/news/acceptance-criteria-must-be-provable.txt", "jsonld": "https://wpnews.pro/news/acceptance-criteria-must-be-provable.jsonld"}}