{"slug": "the-contract-discovery-bottleneck", "title": "The Contract Discovery Bottleneck", "summary": "A developer argues that as AI coding agents make implementation cheaper, verification becomes the bottleneck, because agents and their tests can share the same incomplete assumptions about what \"correct\" behavior means. The developer describes a coding agent that built a password reset flow with a reusable reset link, and notes that a reader later pointed out the specification failed to cover concurrent token reuse. The developer concludes that behavioral specifications must be treated as provisional and updated as new constraints are discovered.", "body_md": "*AI can generate the code. We can verify the behavior. But who decides what correct means?*\n\nI [wrote recently](https://www.kenwalger.com/blog/ai/verification-bottleneck-ai-generated-software/) about a coding agent that built me a password reset\n\nflow with a reset link that worked more than once.\n\nThe bug survived because nobody had written down that a reset link\n\nshould be single use. It was obvious right up until it wasn't.\n\nMy argument was that as AI makes implementation cheaper, verification\n\nbecomes the bottleneck. The feature request said \"build password reset.\"\n\nThe agent built password reset. The happy path worked. The tests passed.\n\nThe implementation looked finished.\n\nWhat nobody had asked was whether the same reset link should work twice.\n\nSo I added an independently written behavioral specification. The agent\n\nimplemented against it. The verifier rejected the reusable token. The\n\nagent fixed the implementation. The verifier passed it.\n\nThat seemed like a useful pattern:\n\nThen I published the article, and the comments started finding things my\n\nspecification didn't say. That exposed a harder problem.\n\nOne reader asked what would happen if two password reset requests using\n\nthe same token arrived at the same time.\n\nI hadn't tested that. My test covered sequential reuse:\n\nBut concurrent reuse is different:\n\nBoth requests check the token while it is still unused. Both proceed.\n\nIf validation and consumption are not a single atomic operation, \"single\n\nuse\" can still produce two successful resets.\n\nThe original invariant was incomplete.\n\nThat doesn't make the specification useless. It makes the specification\n\nprovisional.\n\nThe interesting part is where the new knowledge goes.\n\nOnce somebody discovers that \"single use\" also means competing attempts\n\ncannot both succeed, that should stop being knowledge held by the person\n\nwho noticed it. It belongs in the durable definition of correct\n\nbehavior.\n\nThe specification changes.\n\nWhich means the loop is really closer to this:\n\nThat is messier than the first diagram.\n\nIt is also much closer to engineering.\n\nAnother reader described an integration builder where an agent wrote\n\nboth a connector and the tests for that connector.\n\nEverything passed.\n\nBoth were wrong.\n\nThe connector and its tests encoded the same incorrect assumption about\n\nOAuth token refresh. The mistake only surfaced when a customer's token\n\nexpired during a live session.\n\nThe implementation and test suite were separate artifacts. They were not\n\nindependent in the way that mattered.\n\nThey shared an assumption.\n\nThat distinction matters because \"independent verification\" can sound\n\nlike an organizational property:\n\nNone of those necessarily provides independence.\n\nIf the implementation and verifier derive their definition of correct\n\nbehavior from the same incomplete prompt, they can agree perfectly and\n\nstill be wrong.\n\nThe student is no longer literally grading the same exam.\n\nTwo students have simply studied from the same incorrect answer key.\n\nAnother commenter asked whether property-based testing or giving an\n\nagent an adversarial security persona might do a better job uncovering\n\nthese unstated constraints.\n\nI think both are interesting, but they expose the same boundary.\n\nProperty-based testing can explore a stated invariant extremely well.\n\nIf I tell a framework:\n\n  A successfully consumed reset token must never produce another\n  successful reset.\n\nit can generate combinations and sequences I would never think to\n\nhand-author.\n\nBut it cannot tell me that single use was a requirement if nobody\n\nexpressed it.\n\nAn adversarial agent has a similar problem. Asking a model to \"try to\n\nbreak this\" may produce better tests than asking it to \"write tests for\n\nthis feature.\" But if the adversary shares the same context, model\n\nassumptions, and incomplete understanding of the requirement, how\n\nindependent is it really?\n\nThe question starts shifting from *who writes the tests?* to a more\n\ndifficult one: **where does the definition of correct behavior come from?**\n\nOne of the most interesting examples in the discussion came from a\n\nverification harness rather than generated application code.\n\nA capability test timed out.\n\nThe harness recorded the result as a failure.\n\nBut a timeout did not establish that the capability failed. It\n\nestablished that the harness did not obtain a result within the allotted\n\ntime.\n\nThose are different claims.\n\n```\nFAILED\n```\n\nand\n\n```\nNOT TESTED\n```\n\nare not interchangeable.\n\nThe verifier had turned an observation failure into an assertion about\n\ncapability.\n\nThat's a useful warning for any architecture built around deterministic\n\nverification: deterministic does not mean correct.\n\nA verifier can enforce the wrong invariant with absolute consistency.\n\nSo can a specification.\n\nThe goal isn't to replace an unreliable agent with an infallible\n\nverifier. There is no infallible verifier.\n\nThe goal is to make the definition of correctness explicit enough that\n\nit can be inspected, challenged, tested, and revised independently of\n\nthe implementation.\n\nThis was the question that pushed the argument furthest for me.\n\nIf humans have to write complete behavioral specifications before agents\n\ncan implement anything, haven't we simply moved the bottleneck back to\n\nhumans?\n\nProbably.\n\nAnd worse, the concurrency example demonstrates that humans don't\n\nnecessarily know the complete specification beforehand either.\n\nSo \"humans write the contract\" isn't much of an answer.\n\nAn agent could propose it.\n\nThat sounds circular at first. If the agent proposes the implementation\n\nand proposes the contract, aren't we back to the student grading the\n\nexam?\n\nOnly if proposing the contract and accepting the contract are the same\n\noperation.\n\nThey don't have to be.\n\nAn agent might generate a candidate operating contract:\n\n```\nreset token:\n  may be used once\n  competing attempts cannot both succeed\n  expires after N minutes\n  cannot authorize a different account\n```\n\nA human, another system, or some combination can then challenge that\n\nmuch smaller artifact.\n\nThe question being reviewed becomes:\n\n  Is this an adequate definition of correct behavior?\n\nrather than:\n\n  Is this entire implementation correct?\n\nThat doesn't solve the trust problem, but it reduces its surface area.\n\nReviewing four lines is a different activity than reviewing four hundred.\n\nOne is a conversation about intent. The other is an audit.\n\nBut this runs straight back into the answer key problem.\n\nIf the same model that will implement the feature also proposes the\n\ncontract, they share assumptions. An agent that doesn't know single use\n\nmatters won't propose single use as an invariant. It will produce a\n\nconfident, well-formatted contract with the same hole in it, and now the\n\nhole has been written down and approved.\n\nSo accepting a contract has to do more than approve it. It has to\n\nintroduce something the proposing agent didn't have.\n\nThat might be a person who has debugged this class of bug before. It\n\nmight be a genuinely different model, though I'm unsure how much\n\nindependence that buys. It might be a checklist derived from past\n\nincidents, which is really institutional memory in a form an agent can\n\nread. For a reset token, somebody's list somewhere already says: single\n\nuse, expiry, no account substitution, no concurrent success, session\n\ninvalidation.\n\nThe value comes from the independence of the source, not from the\n\nceremony of the review.\n\nThat may be a more tractable thing to build tooling around than\n\nverification itself.\n\nThis is where the comments changed my framing.\n\nI started with:\n\nI'm less sure that's where it stops.\n\nOnce implementation is cheap and verification is increasingly\n\nautomatable, the harder problem may become discovering the invariants\n\nworth verifying.\n\nCall it contract discovery.\n\nThe requirement says:\n\n  Reset my password.\n\nSomebody has to discover:\n\n  The link works once.\n\nThen:\n\n  Two concurrent attempts cannot both succeed.\n\nThen perhaps:\n\n  The token cannot authorize a different account.\n  \n  A token issued before another successful reset may no longer be valid.\n  \n  A reset invalidates existing sessions.\n\nSome of those are product decisions. Some are security properties. Some\n\nare implementation-independent behavioral invariants. Some may not apply\n\nat all.\n\nThe difficult work is deciding which ones belong to the definition of\n\ncorrect.\n\nAI can help propose them.\n\nProperty-based testing can explore them.\n\nDeterministic systems can enforce them.\n\nProduction incidents will unfortunately discover some of them for us.\n\nBut none of those eliminates the need to decide which claims actually\n\ndefine correctness.\n\nThere is another reason I think this matters beyond generated code:\n\nagents don't just write things anymore. They call things.\n\nAn agent calls an API. The response is `200`. The agent moves on.\n\nBut a `200` says the request was processed. It doesn't say the\n\nconstraint the agent's plan depended on was enforced. Maybe the call\n\ntimed out after the write succeeded, so the retry performed the effect\n\ntwice. Maybe the operation was legitimate the first time and should have\n\nbeen rejected the second.\n\nThat second one should look familiar. It's the reset link, one layer\n\nout.\n\nA bad implementation leaves an artifact somebody can inspect later.\n\nA bad tool call already happened.\n\nIt sent the email. Charged the card. Revoked the access. Posted the\n\nmessage.\n\nThere is no diff to read.\n\nThis is where the contract-discovery problem becomes more consequential.\n\nThe system needs some definition of what the agent is permitted to cause\n\nand what evidence would establish that the intended effect actually\n\nhappened.\n\nI don't think I have the architecture for that yet, but one boundary is\n\nbecoming clearer:\n\n**The specification can be agent-readable without being agent-owned.**\n\nThe agent should be able to see the invariant. Withholding the\n\nrequirement only makes the work guesswork.\n\nBut the agent shouldn't be able to quietly redefine the invariant when\n\nsatisfying it becomes inconvenient.\n\nWhatever accepts, stores, and evaluates the contract needs some\n\nindependence from the reasoning that produced the implementation or\n\naction.\n\nWhere that boundary belongs remains a harder question.\n\nCalling the specification a durable artifact can sound like calling it\n\nan immutable one.\n\nI don't mean that.\n\nA durable specification should change when we learn something about what\n\ncorrect behavior actually requires.\n\nWhat makes it durable is that the knowledge survives the implementation\n\nthat taught us the lesson.\n\nThe reset implementation may be rewritten next month.\n\nThe framework may change.\n\nThe agent may change.\n\nThe database may change.\n\nBut once we've established that two competing reset attempts cannot both\n\nsucceed, that invariant should survive all of them.\n\nThe same applies to an integration. Once a production failure teaches us\n\nwhat token refresh must guarantee, that knowledge should not remain\n\nattached to the incident report or the engineer who debugged it.\n\nIt should become part of what \"correct connector\" means.\n\nThe implementation may be disposable. The accumulated definition of\n\ncorrectness is not.\n\nThere are plenty of uncomfortable questions left.\n\nHow independent does a verifier have to be?\n\nCan two agents using different prompts but the same underlying model\n\nprovide meaningful independence?\n\nWho accepts an agent-proposed contract?\n\nHow do you distinguish a genuine product invariant from an\n\nimplementation detail that shouldn't survive the current code?\n\nWhat happens when two valid invariants conflict?\n\nHow do contracts evolve without quietly weakening previous guarantees?\n\nAnd how do we verify effects in external systems where state is delayed,\n\npartially observable, or distributed?\n\nI don't have good answers to all of those. That's partly why I don't\n\nthink the answer is simply \"write better tests.\" The tests are\n\ndownstream of the harder question.\n\nThe original password-reset bug happened because a rule existed in\n\nsomeone's head and nowhere else.\n\nThe comments on that experiment showed the next problem: writing down\n\none rule doesn't mean you've found all the others.\n\nThat's fine. The specification doesn't have to arrive complete. It has\n\nto provide somewhere for discovered invariants to go, and that somewhere\n\nhas to be a place with a history: versioned, reviewable, and attached to\n\nthe behavior rather than to the incident that revealed it.\n\nMaybe an agent proposes them. Maybe a human notices them. Maybe\n\nproperty-based testing exposes them. Maybe an independent reviewer asks\n\nthe annoying question nobody else asked. And sometimes production will\n\nteach us the expensive way.\n\nThe important part is that each discovery makes the durable definition\n\nof correct behavior better.\n\nAI is making it remarkably cheap to turn an instruction into working\n\ncode.\n\nVerification asks whether the code did what we said. Contract discovery\n\nasks whether we said enough. I'm starting to think that's the harder\n\nproblem.", "url": "https://wpnews.pro/news/the-contract-discovery-bottleneck", "canonical_source": "https://dev.to/kenwalger/the-contract-discovery-bottleneck-48jb", "published_at": "2026-09-10 16:44:00+00:00", "updated_at": "2026-09-10 16:52:33.976770+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "developer-tools", "ai-tools"], "entities": ["Ken Walger"], "alternates": {"html": "https://wpnews.pro/news/the-contract-discovery-bottleneck", "markdown": "https://wpnews.pro/news/the-contract-discovery-bottleneck.md", "text": "https://wpnews.pro/news/the-contract-discovery-bottleneck.txt", "jsonld": "https://wpnews.pro/news/the-contract-discovery-bottleneck.jsonld"}}