The Contract Discovery Bottleneck

A developer argues that as AI coding agents make implementation cheaper, verification becomes the bottleneck, because agents and their tests can share the same incomplete assumptions about what "correct" behavior means. The developer describes a coding agent that built a password reset flow with a reusable reset link, and notes that a reader later pointed out the specification failed to cover concurrent token reuse. The developer concludes that behavioral specifications must be treated as provisional and updated as new constraints are discovered.

AI can generate the code. We can verify the behavior. But who decides what correct means? I wrote recently https://www.kenwalger.com/blog/ai/verification-bottleneck-ai-generated-software/ about a coding agent that built me a password reset flow with a reset link that worked more than once. The bug survived because nobody had written down that a reset link should be single use. It was obvious right up until it wasn't. My argument was that as AI makes implementation cheaper, verification becomes the bottleneck. The feature request said "build password reset." The agent built password reset. The happy path worked. The tests passed. The implementation looked finished. What nobody had asked was whether the same reset link should work twice. So I added an independently written behavioral specification. The agent implemented against it. The verifier rejected the reusable token. The agent fixed the implementation. The verifier passed it. That seemed like a useful pattern: Then I published the article, and the comments started finding things my specification didn't say. That exposed a harder problem. One reader asked what would happen if two password reset requests using the same token arrived at the same time. I hadn't tested that. My test covered sequential reuse: But concurrent reuse is different: Both requests check the token while it is still unused. Both proceed. If validation and consumption are not a single atomic operation, "single use" can still produce two successful resets. The original invariant was incomplete. That doesn't make the specification useless. It makes the specification provisional. The interesting part is where the new knowledge goes. Once somebody discovers that "single use" also means competing attempts cannot both succeed, that should stop being knowledge held by the person who noticed it. It belongs in the durable definition of correct behavior. The specification changes. Which means the loop is really closer to this: That is messier than the first diagram. It is also much closer to engineering. Another reader described an integration builder where an agent wrote both a connector and the tests for that connector. Everything passed. Both were wrong. The connector and its tests encoded the same incorrect assumption about OAuth token refresh. The mistake only surfaced when a customer's token expired during a live session. The implementation and test suite were separate artifacts. They were not independent in the way that mattered. They shared an assumption. That distinction matters because "independent verification" can sound like an organizational property: None of those necessarily provides independence. If the implementation and verifier derive their definition of correct behavior from the same incomplete prompt, they can agree perfectly and still be wrong. The student is no longer literally grading the same exam. Two students have simply studied from the same incorrect answer key. Another commenter asked whether property-based testing or giving an agent an adversarial security persona might do a better job uncovering these unstated constraints. I think both are interesting, but they expose the same boundary. Property-based testing can explore a stated invariant extremely well. If I tell a framework: A successfully consumed reset token must never produce another successful reset. it can generate combinations and sequences I would never think to hand-author. But it cannot tell me that single use was a requirement if nobody expressed it. An adversarial agent has a similar problem. Asking a model to "try to break this" may produce better tests than asking it to "write tests for this feature." But if the adversary shares the same context, model assumptions, and incomplete understanding of the requirement, how independent is it really? The question starts shifting from who writes the tests? to a more difficult one: where does the definition of correct behavior come from? One of the most interesting examples in the discussion came from a verification harness rather than generated application code. A capability test timed out. The harness recorded the result as a failure. But a timeout did not establish that the capability failed. It established that the harness did not obtain a result within the allotted time. Those are different claims. FAILED and NOT TESTED are not interchangeable. The verifier had turned an observation failure into an assertion about capability. That's a useful warning for any architecture built around deterministic verification: deterministic does not mean correct. A verifier can enforce the wrong invariant with absolute consistency. So can a specification. The goal isn't to replace an unreliable agent with an infallible verifier. There is no infallible verifier. The goal is to make the definition of correctness explicit enough that it can be inspected, challenged, tested, and revised independently of the implementation. This was the question that pushed the argument furthest for me. If humans have to write complete behavioral specifications before agents can implement anything, haven't we simply moved the bottleneck back to humans? Probably. And worse, the concurrency example demonstrates that humans don't necessarily know the complete specification beforehand either. So "humans write the contract" isn't much of an answer. An agent could propose it. That sounds circular at first. If the agent proposes the implementation and proposes the contract, aren't we back to the student grading the exam? Only if proposing the contract and accepting the contract are the same operation. They don't have to be. An agent might generate a candidate operating contract: reset token: may be used once competing attempts cannot both succeed expires after N minutes cannot authorize a different account A human, another system, or some combination can then challenge that much smaller artifact. The question being reviewed becomes: Is this an adequate definition of correct behavior? rather than: Is this entire implementation correct? That doesn't solve the trust problem, but it reduces its surface area. Reviewing four lines is a different activity than reviewing four hundred. One is a conversation about intent. The other is an audit. But this runs straight back into the answer key problem. If the same model that will implement the feature also proposes the contract, they share assumptions. An agent that doesn't know single use matters won't propose single use as an invariant. It will produce a confident, well-formatted contract with the same hole in it, and now the hole has been written down and approved. So accepting a contract has to do more than approve it. It has to introduce something the proposing agent didn't have. That might be a person who has debugged this class of bug before. It might be a genuinely different model, though I'm unsure how much independence that buys. It might be a checklist derived from past incidents, which is really institutional memory in a form an agent can read. For a reset token, somebody's list somewhere already says: single use, expiry, no account substitution, no concurrent success, session invalidation. The value comes from the independence of the source, not from the ceremony of the review. That may be a more tractable thing to build tooling around than verification itself. This is where the comments changed my framing. I started with: I'm less sure that's where it stops. Once implementation is cheap and verification is increasingly automatable, the harder problem may become discovering the invariants worth verifying. Call it contract discovery. The requirement says: Reset my password. Somebody has to discover: The link works once. Then: Two concurrent attempts cannot both succeed. Then perhaps: The token cannot authorize a different account. A token issued before another successful reset may no longer be valid. A reset invalidates existing sessions. Some of those are product decisions. Some are security properties. Some are implementation-independent behavioral invariants. Some may not apply at all. The difficult work is deciding which ones belong to the definition of correct. AI can help propose them. Property-based testing can explore them. Deterministic systems can enforce them. Production incidents will unfortunately discover some of them for us. But none of those eliminates the need to decide which claims actually define correctness. There is another reason I think this matters beyond generated code: agents don't just write things anymore. They call things. An agent calls an API. The response is 200 . The agent moves on. But a 200 says the request was processed. It doesn't say the constraint the agent's plan depended on was enforced. Maybe the call timed out after the write succeeded, so the retry performed the effect twice. Maybe the operation was legitimate the first time and should have been rejected the second. That second one should look familiar. It's the reset link, one layer out. A bad implementation leaves an artifact somebody can inspect later. A bad tool call already happened. It sent the email. Charged the card. Revoked the access. Posted the message. There is no diff to read. This is where the contract-discovery problem becomes more consequential. The system needs some definition of what the agent is permitted to cause and what evidence would establish that the intended effect actually happened. I don't think I have the architecture for that yet, but one boundary is becoming clearer: The specification can be agent-readable without being agent-owned. The agent should be able to see the invariant. Withholding the requirement only makes the work guesswork. But the agent shouldn't be able to quietly redefine the invariant when satisfying it becomes inconvenient. Whatever accepts, stores, and evaluates the contract needs some independence from the reasoning that produced the implementation or action. Where that boundary belongs remains a harder question. Calling the specification a durable artifact can sound like calling it an immutable one. I don't mean that. A durable specification should change when we learn something about what correct behavior actually requires. What makes it durable is that the knowledge survives the implementation that taught us the lesson. The reset implementation may be rewritten next month. The framework may change. The agent may change. The database may change. But once we've established that two competing reset attempts cannot both succeed, that invariant should survive all of them. The same applies to an integration. Once a production failure teaches us what token refresh must guarantee, that knowledge should not remain attached to the incident report or the engineer who debugged it. It should become part of what "correct connector" means. The implementation may be disposable. The accumulated definition of correctness is not. There are plenty of uncomfortable questions left. How independent does a verifier have to be? Can two agents using different prompts but the same underlying model provide meaningful independence? Who accepts an agent-proposed contract? How do you distinguish a genuine product invariant from an implementation detail that shouldn't survive the current code? What happens when two valid invariants conflict? How do contracts evolve without quietly weakening previous guarantees? And how do we verify effects in external systems where state is delayed, partially observable, or distributed? I don't have good answers to all of those. That's partly why I don't think the answer is simply "write better tests." The tests are downstream of the harder question. The original password-reset bug happened because a rule existed in someone's head and nowhere else. The comments on that experiment showed the next problem: writing down one rule doesn't mean you've found all the others. That's fine. The specification doesn't have to arrive complete. It has to provide somewhere for discovered invariants to go, and that somewhere has to be a place with a history: versioned, reviewable, and attached to the behavior rather than to the incident that revealed it. Maybe an agent proposes them. Maybe a human notices them. Maybe property-based testing exposes them. Maybe an independent reviewer asks the annoying question nobody else asked. And sometimes production will teach us the expensive way. The important part is that each discovery makes the durable definition of correct behavior better. AI is making it remarkably cheap to turn an instruction into working code. Verification asks whether the code did what we said. Contract discovery asks whether we said enough. I'm starting to think that's the harder problem.