The Hard Part of AI Coding Is the Second Change

A synthetic experiment against Uncle's verifier found that a signed attestation of a build whose tests and audit had failed still returned VERIFIED, because the verifier by default reads its trust root from the repository, allowing an attacker's key and a replaced trust file to authenticate the record. The author, writing on brian.biz, argues that attestation validity and release readiness are separate questions and that agents must not be able to change the rules governing their own acceptance, proposing that independent workflows produce evidence while humans define acceptance policy and approve requirement changes. The 15-case transcript covers single-change verifier behavior only, with the harness controlling keys and approvals and the verifier not rerunning tests.

The Hard Part of AI Coding Is the Second Change An AI agent builds a payment service. The tests pass. The demo works. Then you ask for retries. The agent adds them, but a timeout can mean the charge succeeded and the response got lost. Now a retry charges the customer twice. The feature works. The system got worse. The test suite may even be larger. Quantity has always been easier to measure than judgment. AI makes building something that works once cheaper. Maintaining it means preserving what must stay true. Payments must not duplicate. Users must not see another tenant's data. Migrations must tolerate the previous application version. Before accepting a change, I want four answers: - Specification: What must remain true? - Validation: Which checks support that claim? - Provenance: Does the evidence describe the code we're shipping? - Authorization: Who can approve it? Use CI and branch protection. Bind evidence to the exact code, and put human gates where requirements change or risk is accepted. An agent's permission to work isn't permission to approve itself. Agents propose changes. Independent workflows produce evidence, and humans define the acceptance policy, approve changes to requirements, and accept risk. Routine changes may pass automatically under that policy, but agents must not be able to change the rules governing their own acceptance. What a valid signature does and doesn't tell you I tested this with a synthetic experiment https://brian.biz/blog/ai-agents-keep-it-working/experiment/verify demo.py against Uncle https://github.com/unclehq/uncle 's verifier. In one case, a signed record of a build whose tests and audit failed came back VERIFIED . The record was authentic under the fixture's trust root. That didn't make the software ready to ship. These are two separate questions: - Attestation validity: Is this record authentic, intact, and bound to the expected artifact? - Release readiness: Does the record satisfy the required tests, reviews, approvals and acceptance policy? The verifier answers the first question. A verifier that authenticates a record of failed tests is behaving correctly, and its output said so: "NOT READY ... attested as recorded, not as correct." The failure occurs when another system mistakes authenticity for authorization to release. An authentic record of failed tests is not evidence that a release should proceed. The 15 cases full transcript https://brian.biz/blog/ai-agents-keep-it-working/experiment/results.json cover single-change verifier behavior only. The harness controls the keys and approvals, and the verifier doesn't rerun tests. They say nothing about independent trust or a production workflow. The trust boundary Any scheme like this assumes the agent cannot modify the signing keys, the verifier's trust roots, or the effective acceptance policy. It also assumes that CI configuration and tests the agent can edit don't silently become trusted policy. If the agent can change any of these, a signature only authenticates a record produced inside the same compromised boundary. The experiment shows a case of this. An attacker's key signed the attestation and the repository's own trust file was replaced to match. The verifier returned VERIFIED, because by default it reads its trust root from the repo. Supplying the root from outside the repo made the same attestation fail. A local approval prompt isn't cryptographic proof of human intent either. A proposal anyone can implement The question is how to preserve that evidence across successive changes without allowing the agent to approve its own work. None of these primitives are new. Git provides content-addressed history, in-toto and SLSA define provenance and attestations, Sigstore handles signing and identity, and CI with branch protection enforces much of this inside a pull request. The open problem is connecting requirements, evidence and approvals across agent-driven changes without letting the agent certify its own work. Git records how code changed. This graph would record which changes were accepted, under which requirements and policies, and on the basis of which evidence. The two histories are related, but they answer different questions. CI already records test results, and provenance systems already bind evidence to artifacts. The addition is linking the acceptance decisions themselves, across successive changes. What follows is a proposed design, not something I'm claiming Uncle fully does today. 1. Attest each accepted change. The record holds the source tree hash, the requirements and policy version, hashes of the test and review evidence, and the identities and scope of human approvals. A trusted workflow signs it with a key the coding agent cannot use. 2. Link to what came before. Each attestation includes the hashes of the preceding accepted attestations. These links form a directed acyclic graph DAG : a normal change references its predecessor, and a merge references every parent and adds validation of the combined result. Parent approvals alone don't approve a merge.For example, suppose two agents independently produce changes B and C from accepted state A, and both pass their required checks. Merging B and C produces state D, which has a new tree identity. D needs its own validation and acceptance evidence; the approvals for B and C do not automatically approve their combination. 3. Verify by walking the graph. Starting from the current PR, a verifier follows the links to an explicitly trusted starting point, checks signatures against independently managed trust roots, matches evidence to artifacts, and enforces the acceptance policy. Missing evidence, failed required checks or absent approvals must prevent acceptance. Verified history can be cached, provided policy and trust changes invalidate the affected entries. Changes to the acceptance policy need independent authorization too. An agent can't loosen the requirements it is judged against, so a rejected change doesn't become acceptable because the policy moved under it. An authorized policy change can legitimately alter future decisions or trigger explicit reevaluation under the new policy. 4. Keep the records out of the agent's reach. Store the evidence durably, and anchor accepted graph heads outside the agent's control so a rewritten history can be detected. Cryptography protects the records and their relationships. It cannot prove the tests were sufficient or the software is correct. You can build this around existing CI and review tools, whether or not you use Uncle. Back to the retries Here is how that design would handle the payment change: 1. The agent implements retries. 2. Policy requires idempotency evidence, so those tests run in a workflow the agent doesn't control. 3. The results are bound to the exact source tree, and that workflow signs the attestation. 4. A failed required check blocks acceptance, however confident the agent's summary is. 5. The next change references this one, but approvals don't carry over where the code or policy they covered has changed. It still can't guarantee correctness. The tests might miss a race condition, and production will find paths the validation environment never saw. The four questions govern whether a change is accepted. They don't replace observability, rollback, incident response or engineering judgment. Generating the next version is getting cheap. Deciding whether to accept it is still the job.