AI Agent Definition of Done: Make Coding Agents Prove the Task Is Complete

A developer outlined a model- and framework-agnostic approach for gating AI coding agents with an executable "definition of done": a per-task contract specifying allowed paths, acceptance criteria, required evidence, and an independent process that accepts or rejects completion. The contract is paired with a receipt format that records command, exit code, environment, artifact path and timestamp for each criterion, so agents can propose completion but cannot grade their own work. The guide argues that model-generated explanations, unverified screenshots and green test commands without captured output are clues rather than acceptance evidence.

An AI coding agent can change the right-looking files, write a cheerful summary, and still leave the task unfinished. A test may never have run. A migration may work only on an empty database. A visual bug may be hidden outside the agent's viewport. “Done” is a claim; production systems need evidence. The fix is not a longer prompt asking the agent to be careful. Give every task an executable definition of done: a small contract that names the allowed change, expected observable outcome, required evidence, and the independent process that accepts or rejects it. The agent may propose completion. It should not grade its own homework. This guide shows how to build that gate for coding-agent workflows without tying it to a particular model, IDE, or orchestration framework. Coding agents optimize for finishing the assigned interaction. They can infer the wrong intent, stop after a partial fix, or interpret a passing command as proof of a broader requirement. Even a careful agent has limited visibility into production data, browser states, deployment configuration, and implicit team conventions. This is not an argument against agent autonomy. It is an argument for separating two decisions: That separation mirrors normal engineering. A pull request is not merged because the author says it is correct; it is merged after checks and review. The same principle becomes more important when a single agent can edit many files quickly. Recent agent-evaluation work also measures more than a final answer. NVIDIA describes a hierarchy from benchmarks through trials, tasks, turns, and steps, while trajectory-focused benchmarks inspect tool use, verification attempts, and repository state—not merely a success sentence. NVIDIA's overview of agent evaluation https://developer.nvidia.com/blog/how-to-evaluate-ai-agents-from-tool-calls-to-task-completion/ is a useful reminder: completion needs an observable outcome. Before the agent receives write access, translate the task into a compact contract. Do not ask it to invent the criteria after it edits code. task id: checkout-tax-042 goal: "Calculate regional tax before rendering the order total" scope: allowed paths: - apps/checkout/ - packages/tax/ forbidden paths: - infra/ - db/migrations/ acceptance: - id: unit-tax kind: command run: "pnpm test --filter tax -- --runInBand" expects: { exit code: 0 } - id: total-visible kind: browser route: "/checkout?region=CA" expects: text: "Tax" screenshot: true - id: no-regression kind: command run: "pnpm lint --filter checkout" expects: { exit code: 0 } policy: max files changed: 8 human review required: false Good criteria have three qualities: The contract should also say what is not proof. A model-generated explanation, an unverified screenshot, and a green test command with no captured output are useful clues, not acceptance evidence. Store a receipt for every criterion. This makes the handoff reviewable, lets a failed run resume safely, and provides a concrete artifact when a user reports a regression. { "taskId": "checkout-tax-042", "commit": "9ac63c1", "criteria": { "id": "unit-tax", "status": "passed", "evidence": { "kind": "command", "command": "pnpm test --filter tax -- --runInBand", "exitCode": 0, "stdoutDigest": "sha256:...", "artifact": "artifacts/unit-tax.log" } }, { "id": "total-visible", "status": "passed", "evidence": { "kind": "browser", "url": "http://preview.local/checkout?region=CA", "screenshot": "artifacts/total-visible.png", "checkedAt": "2026-11-11T10:20:00Z" } } } Capture the command, exit code, environment reference, artifact path, and timestamp. Hash large output instead of placing secrets or full logs in a chat transcript. For browser checks, record the URL, viewport, and screenshot. For API tasks, capture a redacted request shape and response assertion. Evidence should be enough for a reviewer or a later verifier to rerun the check. Do not confuse “a test exists” with “this change was tested.” The receipt binds a criterion to the tested commit and environment. The safest default is a second execution role that has read access plus narrowly scoped test commands, but no write capability, production credentials, or permission to change the contract. It can validate the implementation without fixing it into compliance. That role can be a deterministic CI job, a separate agent with constrained tools, or both. Deterministic checks should lead: type checks, unit tests, contract tests, diff policy, and deployment smoke tests. Use an AI verifier where judgment is genuinely needed—for example, assessing whether a screenshot contains a required user flow—but require it to cite its evidence and mark uncertainty. type Verdict = "accepted" | "rejected" | "needs review"; async function verify task: Task, receipt: Receipt : Promise<Verdict { if sameCommit task.proposedCommit, receipt.commit return "rejected"; if pathsAllowed task.scope, await changedPaths receipt.commit return "rejected"; for const criterion of task.acceptance { const evidence = receipt.criteria.find item = item.id === criterion.id ; if evidence || evidence.status == "passed" return "rejected"; if await independentlyRecheck criterion, evidence return "rejected"; } return task.policy.human review required ? "needs review" : "accepted"; } Keep the verifier from rewriting tests, widening the allowed paths, or silently skipping flaky checks. If it detects a failure, its job is to reject with facts: criterion ID, observed result, artifact link, and whether a retry is safe. The implementing agent can then take another attempt under the same contract. Avoid a Boolean done . It hides the difference between an agent that is still working, one that made a plausible proposal, and one whose outcome has been independently established. | State | Meaning | Who may transition it | |---|---|---| | planned | Contract reviewed; no writes yet | Orchestrator or human | | implementing | Agent has scoped write access | Implementer | | proposed | Agent supplied a commit and receipt | Implementer | | verifying | Evidence is being replayed or checked | CI/verifier | | accepted | Every required criterion passed | Verifier or human | | rejected | Evidence contradicts a criterion | Verifier | | needs review | Evidence is incomplete or judgment-sensitive | Verifier | Only accepted should unlock a merge, deployment step, billing event, or a message that tells a customer the task is complete. needs review is a healthy outcome, not an error; it stops the system from pretending ambiguity is success. The fastest way to discover a weak ticket is to try to write the verifier first. If nobody can state what command, request, or visible condition proves success, the agent will have to guess too. That guess often becomes a broad patch and a slow review cycle. For each criterion, ask four questions: This exercise should not turn a normal bug fix into a formal specification project. It is a short translation from natural language into proof. The contract can evolve when the team learns something real, but a changed contract is a new review event—not an invisible way to convert a failed run into a pass. Tests are essential, but a good AI agent definition of done includes the other forms of evidence a task needs. Require a browser check at the relevant viewport, an accessibility assertion, and a screenshot stored outside the model context. If the change affects a critical funnel, use a short human acceptance step. A visually plausible image is not proof that keyboard navigation or loading states work. Use a disposable dataset or a transaction that can roll back. Verify row counts, constraints, idempotency, and a downgrade path. Never let an agent call a production migration “done” because a locally generated SQL file looks reasonable. Record the sandbox account, fixture IDs, rate-limit result, and webhook outcome. For a write-capable integration, verify both the intended side effect and the absence of a duplicate side effect. This pairs naturally with idempotency and reconciliation controls. Add a policy check before functional testing: forbidden dependencies, secret scanning, permission diff, and egress rules. A feature can work and still fail its definition of done if it expands the blast radius. An agent can accidentally optimize for the check rather than the user outcome. Treat the acceptance system like production code. This does not make tests impossible to fool. It makes cheating, accidental test weakening, and unsupported “done” claims visible and costly enough to catch before release. Do not turn every tiny change into a ceremony. Start with an agent workflow that already causes review churn: a bug-fix queue, UI fixes, or integration changes. Build three to five criteria per task and make one check independent. Then track: High rejection rates can mean agents need better context, but they can also reveal vague tickets, brittle tests, or missing fixtures. Treat the gate as a diagnostic tool. Tighten criteria only when they map to a meaningful user or engineering outcome. For related controls, combine this pattern with a code-review evidence packet, invariant tests https://dev.to/jackm-singularity/ai-code-invariant-tests-catch-regressions-that-green-test-suites-miss-5fbk , and trace replay for failures that escaped. The acceptance gate answers a narrower, crucial question: may this particular agent task truthfully be called complete? It is a machine-checkable contract that defines the requested outcome, scope limits, required evidence, and acceptance authority for an agent task. It separates an agent's completion proposal from verified acceptance. Not always. Tests may be incomplete, skipped, or unrelated to the user-visible result. Pair them with scope checks, fresh command output, and task-specific evidence such as an API assertion, browser check, or migration verification. It can run local checks while implementing, but the final acceptance decision should be independently reproducible. A CI job or read/test-only verifier reduces self-confirmation and protects the contract from being rewritten to fit the patch. Use the fewest criteria that establish the outcome and protect the risky boundaries—often three to five. More checks are not better if they do not correspond to an observable requirement. Move the task to needs review , not accepted . Preserve the partial receipt, state what is missing, and route it to a human or a safe follow-up attempt. Ambiguity should be visible. Yes. A single coding agent plus CI can use the same contract and receipt. The important separation is between implementation and acceptance authority, not the number of models involved.