Can You Trust Code You Didn't Write?

Developer Tony describes a verification workflow for code produced by AI coding agents such as Claude Code, arguing that an agent reporting "implementation complete" or "all tests passed" does not establish that the requested behavior actually works. His approach pairs a pre-implementation acceptance contract of observable behaviors with focused authorization tests and a recorded evidence trail, illustrated on a multi-tenant TypeScript API with a group-based permission bug.

I'm Tony. I build and run web products, and Claude Code is my main development tool. A coding agent finishes implementing a feature, reports that everything is complete, and presents a summary of the changes. The code looks reasonable, the tests appear green, and you have another task waiting. But what exactly did the agent prove? Generating working code and verifying correct behavior are different engineering activities. I use a verification workflow that starts with observable acceptance criteria, checks the implementation at several levels, and records evidence before accepting a change. I'll walk through it using a TypeScript application with an admin interface and a multi-tenant API, including an authorization bug that ordinary tests can miss. You'll leave with a reusable verification checklist and a completed example showing what an acceptable evidence trail looks like. A coding agent can produce a convincing implementation without establishing that it solves the original problem. It might modify the right files, follow the existing patterns, and run the available tests successfully. None of those results automatically establishes that the requested behavior works. They answer narrower questions. Consider four statements an agent might make: | Statement | What it actually establishes | |---|---| | "Implementation complete" | The agent produced code it considers finished. | | "TypeScript compilation passed" | The compiler found no errors covered by that check. | | "All tests passed" | The executed tests passed under their configured conditions. | | "The requested behavior works" | Requires evidence that the implementation satisfies the acceptance criteria. | The difference matters because tests only exercise the scenarios someone anticipated. A passing authorization test might establish that a resource owner can access a document, while saying nothing about another authorized member or an unrelated tenant. A successful browser interaction might show text appearing in an editor without proving that the correct value was saved. Even a valid download can contain files with incorrect permissions. I think of this as a verification gap. The agent's final message belongs on one side; evidence that supports the product requirements belongs on the other. My responsibility is to close that gap, not merely decide whether the generated code looks plausible. The same distinction applies to human-written code, but agents change the amount of implementation a developer can produce without personally tracing every decision. That makes a disciplined acceptance process more important. Before implementation, I want a short contract describing what the system must do and how we will know it works. This is not a complete specification or an elaborate test plan. It is a set of observable behaviors that can be checked independently of the agent's explanation. Imagine a pnpm TypeScript monorepo with the following structure: apps/ admin/ React admin application api/ Multi-tenant API packages/ ui/ Shared UI components shared/ Shared contracts and utilities Suppose the API serves multiple customer organizations. A resource can be associated with several groups, and users receive access through group membership. The requested change is to correct a permission check that incorrectly rejects some authorized users. Here is the acceptance contract I would give the agent: Task: Correct group-based access to shared resources. Expected behavior: 1. A member of any group attached to a resource can access that resource within the same tenant. 2. A user outside all attached groups is denied. 3. A user from another tenant is denied, even when group identifiers overlap. 4. Existing resource-owner access remains unchanged. Evidence required: - Focused authorization tests for all four cases. - A regression test where the authorized user belongs to the second attached group. - Confirmation that the regression test fails without the fix and passes with it. - A summary of changed files and remaining risks. Constraints: - Preserve tenant isolation. - Do not change unrelated authorization behavior. - Do not modify tests merely to make them pass. This contract specifies results rather than implementation details. I have not instructed the agent to use a particular SQL query, ORM method, or function structure. It can choose an approach that fits the existing code, but it cannot redefine successful completion as "the code compiles." The edge cases carry much of the value. Testing an owner and an unauthorized stranger is insufficient when access depends on several relationships. The important case is a user who belongs to a valid group that the previous implementation failed to consider. For a bug fix, I also provide the reproduction command, relevant stack trace when one exists, and steps needed to reproduce the behavior. I state whether the failure is intermittent. These details reduce guesswork and give the agent a concrete starting point. The principle is simple: define success before evaluating the implementation. Otherwise, the agent's solution can influence what everyone considers an acceptable test. I separate implementation instructions from acceptance decisions. The agent can inspect the repository, propose a solution, edit code, and execute tests. It can also identify missing cases and explain technical tradeoffs. But I do not treat its final assessment as independent evidence. For changes with unclear scope, I start Claude Code in plan mode: claude --permission-mode plan Plan mode lets Claude read the project and propose an approach without editing files until approval. I use that boundary to examine assumptions, identify impacted modules, and check whether the proposed tests cover the acceptance contract. The plan is useful because it exposes decisions before implementation makes those decisions harder to notice. For the authorization example, I might use this prompt: Inspect the shared-resource authorization flow. Identify: - How resources are associated with groups. - Where tenant isolation is enforced. - Which tests cover users with multiple group paths. - Why membership in the second attached group fails. Propose the smallest safe change and the tests needed to verify it. Do not edit files yet. After reviewing the proposal, I authorize implementation and ask for a report containing the actual commands and outcomes. I also ask the agent to identify missed edge cases. This makes the agent useful as an implementation partner without making it the final authority on correctness. An instruction in CLAUDE.md can require the agent to include test evidence in its response. That is a useful working convention, not an enforcement mechanism. The presence of a well-formatted report does not prove that the underlying evidence is valid. If the change moves in the wrong direction, Claude Code's /rewind can restore the conversation and code to a checkpoint. That helps manage the implementation process, but it does not replace version control or acceptance testing. Verification works better when each check has a specific responsibility. Compilation, automated tests, browser interactions, and human review observe different properties of the system. Running more checks is not automatically better; choosing checks that cover different failure modes is what matters. I start with the checks already defined by the repository. For a TypeScript project, these typically include type checking, linting, and build validation. They reveal broken imports, incompatible types, invalid code patterns, and some integration mistakes. For example: pnpm -F api exec tsc --noEmit This command only makes sense when the workspace and TypeScript configuration support it. A successful result tells me that the relevant TypeScript checks passed. It does not establish that group membership is evaluated correctly or that the API returns the expected authorization decisions. I record which command ran, its exit status, and whether the check covers the changed code. "Static checks passed" is less informative than the exact command and result. Consider this incorrect permission check: js const group = await db.resourceGroup.findFirst { where: { resourceId, }, } ; const allowed = group ? await isGroupMember userId, group.groupId : false; The implementation checks membership against the first group returned by the database. If a resource belongs to groups A and B, a user belonging only to B can be denied access. The code compiles, and a test using a member of A might pass. The correction should ask whether there is at least one qualifying membership, not whether the user belongs to one arbitrarily selected group. One possible approach is a relational query that keeps the tenant condition within the authorization predicate: js const allowed = await db.resourceGroup.count { where: { resourceId, resource: { tenantId, }, group: { tenantId, memberships: { some: { userId, }, }, }, }, } 0; The exact query depends on the application's data model and ORM schema. Its essential property is that it evaluates all eligible relationships while preserving tenant boundaries. The surrounding authorization flow must still validate the user's tenant context and any additional access rules. The regression test is more valuable than a generic "authorized user succeeds" test: js it "allows a member of the second attached group", async = { const resource = await createResource { tenantId: tenantA.id, groupIds: groupA.id, groupB.id , } ; const user = await createUser { tenantId: tenantA.id, groupIds: groupB.id , } ; const response = await requestResource { resourceId: resource.id, asUser: user, } ; expect response.status .toBe 200 ; } ; Here, createResource , createUser , and requestResource represent application-specific test helpers. A real implementation should use the repository's existing fixtures and request utilities, with deterministic setup that makes the relevant group ordering explicit. A passing test is useful. A test that fails against the broken implementation and passes against the correction provides stronger evidence that it detects the intended defect. One way to perform this check in a controlled working tree is: git stash push src/auth/permissions.ts npm test -- permissions git stash pop This sequence assumes the working-tree state, stash contents, and test command have been inspected first. The stash must actually restore the known broken implementation for the test to mean anything. It can also stash other uncommitted edits in the named file, so I would use a clean, isolated working tree rather than run it casually against active work. Notice another operational detail: combining the test and restore operations with ; allows the restore attempt to run even if the test fails. With three separate commands, I must explicitly execute the restoration after the expected failure and inspect the resulting working-tree state. The evidence I want is specific: the regression test fails for the expected authorization reason without the fix, then passes with the correction restored. A failure caused by missing fixtures, invalid imports, or environment setup is not proof that the regression test detects the original bug. The repository might use pnpm instead: pnpm -F api test -- permissions I use the command that matches the actual project rather than changing package managers simply to match an example. API tests do not establish that the browser handles state correctly. A common case is automation entering text into a rich-text editor built with DraftJS-style or React-managed state. The text becomes visible, but the application saves a default value because the submitted content lives in parent component state. A screenshot of the editor would look correct while the saved record would be wrong. The appropriate fix is to update the value through the component's supported onChange mechanism, then save and read back the persisted content. The verification target is the reopened or fetched record, not merely the visible text before submission. For the authorization example, a browser check can establish that an authorized member can open the shared resource and an unauthorized member receives the intended denial. That user-facing result complements the API tests without replacing them. I cover browser automation patterns in more detail in the article on browser testing with Playwright in this series. Automated checks can establish consistency with written expectations. They cannot decide whether those expectations fully reflect the product's intended policy. A developer still needs to evaluate whether the change introduces unintended access, violates an architectural boundary, or relies on an assumption the tests never challenge. For authorization code, I examine query scoping, user identity, tenant ownership, and any caching that might retain stale permissions. I also look for inconsistent behavior between endpoints using the same permission model. Those concerns are part of engineering review, not defects a green test suite automatically rules out. After executing the checks, I return to the original acceptance contract and inspect the actual diff. The question is not simply whether I understand the implementation. I want to know whether every change serves the requested behavior. For the authorization fix, the intended scope is narrow: correct membership evaluation and add meaningful coverage. A change that also modifies authentication middleware, restructures tenant resolution, or removes failing assertions deserves separate justification. Scope expansion increases the number of assumptions I need to verify. My diff review focuses on five questions: Another useful example is generating a downloadable ZIP package. The download can succeed, contain the correct files, use the expected version, and pass a secrets check. Yet its setup script might lose the executable bit because the ZIP library does not preserve Unix file modes. After extraction, this result is wrong: -rw-r--r-- scripts/setup.sh Running the script directly produces: ./scripts/setup.sh: Permission denied For a package where scripts are expected to run directly on Unix, a packaging rule can assign mode 0755 to files beginning with and 0644 to ordinary files. Verification should then inspect the extracted archive, not just the source directory. The expected extracted permission is: -rwxr-xr-x scripts/setup.sh This example demonstrates why a passing pipeline can still miss an important delivery requirement. The implementation artifact must be checked in the form and environment where someone actually uses it. I prefer a compact verification report over a long transcript of everything the agent attempted. The report should let another developer understand the requested behavior, identify the executed checks, reproduce important results, and see what remains uncertain. It should also distinguish the agent's statements from recorded observations. The following is a completed example for our fictional authorization fix. The paths and outcomes illustrate what a report should contain; they are not claims that these commands were executed against a real repository. agent-change-verification.md Agent Change Verification Task Correct group-based access to shared resources. Acceptance criteria - x Member of any attached group is allowed. - x Nonmember is denied. - x User from another tenant is denied. - x Existing owner access remains unchanged. Files changed - src/auth/permissions.ts Replaced first-group lookup with membership query. - src/auth/permissions.test.ts Added second-group and tenant-isolation cases. Checks executed | Check | Result | Evidence | |---|---|---| | TypeScript check | PASS | typecheck.log | | API permission tests | PASS | permissions-test.log | | Regression without fix | EXPECTED FAIL | regression-red.log | | Regression with fix | PASS | regression-green.log | | Focused diff review | PASS | review-notes.md | Expected vs observed Expected: A user in group B can access a resource shared with groups A and B within the same tenant. Observed: The API returns 200 for the group B member. The regression test fails with the original first-group logic and passes with the fix. Unverified scenarios - Concurrent group membership changes. - Cache invalidation after permission updates. - Browser-level permission error presentation. Known limitations - Tests use a local test database. - Production data distribution was not reproduced. - No performance evaluation of the new query. Human review decision APPROVED FOR REVIEW, NOT DEPLOYMENT. Reason: Acceptance tests pass and regression behavior is demonstrated. Query execution characteristics and cache behavior require separate review. The report does not need to claim absolute confidence. Its value comes from making the evidence inspectable and showing where the evidence stops. In a real repository, the referenced logs and review notes should exist, correspond to the tested revision, and be accessible to the reviewer. I find it useful to rank evidence by what it establishes: This is an ordering of evidence strength, not a replacement for evaluating test quality or requirement coverage. A read-back confirms a particular observed result; it does not establish that every untested condition is safe. For write operations, I like tools that return both verified and evidence . The first indicates whether post-action verification succeeded. The second explains what the system returned when the tool read the result back. { "operation": "deleteResource", "verified": true, "evidence": { "method": "GET", "path": "/resources/example-id", "status": 404 } } In this example, the write tool deleted a resource and then performed a separate read. A successful delete request alone would not provide equivalent evidence. The observed 404 supports the claim that the resource is no longer retrievable at that endpoint, assuming the read uses the appropriate identity and tenant context. For pull requests, I can use gh pr create to connect a Claude Code session with a PR and reopen that session using claude --from-pr <number . The PR then becomes a practical place to review the diff, attach test evidence, and ask Claude to flag risks. The review decision still belongs to the people responsible for accepting the change. A verification checklist helps make acceptance consistent, but it cannot compensate for missing requirements or an environment that fails to represent production behavior. Authorization, billing, data migration, destructive operations, and sensitive-data handling deserve closer attention because incorrect behavior can have consequences beyond a single feature. Weak test coverage also changes how much confidence I assign to a passing suite. If the application has no realistic integration environment, no tenant-isolation fixtures, or no way to inspect persisted changes, the verification report should say so. The correct response is to identify the missing evidence, not lower the acceptance criteria until everything appears green. I also separate new failures from existing failures. If an unrelated test was already failing, I do not edit it, skip it, or change its expectations just to produce a green result. I document the failure in the PR, explain why it appears unrelated, and leave the decision visible to reviewers. Some decisions remain fundamentally human. Whether two groups should share access to a sensitive resource is a product and security policy question before it is a query implementation question. An agent can implement the stated rule and test that implementation, but the team must decide whether the rule is correct. The workflow I use is therefore not "make the agent pass every check." It is to define observable expectations, collect independent evidence, review the change against its intent, and explicitly record what has not been established. That process makes code written by an agent easier to understand, verify, and maintain. This is the foundation of Making Coding Agents Reliable , a six-article series about practical workflows for working with AI coding agents. The next article covers CLAUDE.md and Skills, including how to make project instructions useful without confusing instructions with enforcement. What's the one check you never skip before accepting an AI-generated change?