An agent writes the function, writes the covering test in the same pass, CI turns green, and the reviewer merges on a clean stack. Months later an edge case breaks in production, you open the test that was supposed to cover it, and it stays green. It stayed green because it never knew the requirement; it copied the implementation's own assumption back at it.
A real test derives the expected value from somewhere the code does not: a written rule, a known worked example, an invariant from the domain, or a bug report that already happened. A mirror test derives the expected value by running the new function, reading its output, and hardcoding that output back into the assertion. The test then verifies that the function returns what the function returns. When the implementation changes its own behavior, the agent tends to update the expected value in the same pass, and the coupling closes permanently.
The shallow version is an assertion that only checks the response is non-null, or that the call does not throw. It proves the function ran, nothing about whether the answer is correct.
When a human writes a test after thinking about the requirement, the test writer can disagree with the implementation and catches a wrong branch. When the same model generates code and test from one prompt, both inherit the same unstated assumption: how empty input behaves, what counts as a valid identifier, which timezone the date means, whether an out-of-range value should clamp or reject. The model has no independent oracle, so the test cannot contradict the code it just wrote. Green in that case is a syntax and plumbing check, not evidence of correctness.
On functions that sit on a trust boundary, flip one boundary condition in the implementation deliberately: clamp instead of reject, drop the empty-input guard, shift a rounding rule. Run the suite. If every test still passes, the test is not touching that branch and it provides no safety net. This is a lightweight version of mutation testing; you do not need a full framework for a handful of risky functions, just one local flip per probe and a clean revert after. A test that survives a mutation it should kill is advertising coverage it does not have.
When a bug does arrive, write the failing characterization test for the exact reported input first, before touching the fix. That test carries the behavior from outside the codebase, so it cannot mirror the repaired implementation. It also becomes the permanent guard that prevents a future agent from smoothly reverting the fix while keeping the suite green.
Tests generated in the same pass as the code are a smoke check. They confirm the wiring runs; they do not confirm the answer is right. Separate the oracle from the implementation, and green starts to mean something again.