The most dangerous AI-generated diffs aren't the ones that throw errors; they are the ones that pass every test and look polished, but would lose a customer money the first time they hit production. The problem isn't a lack of intelligence, but an abundance of confidence. In a standard dev loop, we ask for a feature, the AI writes it, the tests go green, and we ship. We are essentially treating correctness as a coincidence that happens to ride along with the AI's confidence.
I spent 30 days letting AI write 100% of my application logic—zero manual typing—to figure out what actually prevents these silent failures. It wasn't about finding a "better" model or a magic prompt. The solution was a structural rule.
Why asking an AI to review its own code is a waste #
Most people try to safeguard their code by asking the AI to check its work with a prompt like "Review this code and tell me if it's correct." This is almost useless because you're asking a model to agree with itself. An LLM is exceptionally good at confirming its own logic; it basically grades its own homework and gives itself an A.
The core issue is that "confirming" and "breaking" are two different cognitive tasks. If you ask for confirmation, that is exactly what the model will deliver. To get a real result, you have to stop asking for a blessing and start asking for a failure.
Implementing the Refutation Gate #
The rule is simple: nothing gets merged until a second reader—whose sole purpose is to break the code—has tried and failed to do so. For this to work, you need three specific components.
1. Isolate the reviewer from the author
The model that wrote the code cannot be the one that approves it. Even if you use the same model, you must use a separate, clean context. The author's context is already "poisoned" because it spent thousands of tokens convincing itself the solution was correct. A fresh session with no memory of the writing process is the only way to get an independent perspective.
Ideally, use a different model family entirely. Different models have different training distributions and blind spots. If two models come from the same family, they often share the same misconceptions about what "clean" code looks like, meaning they will both miss the same landmines.
2. Use a break-it brief instead of a bless-it brief
Stop asking if the code "looks correct." Give the reviewer a job that can only be completed by finding a flaw.
The wrong way:
Review this diff and tell me if it looks correct.
The right way:
This code is broken. I know it is — I just don't know how yet.
Your job is to produce the specific input, sequence, or state that
makes it fail. Assume:
- the network drops a packet at the worst possible moment
- two of these run at the same time
- the database write fails AFTER the external call succeeds
- the user does the thing no sane user would do
Give me the exact scenario that loses data or loses a customer money.
If you truly cannot find one, say so explicitly and explain what
would have to be true for that to be the case.
By framing the prompt this way, you shift the model's goal from "seeking agreement" to "hunting failures." You are forcing it to simulate edge cases—like race conditions or partial database failures—rather than just glancing at the syntax.
Next Miso lets me run local music models without the CLI headache →
All Replies （3） #
Finally! I'm tired of flaky tests. Does this work with Pytest-mock or is there a better library for this?
I want to try this tonight. Does this logic actually work with Pydantic v2 or will it crash the gate?
I want to try this tonight. I'm using LangGraph to handle the state, but the loops keep timing out at 40 seconds.