herdr
to orchestrate Claude3.5 Sonnet and GPT-4o (simulating a high-reasoning setup), where one model proposes the fix and the other acts as the "destructive" reviewer trying to break the logic.
The Logic Behind Adversarial Review #
The problem with using a single AI for code review is the "confirmation bias" loop. If Claude writes the code and Claude reviews it, it often misses its own logical fallacies. By pitting Claude's architectural strengths against GPT-4o's strict adherence to constraints, you create a synthetic tension that mimics a real senior dev review.
In my tests, this "adversarial" approach caught three race conditions in a Go concurrency module that a single-model pass completely ignored.
Deployment Setup #
To get this running, you need a CLI orchestrator like herdr
to pipe the output of one model into the prompt of another. Here is the basic configuration I use to trigger the adversarial loop.
First, I set up the "Proposer" (Claude 3.5 Sonnet) to handle the initial implementation. Then, I pipe that output into the "Critic" (GPT-4o) with a very specific system prompt designed to be pedantic.
herdr run "Implement the cache invalidation logic in src/cache.ts" --model claude-3-5-sonnet > proposal.txt
herdr run "Find every possible way this code fails, specifically looking for memory leaks and race conditions: $(cat proposal.txt)" --model gpt-4o > critique.txt
herdr run "Fix the following issues identified by the reviewer: $(cat critique.txt) based on original code: $(cat proposal.txt)" --model claude-3-5-sonnet > final_version.txt
Performance Benchmarks #
I ran this setup against a set of 10 complex TypeScript functions with intentional bugs (off-by-one errors, null pointers, and async leaks).
Single Model (Claude 3.5): 60% bug detection rate.Single Model (GPT-4o): 50% bug detection rate.Adversarial Loop (Claude -> GPT -> Claude): 90% bug detection rate.
The jump in quality comes from the Critic model being told its only job is to find faults, not to be "helpful" or "encouraging."
Prompt Engineering for the Critic #
The secret is the system prompt for the second model. If you use a generic "review this code" prompt, the AI is too nice. I use a "Destructive Reviewer" persona.
System Prompt for Critic:
You are a cynical, world-class Staff Engineer. Your goal is to reject the PR.
Do not praise the code. Only list:
1. Critical bugs (Logic errors, crashes)
2. Performance bottlenecks (O(n^2) traps)
3. Edge cases the author missed.
If the code is perfect, you failed. Find something wrong.
The "Fix" Cycle #
When the Critic finds a bug, the Proposer sometimes hallucinates a fix that introduces a new bug. To prevent this, I've implemented a validation step. I force the Proposer to write a test case that fails on the current code but passes on the proposed fix.
Example Bug Fix Loop:
Critic:"TheuseEffect
inUserPanel.tsx
lacks a dependency array, causing an infinite re-render loop."Proposer Fix: Adds[]
to the dependency array.Verification: I run thenpm test
command via the CLI. If the test fails, the loop restarts.
This AI workflow turns code review from a "vibes-based" check into a rigorous verification process. It's slower than a single prompt, but it's the only way to get production-ready code from an LLM agent without spending two hours manually debugging the output.
Next LLM Drift Tracking: Why My Alerts Were All Wrong →