{"slug": "what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself", "title": "What Do You Do While AI Codes? I Make Mine Argue With Itself.", "summary": "A developer built AdversarialDebate, an open-source multi-model code review system in which two LLMs analyze a pull request in parallel with no shared context, commit independent verdicts, and then debate under a structural constraint that prevents either model from anchoring on the other's output. Testing across 411 debates on 70 public-repo pull requests produced an 88.7% binary match against a corrected 2,333-row dataset at a total cost of $0.42 for 360 reviewer runs, and the developer reports the best pairing was GPT-4o-mini with Mistral Small 3.2, arguing that diversity of training objective predicts debate quality better than raw model capability.", "body_md": "Be honest: what do you actually do while the agent types?\n\nI used to just watch. Not read, *watch*. Scroll the diff as it streamed in, nod at code I hadn't fully parsed, and tell myself I'd review it properly at the end. Sometimes I did. Often I didn't. The question \"what do you do while AI codes?\" keeps showing up in the dev.to feed, and the comments make it clear I'm not alone. The struggle to stop mid-task and just wait is real.\n\nSo I tried to fill the wait with something useful. I built a system where a second model tries to *break* the first model's work. Then I learned the thing that changed how I think about AI review: **89% of what I built was theater.**\n\nI pointed two LLMs at the same pull request and asked them to argue. It worked beautifully: confident claims, point-by-point rebuttals, a clean verdict at the end.\n\nThen I read the raw logs.\n\nThe second model wasn't analyzing anything. It was replaying pre-generated text. Both models were exchanging messages, no positions were changing, no new evidence was being cited. The transcript *sounded* like a debate. It was a recording of a debate.\n\nIf I hadn't looked at the raw output, I would have shipped it and called it a working system. That's the part that still bothers me.\n\nHere's what most multi-model review workflows actually do:\n\nIt isn't. It's a *validation*. The moment Model B's context contains Model A's verdict, B is anchored. It's doing social-pressure resistance, not analysis. Humans fail this constantly. That's why double-blind review exists. We invented that safeguard decades ago and keep leaving it out of AI pipelines.\n\nThe fix is mechanical: **Model B cannot see Model A's output until B has fully committed its own position.** Not prompted into independence. Structurally prevented from anchoring.\n\nThat one constraint is the entire project: [AdversarialDebate](https://github.com/deghosal-2026/adversarial-debate).\n\nTwo models analyze in parallel with zero shared context. Each commits a verdict with structured claims and evidence. Then a bounded debate opens: every objection must reference a specific counter-claim, every claim must cite specific text from the artifact. When they can't resolve, the output is a **structured disagreement report**, both positions preserved and unresolved points documented.\n\nThat last part matters. Most multi-agent systems are engineered toward consensus. Disagreement is treated as a failure to be smoothed over. But if two independent reviewers reach different conclusions with different evidence, *that tension is the signal*. Collapsing it into one confident verdict throws away the exact thing you built two reviewers to find.\n\nTheater rate across 217 debates in v0.2.0: **0%.**\n\nI ran the first real field test on **70 actual pull requests from public repos**, 411 debates. The core question: do the claims correspond to what actually went wrong in those PRs?\n\nBy v0.2.0: 150 artifacts across four domains, **88.7% binary match** against the corrected 2,333-row dataset, near-zero fabrications. Total cost: **$0.42 for 360 reviewer runs.**\n\nLess than lunch. Compute was never the bottleneck. Prompt engineering and ground-truth measurement were.\n\nThe best pairing wasn't the two smartest models. It was **GPT-4o-mini + Mistral Small 3.2**, a small OpenAI model and a small European one.\n\nGPT + Gemini produced the *worst* debates: lots of rounds, almost no concessions, no resolution. Two RLHF-heavy models from big US labs agree fast because they were trained to prefer harmony. GPT-mini + Mistral produced genuine disputes, real concessions, and eventual convergence.\n\nMy working theory: **it's not raw capability that predicts debate quality, it's diversity of training objective.** A later run refined this: the effect may be as much about non-Mistral models rubber-stamping each other, so the practical rule became \"always include Mistral.\" I hold the theory loosely.\n\nI have not proven this rigorously. It matches everything in my data, and it's why my model-selection advice starts with \"pick from different labs\" before anything about benchmark scores.\n\n88.7% sounds good. It is good, compared to single-pass review. But there's a problem:\n\n**False negatives are invisible.**\n\nIf both models miss a real issue and converge on \"looks fine,\" you get a clean verdict and no indication anything is wrong until it surfaces downstream. The 11.3% that's partial or wrong clusters in the narrative domains: incident reports, change proposals, where ground truth is fuzzier. Generic prompts that work on code don't port cleanly to a postmortem.\n\nRequiring two independent models to *both* miss an issue is a much higher bar than single-pass review. It is not a guarantee, and I won't pretend it is.\n\nWhat do I do while AI codes? I don't watch anymore. I let a second model try to tear the first one's work apart while I do something that actually needs a human: reading the failure modes, writing the eval, deciding what \"good\" means.\n\nIt's not passive. It's a different kind of attention. And honestly, it's less lonely than watching a cursor.\n\n**So what do you actually do while the model types?** Do you review live, review after, or have you found a way to use the wait?", "url": "https://wpnews.pro/news/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself", "canonical_source": "https://dev.to/debashish_ghosal/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself-2gl7", "published_at": "2026-09-19 06:31:24+00:00", "updated_at": "2026-09-19 06:54:18.615465+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "developer-tools", "ai-research"], "entities": ["AdversarialDebate", "GPT-4o-mini", "Mistral Small 3.2", "OpenAI", "Mistral", "Gemini", "GitHub", "dev.to"], "alternates": {"html": "https://wpnews.pro/news/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself", "markdown": "https://wpnews.pro/news/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself.md", "text": "https://wpnews.pro/news/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself.txt", "jsonld": "https://wpnews.pro/news/what-do-you-do-while-ai-codes-i-make-mine-argue-with-itself.jsonld"}}