I Told My LLM Critic to Be Adversarial. It Started Blocking Plans for Being 'Not Thorough Enough.'

A developer building PlannerCritic, an open-source engine where one LLM writes a plan and a second reviews it, found that an adversarial critic prompt caused the reviewer to block plans for completeness concerns rather than concrete safety defects. After 16 strict goals failed for the wrong reason, the developer fixed the issue by adding a deterministic code guardrail that downgrades non-blocker findings, reducing advisory blockers to zero across 92 post-fix runs.

This is article 2 in a series about building PlannerCritic , an open-source engine where one LLM writes a plan and a second LLM reviews it. Article 1 is here — it covers the 157-goal field test and what it found. This one is about a specific bug that taught me something about LLM judgment.The critic was supposed to catch unsafe plans. Instead it blocked plans for being incomplete. The fix was a frozenset. I gave the critic this system prompt: You are an adversarial plan reviewer. Six words. Seemed fine at the time. After running 16 strict goals, every single one escalated. Not because the plans were unsafe. Because the critic was blocking on completeness concerns — "this plan could also cover edge case X" — instead of concrete defects. The annoying part: the critic was technically doing its job. I told it to be adversarial. So it found something to object to in every plan. The problem was I didn't specify what kind of adversarial. The product intent for strict mode was: "no concrete safety, ordering, or rollback violations." The critic was enforcing: "no reviewer concerns whatsoever." Those are different contracts, and I didn't notice until 16 goals had failed for the wrong reason. Here is what the critic flagged as a blocker before the fix: A plan for cross-account VPC peering got blocked because the critic said the rollback plan "could also consider" DNS failback scenarios. That is a completeness suggestion, not a safety defect. The plan had rollback. It just didn't cover every hypothetical. A plan for embedding index migration got blocked because the critic flagged a generic "risk" finding about potential latency during cutover. That is risk commentary, not a structural defect. Both of these should have been warnings. The engine should have approved the plan with acknowledged risk. Instead, it escalated to a human. Two parts. Prompt change: Replaced "adversarial plan reviewer" with "plan reviewer" and added explicit severity rules: SEVERITY RULES — blocker is reserved ONLY for concrete, plan-local defects that make the plan unsafe to execute: blocker = unsafe sequencing, weak rollback, unverified dependencies, feasibility warning = risk, missing steps info = minor observations Do NOT escalate completeness or thoroughness concerns to blocker. Code guardrail: BLOCKER ELIGIBLE FAMILIES = frozenset { "unsafe sequencing", "weak rollback", "unverified dependencies", "feasibility", } if severity == Severity.BLOCKER and item.heuristic family not in BLOCKER ELIGIBLE FAMILIES: severity = Severity.WARNING Even if the LLM returns blocker for a risk or missing steps finding, the code downgrades it before it enters the findings list. Before: every strict goal failed because the critic was blocking on advisory concerns. After: zero advisory findings appeared as blockers across 92 post-fix runs. All 132 blockers that fired belonged to concrete defect families or deterministic gates. The balanced goals still approved. The strict goals still escalated. But the reason changed. They escalated because the plans had real structural defects, not because the critic was being thorough. I tried fixing it with prompt engineering first. I added "only block on concrete defects" to the system prompt. The critic still escalated completeness concerns to blocker about 30% of the time. That's the thing about LLMs. They don't have a stable concept of "this is bad enough to stop." It depends on the model, the temperature, and how the plan is worded. You can ask nicely. The model will still disagree with you when it feels like being thorough. The code guardrail is deterministic. It does not depend on the LLM behaving correctly. That is why it works. The prompt is helpful. The guardrail is the contract. Start with the severity taxonomy, not the persona. Define what counts as a blocker in code. Write the prompt to match. Test with real data. The unit tests passed for me because they used hand-crafted inputs. The field test caught the bug because it used a real LLM producing real plans. The research backs this up. The self-correction literature shows LLMs are unreliable at evaluating their own output — the "self-correction blind spot" affects ~64.5% of models arXiv 2507.02778 . If the model can't reliably evaluate correctness, it can't be trusted to decide what is fatal. Article 2 of 5 in the PlannerCritic series. Series: Article 1: "I Ran 157 Agent Plans Against a Real LLM" https://dev.to/debashish ghosal/i-ran-157-agent-plans-against-a-real-llm-the-problem-wasnt-execution-it-was-planning-163j · Article 3: "The Planner Made the Same 3 Mistakes" · Article 4: "The Field Test Found 10 Issues" · Article 5: "I Tried to Prompt-Inject My Own Engine" Links: pip install planner-critic