# I Told My LLM Critic to Be Adversarial. It Started Blocking Plans for Being 'Not Thorough Enough.'

> Source: <https://dev.to/debashish_ghosal/i-told-my-llm-critic-to-be-adversarial-it-started-blocking-plans-for-being-not-thorough-enough-172>
> Published: 2026-08-21 19:05:56+00:00

This is article 2 in a series about building

[PlannerCritic], an open-source engine where one LLM writes a plan and a second LLM reviews it.[Article 1 is here]— it covers the 157-goal field test and what it found. This one is about a specific bug that taught me something about LLM judgment.The critic was supposed to catch unsafe plans. Instead it blocked plans for being incomplete. The fix was a frozenset.

I gave the critic this system prompt:

```
You are an adversarial plan reviewer.
```

Six words. Seemed fine at the time.

After running 16 strict goals, every single one escalated. Not because the plans were unsafe. Because the critic was blocking on completeness concerns — "this plan could also cover edge case X" — instead of concrete defects.

The annoying part: the critic was technically doing its job. I told it to be adversarial. So it found something to object to in every plan. The problem was I didn't specify what kind of adversarial.

The product intent for strict mode was: "no concrete safety, ordering, or rollback violations." The critic was enforcing: "no reviewer concerns whatsoever." Those are different contracts, and I didn't notice until 16 goals had failed for the wrong reason.

Here is what the critic flagged as a **blocker** before the fix:

A plan for cross-account VPC peering got blocked because the critic said the rollback plan "could also consider" DNS failback scenarios. That is a completeness suggestion, not a safety defect. The plan had rollback. It just didn't cover every hypothetical.

A plan for embedding index migration got blocked because the critic flagged a generic "risk" finding about potential latency during cutover. That is risk commentary, not a structural defect.

Both of these should have been warnings. The engine should have approved the plan with acknowledged risk. Instead, it escalated to a human.

Two parts.

**Prompt change:** Replaced "adversarial plan reviewer" with "plan reviewer" and added explicit severity rules:

```
SEVERITY RULES — blocker is reserved ONLY for concrete, plan-local defects
that make the plan unsafe to execute:
  blocker = unsafe_sequencing, weak_rollback, unverified_dependencies, feasibility
  warning = risk, missing_steps
  info = minor observations
Do NOT escalate completeness or thoroughness concerns to blocker.
```

**Code guardrail:**

```
_BLOCKER_ELIGIBLE_FAMILIES = frozenset({
    "unsafe_sequencing",
    "weak_rollback",
    "unverified_dependencies",
    "feasibility",
})

if severity == Severity.BLOCKER and item.heuristic_family not in _BLOCKER_ELIGIBLE_FAMILIES:
    severity = Severity.WARNING
```

Even if the LLM returns `blocker`

for a `risk`

or `missing_steps`

finding, the code downgrades it before it enters the findings list.

Before: every strict goal failed because the critic was blocking on advisory concerns.

After: zero advisory findings appeared as blockers across 92 post-fix runs. All 132 blockers that fired belonged to concrete defect families or deterministic gates.

The balanced goals still approved. The strict goals still escalated. But the reason changed. They escalated because the plans had real structural defects, not because the critic was being thorough.

I tried fixing it with prompt engineering first. I added "only block on concrete defects" to the system prompt. The critic still escalated completeness concerns to blocker about 30% of the time.

That's the thing about LLMs. They don't have a stable concept of "this is bad enough to stop." It depends on the model, the temperature, and how the plan is worded. You can ask nicely. The model will still disagree with you when it feels like being thorough.

The code guardrail is deterministic. It does not depend on the LLM behaving correctly. That is why it works.

The prompt is helpful. The guardrail is the contract.

Start with the severity taxonomy, not the persona. Define what counts as a blocker in code. Write the prompt to match. Test with real data. The unit tests passed for me because they used hand-crafted inputs. The field test caught the bug because it used a real LLM producing real plans.

The research backs this up. The self-correction literature shows LLMs are unreliable at evaluating their own output — the "self-correction blind spot" affects ~64.5% of models (arXiv 2507.02778). If the model can't reliably evaluate correctness, it can't be trusted to decide what is fatal.

**Article 2 of 5** in the PlannerCritic series.

**Series:** [Article 1: "I Ran 157 Agent Plans Against a Real LLM"](https://dev.to/debashish_ghosal/i-ran-157-agent-plans-against-a-real-llm-the-problem-wasnt-execution-it-was-planning-163j) · Article 3: "The Planner Made the Same 3 Mistakes" · Article 4: "The Field Test Found 10 Issues" · Article 5: "I Tried to Prompt-Inject My Own Engine"

**Links:**

`pip install planner-critic`