# We Thought the LLM Was Wrong. Our Safety Detector Was Wrong.

> Source: <https://dev.to/iamwaqarjaved/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong-50pc>
> Published: 2026-09-22 06:35:19+00:00

There is a hidden dependency in a lot of LLM safety benchmarks:

the detector.

You send an adversarial prompt to a model, collect its response, and then some classifier decides whether that response represents refusal, compliance, or ambiguity.

Eventually those classifications become percentages in a safety report.

**But what happens when the detector itself is wrong?**

We encountered exactly that problem while testing our open-source AI security evaluation framework.

What began as an investigation into apparently inconsistent model behavior eventually uncovered:

• Unicode normalization failures

• incomplete refusal vocabularies

• cross-model differences

• and, most importantly, false PASS classifications introduced by our own detector improvement

The last failure was the most concerning because uncertainty is visible. A false PASS isn't.

**Full investigation:**

[https://agentsafelabs.com/blog/we-thought-wed-found-a-model-bug-wed-actually-found-a-detector-bug/](https://agentsafelabs.com/blog/we-thought-wed-found-a-model-bug-wed-actually-found-a-detector-bug/)

**Open-source implementation:**

[https://github.com/AgentSafeLabs/safelabs-eval](https://github.com/AgentSafeLabs/safelabs-eval)

I'd be interested in how other developers are testing the classifiers they use to evaluate LLM behavior.