{"slug": "we-thought-the-llm-was-wrong-our-safety-detector-was-wrong", "title": "We Thought the LLM Was Wrong. Our Safety Detector Was Wrong.", "summary": "AgentSafeLabs found that its open-source AI security evaluation framework was producing false PASS classifications due to bugs in its own safety detector rather than in the LLMs being tested. The investigation uncovered Unicode normalization failures, incomplete refusal vocabularies, and cross-model differences, with the detector improvement itself introducing the most concerning false positives. The team has published the full investigation and the open-source implementation on GitHub.", "body_md": "There is a hidden dependency in a lot of LLM safety benchmarks:\n\nthe detector.\n\nYou send an adversarial prompt to a model, collect its response, and then some classifier decides whether that response represents refusal, compliance, or ambiguity.\n\nEventually those classifications become percentages in a safety report.\n\n**But what happens when the detector itself is wrong?**\n\nWe encountered exactly that problem while testing our open-source AI security evaluation framework.\n\nWhat began as an investigation into apparently inconsistent model behavior eventually uncovered:\n\n• Unicode normalization failures\n\n• incomplete refusal vocabularies\n\n• cross-model differences\n\n• and, most importantly, false PASS classifications introduced by our own detector improvement\n\nThe last failure was the most concerning because uncertainty is visible. A false PASS isn't.\n\n**Full investigation:**\n\n[https://agentsafelabs.com/blog/we-thought-wed-found-a-model-bug-wed-actually-found-a-detector-bug/](https://agentsafelabs.com/blog/we-thought-wed-found-a-model-bug-wed-actually-found-a-detector-bug/)\n\n**Open-source implementation:**\n\n[https://github.com/AgentSafeLabs/safelabs-eval](https://github.com/AgentSafeLabs/safelabs-eval)\n\nI'd be interested in how other developers are testing the classifiers they use to evaluate LLM behavior.", "url": "https://wpnews.pro/news/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong", "canonical_source": "https://dev.to/iamwaqarjaved/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong-50pc", "published_at": "2026-09-22 06:35:19+00:00", "updated_at": "2026-09-22 06:52:43.130160+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research", "ai-tools", "mlops"], "entities": ["AgentSafeLabs", "GitHub", "safelabs-eval"], "alternates": {"html": "https://wpnews.pro/news/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong", "markdown": "https://wpnews.pro/news/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong.md", "text": "https://wpnews.pro/news/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong.txt", "jsonld": "https://wpnews.pro/news/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong.jsonld"}}