# AI Models Disagree on Code Flaws

> Source: <https://osintsights.com/ai-models-disagree-on-code-flaws>
> Published: 2026-11-18 10:06:28+00:00

**Recommendation:** Run vulnerability scans through multiple models and compare their findings; Echo's test found three leading models agreed on just 1 of 121 potential flaws.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

Imagine a scenario where an attacker and a defender use different AI models to identify code flaws - the defender might completely miss the vulnerability that's about to be exploited. A recent test by Echo, a software security firm, found that three leading AI models produced vastly different results, agreeing on just one potential issue out of 121.
