AWS puts AI vulnerability detection to the test, and false positives pile up

AWS released the Deception Benchmark, a public dataset and evaluation process that measures how well AI models distinguish genuine security vulnerabilities from code that looks risky but is safe, and the results show false positives piling up. AWS said it is making the benchmark publicly available so researchers can use the dataset and evaluation process without repeating the cost of generating and refining the samples. High false-positive rates in AI-assisted vulnerability triage, penetration testing, threat modeling, incident response, and code review can create more work, increase alert fatigue, and reduce confidence in the tools, according to Help Net Security.

AWS’ Deception Benchmark measures how well AI models distinguish genuine security vulnerabilities from code that looks risky but is safe. AWS is making it publicly available so researchers can use the dataset and evaluation process without repeating the cost of generating and refining the samples. Security teams use AI for vulnerability triage, penetration testing, threat modeling, incident response, and code review. High false-positive rates can create more work, increase alert fatigue, and reduce confidence in … More https://www.helpnetsecurity.com/2026/09/14/aws-deception-benchmark-security-vulnerabilities/ The post AWS puts AI vulnerability detection to the test, and false positives pile up https://www.helpnetsecurity.com/2026/09/14/aws-deception-benchmark-security-vulnerabilities/ appeared first on Help Net Security https://www.helpnetsecurity.com .