AI For Security Review In Application Code

In a 2025 benchmark comparing three industry static analysis tools against three frontier LLMs on sixty-three real vulnerabilities in ten C# projects, the LLMs achieved F1 scores between 0.75 and 0.80, significantly outperforming the best static tool (Snyk Code at 0.55) and the worst (SonarQube at 0.26). However, the LLMs win on recall while losing badly on precision, with a separate IDOR detection analysis finding that 88% of issues flagged by a popular AI coding agent were false positives. The trade-off means AI security review catches more real vulnerabilities but also generates substantial noise, requiring careful pipeline design to separate signal from false alarms.