Benchmarks are not what matters in the real world to product security #
One of our core beliefs is that no current benchmark accurately reflects what customers need from autonomous AI AppSec. You’ll see numbers thrown around, like number of bugs detected or cost per bug, and while those are helpful, they don’t get to the core concern - how does this technology actually decrease risk in practice?
For us it comes down to practical, real world questions:
- What is it finding?
- Does what it finds actually matter?
- How much work and time is required for my team to validate the findings?
- Do you compress the amount of time, effort, and costs required to go from find to fix?
Kia’s challenge #
Product security teams were already short staffed even before AI expanded the attack surface. Kia’s Red Team was looking to not just find more bugs, but to accelerate remediating important vulnerabilities.
How Kia is using Xint #
Kia’s Red Team brought in Xint to take on the repetitive, time-consuming parts of security testing. Xint finds vulnerabilities and delivers validated findings with clear remediation guidance, so the team spends less time chasing false positives and more time getting real issues fixed. With Xint, the same single tester can now cover more ground on every project.
With Xint, Kia’s Red Team has increased bug discovery by 33%, bug triage by 17%, and bug remediation by 50% in an average month, all with the same headcount of one tester. Overall, the team now handles roughly 30–40% more vulnerabilities per month than before. For Kia, the important number isn’t the quantity of bugs they find, but rather the increased throughput in real vulnerabilities addressed before they become a problem.