{"slug": "30-stats-every-engineering-leader-should-know-about-ai-era-software-quality", "title": "30 stats every engineering leader should know about AI-era software quality", "summary": "A vendor-authored roundup compiles 30 sourced statistics on AI-era software quality, each tagged by evidence type and dated, and opens by debunking the long-cited claim that production bugs cost 100 times more to fix than design-stage ones, tracing it to 1987 textbook notes citing IBM course material possibly dating to 1967. The cited figures include DORA 2025 finding 90% of software professionals use AI while 30% trust its output little or not at all, and Stack Overflow 2026 finding only 6.6% of developers trust AI with important decisions.", "body_md": "Most statistics in this field do not survive being looked up. Take the claim that a production bug costs 100 times more to fix than one caught in design. It traces back not to a study, but to IBM internal course notes cited in a 1987 textbook, whose underlying data “could be as old as 1967” ([The Register](https://www.theregister.com/2021/07/22/bugs_expense_bs)). It has been repeated for forty years as though it were research.\n\nEvery figure below names its source, its date and its sample, and links to somewhere you can check it. Each also carries a tag showing what kind of evidence it is. A peer-reviewed paper, a regulatory filing and a vendor’s customer survey are not equally strong, and you should not have to guess which one you are reading. Antidote sells autonomous verification, so read our own framing with the same scepticism.\n\n**1. 90% of software professionals now use AI, but 30% trust its output little or not at all.**\n\n23% trust it \"a little\" and 7% \"not at all\". Nearly 5,000 respondents worldwide.\n\n*Industry survey, Google-run* · [DORA 2025, Google](https://blog.google/innovation-and-ai/technology/developers-tools/dora-report-2025/)\n\n**2. Only 6.6% of developers trust AI with important decisions.**\n\nThe most common answer, at 48%, was \"I trust it when I can easily verify the output\". 14,304 answered the question, in a survey of more than 30,000 people across 169 countries, published October 2026.\n\n*Developer survey* · [Stack Overflow 2026](https://survey.stackoverflow.co/2026/ai/data/ai-trust)\n\n**3. 96% of developers don't fully trust AI code to be correct, yet only 48% always check it before committing.**\n\nAI already accounts for 42% of committed code, and developers expect 65% by 2027. More than 1,100 developers, January 2026.\n\n*Vendor survey* · [Sonar](https://www.sonarsource.com/company/press-releases/sonar-data-reveals-critical-verification-gap-in-ai-coding/)\n\n**4. 70% of software leaders believe application quality is already suffering.**\n\n93% have adopted AI coding tools, and 40% generate more than 40% of their code with AI. Of 273 CTOs, CIOs, developers and QA directors, 86% are raising testing budgets by more than 11% this year.\n\n*Vendor survey* · [SmartBear](https://www.businesswire.com/news/home/20260318326658/en/SmartBear-Survey-70-of-Software-Experts-Concerned-that-Application-Quality-is-Suffering-Given-Faster-AI-Code-Development)\n\n**5. \"Almost right, but not quite\" is developers' biggest frustration with AI tools.**\n\n66% of 31,476 developers chose it, the most-selected of seven options. Runner-up: 45.2% say debugging AI-generated code takes longer. This is from 2025, the most recent edition to ask the question.\n\n*Developer survey* · [Stack Overflow 2025](https://survey.stackoverflow.co/2025/ai)\n\n**6. AI-co-authored pull requests carry 1.7 times more issues than human ones.**\n\n10.83 issues per PR against 6.45, across 320 AI-co-authored and 150 human-only open-source PRs. Logic and correctness issues were 75% more common. Authorship was inferred, not confirmed.\n\n*Vendor analysis* · [CodeRabbit](https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report)\n\n**7. AI-generated code introduces a known security flaw 45% of the time, no better than two years ago.**\n\nMore than 95% of it is syntactically correct. Defences against cross-site scripting passed just 15% of the time. 150+ models, four languages, March 2026.\n\n*Vendor benchmark* · [Veracode](https://www.veracode.com/blog/spring-2026-genai-code-security/)\n\n**8. Code duplication is up 81% and refactoring has collapsed.**\n\nDuplicated blocks rose from 40.3 to 73.0 per million changed lines between 2023 and 2026. Moved or refactored lines fell from 21% of changes in 2022 to 3.8%. 623 million analysed changes.\n\n*Vendor telemetry* · [GitClear](https://www.gitclear.com/the_ai_code_quality_maintainability_gap)\n\n**9. At teams with high AI adoption, incidents per merged PR more than tripled.**\n\nUp 243%, with monthly incidents up 58% and bugs per developer up 54%. Task throughput per developer rose only 34%. 22,000 developers across 4,000+ teams, two years of data to March 2026. It is a before-and-after comparison within teams, not a controlled experiment.\n\n*Vendor telemetry* · [Faros AI](https://pages.faros.ai/hubfs/AI_Engineering_Report_2026_The_Acceleration_Whiplash_Faros.pdf)\n\n**10. Median time in code review rose 441%, and PRs merged with no review rose 31%.**\n\nSame teams, same comparison. The bottleneck has moved from writing code to checking it.\n\n**11. Main-branch success fell to 70.8%, its lowest in over five years, before recovering to 78.3%.**\n\nAcross 28.7 million workflows in September 2025, median teams' main-branch throughput fell 7% while feature-branch throughput rose 15%. By August 2026, across 22.1 million workflows, success had recovered to 78.3%.\n\n*Vendor telemetry* · [CircleCI 2026](https://circleci.com/landing-pages/assets/2026-state-of-software-delivery-report.pdf), [CircleCI Q3 2026](https://circleci.com/landing-pages/assets/2026-q3-state-of-software-delivery-report.pdf)\n\n**12. Developers saving 10+ hours a week with AI fell from 68% to 26% in a year.**\n\n65% spend time correcting AI output that missed the original intent. Only 17% report a clear AI strategy across the delivery lifecycle. 1,560 respondents, self-reported.\n\n*Vendor survey* · [Atlassian via InfoQ](https://www.infoq.com/news/2026/10/atlassian-devex-2026/)\n\n**13. 87% were confident they could tell within 24 hours whether AI code caused an incident. Of those that had one, 34% couldn't.**\n\nHarris Poll, 1,528 developers and technology buyers, six countries, June 2026.\n\n*Vendor survey* · [GitLab](https://s204.q4cdn.com/984476563/files/doc_news/GitLab-Research-Reveals-Organizations-Are-Generating-AI-Code-Faster-Than-They-Can-Control-It-2026.pdf)\n\n**14. 84% of Google's pass-to-fail test transitions involved a flaky test.**\n\nThat's against a steady background of about 1.5% of all test runs reporting a flaky result.\n\n*Company engineering data* · [Google Testing Blog, 2016](https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html)\n\n**15. A quarter of Google's tests using the Android emulator were flaky.**\n\n25.46% of them, against a general finding that \"the larger the test, the more likely it will be flaky\".\n\n*Company engineering data* · [Google Testing Blog, 2017](https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html)\n\n**16. Only 1.23% of Google's test targets ever found a real breakage.**\n\nIn one month covering more than 4 billion test outcomes, 2.07% of 5.5 million targets both passed and failed. Only 1.23% remained once flaky targets were filtered out.\n\n*Peer-reviewed* · [Memon et al., ICSE-SEIP 2017](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45861.pdf)\n\n**17. Up to half of all builds contain a flaky failure, and 86% of flaky tests only flake in CI.**\n\nThe figure was 14% to 52% of builds, depending on the project, across five Microsoft projects over 30 days, and 4.6% of individual tests were flaky. Of 315 flaky tests re-run 100 times on a local machine, 86% never flaked there.\n\n*Peer-reviewed* · [Lam et al., ISSTA 2019](https://www.microsoft.com/en-us/research/wp-content/uploads/2019/11/LamETAL19RootFinder.pdf)\n\n**18. Two thirds of Mozilla's job failures were not real failures.**\n\n25,871 of 38,596 failures across nearly 2 million runs, each classified by hand rather than by heuristic.\n\n*Peer-reviewed* · [Lampel et al., ESEC/FSE 2021](https://publications.cispa.saarland/3474/2/LJA%2B21.pdf)\n\n**19. Flaky tests outnumbered genuine failures four to one at Facebook.**\n\nIn CI data where almost 99.9% of dependency-selected test targets passed.\n\n*Peer-reviewed* · [Machalica et al., ICSE-SEIP 2019](https://arxiv.org/pdf/1810.05286)\n\n**20. Poor software quality costs the US at least $2.41 trillion.**\n\nOn top of that is about $1.52 trillion of accumulated technical debt, reported separately. A modelled estimate from 2022, with no later edition.\n\n*Consortium estimate* · [CISQ](https://www.it-cisq.org/wp-content/uploads/sites/6/2022/11/CPSQ-Report-Nov-22-2.pdf)\n\n**21. The average US data breach now costs a record $11.5 million, and one in four malicious breaches is AI-enabled.**\n\nThe global average is $4.99 million, up 12%, across 602 organisations breached between March 2025 and February 2026. AI-enabled attacks rose 56% in a year.\n\n*Vendor study* · [IBM via CFO.com](https://www.cfo.com/news/average-cost-of-data-breaches-is-up-12-this-year-IBM/827713/), [IBM X-Force](https://www.ibm.com/think/x-force/2026-cost-of-a-data-breach-ai-adversaries-enterprise-risk)\n\n**22. Downtime costs Global 2000 companies $600 billion a year, and every technology leader surveyed had an AI-related outage.**\n\nThat's up 50% in two years, or roughly $300 million per company. Half cited incorrect AI-driven automation. 2,000 executives, 20 countries, 2026.\n\n*Vendor survey* · [Splunk](https://www.splunk.com/en_us/blog/cio-office/ai-downtime-risks-resilience.html), [Cisco newsroom](https://newsroom.cisco.com/c/r/newsroom/en/us/a/y2026/m05/the-600-billion-wake-up-call-new-splunk-research-reveals-downtime-is-a-systemic-business-crisis.html)\n\n**23. 10 of 2025's 16 major cloud outages were caused by configuration or deployment changes.**\n\nCovers AWS, Azure, Google Cloud and Cloudflare, mostly from official post-mortems. Two examples: a database permissions change took Cloudflare down for about five and a half hours in November 2025. AWS's October 2025 us-east-1 outage cost US companies an estimated $500–650 million.\n\n*Vendor analysis of post-mortems* · [Control Plane](https://controlplane.com/blog/post/cloud-outage-report-2026), [Cloudflare](https://blog.cloudflare.com/18-november-2025-outage), [Parametrix via Insurance Edge](https://insurance-edge.net/2025/10/30/parametrix-estimates-aws-outage-costs/)\n\n**24. One bad update cost the Fortune 500 an estimated $5.4 billion, and Delta around $500 million on its own.**\n\nCrowdStrike, July 2024; the Fortune 500 figure excludes Microsoft and was modelled by an outage insurer. Delta reported $380 million of lost revenue and $170 million of extra non-fuel cost, offset by $50 million of unburned fuel, across 7,000 cancellations in five days.\n\n*Insurer model · Regulatory filing* · [Parametrix](https://www.parametrixinsurance.com/reports-white-papers/crowdstrikes-impact-on-the-fortune-500), [Delta, SEC filing](https://www.sec.gov/Archives/edgar/data/27904/000168316824007033/delta_ex9901.htm)\n\n**25. UK banks logged 803 hours of unplanned outage in two years.**\n\nMore than 33 days, across at least 158 incidents at nine institutions between January 2023 and February 2025, on their own returns to Parliament.\n\n*Parliamentary data* · [Treasury Committee, 2025](https://committees.parliament.uk/committee/158/treasury-committee/news/205611/)\n\n**26. A 31% faster page lifted Vodafone's sales by 8%.**\n\nA genuine 50/50 A/B test with no visual or functional differences between versions. Around 34,000 visits per version per day, all paid media traffic.\n\n*Controlled experiment* · [Google web.dev](https://web.dev/case-studies/vodafone)\n\n**27. A 1.09% crash rate is enough for Google Play to flag your app.**\n\nAbove that share of daily users experiencing a crash, an app crosses Google's \"bad behaviour\" threshold. For a single device model, the threshold is 8%.\n\n*Platform policy* · [Google Play Console](https://support.google.com/googleplay/android-developer/answer/9844486)\n\n**28. Shoppers cut spending after 47% of bad experiences.**\n\nDown 8 points year on year, but 58% in online retail and 62% in fast food. 20,000+ consumers, 14 countries, Q3 2025.\n\n*Vendor survey* · [Qualtrics XM Institute](https://www.qualtrics.com/articles/customer-experience/3-trillion-risk-due-bad-customer-experiences-2026/)\n\n**29. Gartner expects 70% of enterprises to have AI-augmented testing tools by 2028.**\n\nUp from 20% in early 2025. From Gartner's first Magic Quadrant for AI-Augmented Software Testing Tools, October 2025.\n\n*Analyst forecast* · [Gartner, via Keysight](https://markets.ft.com/data/announce/detail?dockey=600-202511111100BIZWIRE_USPRX____20251111_BW531652-1)\n\n**30. Only 15% of companies have scaled AI in quality engineering enterprise-wide.**\n\n43% are still experimenting and 30% have it in limited use cases. Gen AI non-adopters rose from 4% to 11%. 2,000+ senior executives, 22 countries.\n\n*Vendor survey* · [World Quality Report 2025-26](https://www.sogeti.com/wp-content/uploads/sites/3/2025/11/2025_11_13_World_Quality_Report_2025.pdf)\n\nRead together, these numbers tell a consistent story. Writing code has stopped being the bottleneck; checking it now is. Developers say they trust AI when they can verify its output (2), but fewer than half verify it (3). Review queues have grown more than fivefold (10), more code ships with no review at all, and incidents per change are rising (9). Meanwhile the test suites meant to catch problems spend much of their time raising false alarms (14–19).\n\nMost failures that reach users show up as behaviour: a broken checkout, a crash, a config change that takes a region down. So the useful question is not whether more code passes review, but whether what you are about to ship still works for the people using it. You can only answer that by checking real behaviour, with evidence, before every release.\n\n*Originally published on the [Antidote blog](https://getantidote.com/blog/stats-ai-era-software-quality?utm_source=devto&utm_medium=syndication&utm_campaign=c18). I work at Antidote, which sells autonomous verification, so read our framing with the same scepticism.*", "url": "https://wpnews.pro/news/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality", "canonical_source": "https://dev.to/willmacpherson/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality-2ahi", "published_at": "2026-11-16 19:12:42+00:00", "updated_at": "2026-11-16 19:21:39.077604+00:00", "lang": "en", "topics": ["ai-tools", "ai-products", "artificial-intelligence"], "entities": ["Antidote", "Google", "DORA", "Stack Overflow", "Sonar", "SmartBear", "CodeRabbit", "Veracode"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality", "markdown": "https://wpnews.pro/news/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality.md", "text": "https://wpnews.pro/news/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality.txt", "jsonld": "https://wpnews.pro/news/30-stats-every-engineering-leader-should-know-about-ai-era-software-quality.jsonld"}}