# Measuring engineering productivity is harder than ever > Source: > Published: 2026-07-21 07:30:28+00:00 You have **1** article left to read this month before you need to [register](/register) a free LeadDev.com account. Estimated reading time: 11 minutes **Key takeaways:** - AI hasn’t **broken productivity**– it’s broken our** proxies for measuring it**. **Activity and business value are pulling apart**. The most valuable engineering work is increasingly invisible to traditional dashboards.- Sort your metrics into activity and outcome. **Most dashboards are counting the wrong things**. [Measuring engineering productivity](https://leaddev.com/reporting/its-time-to-rethink-how-we-measure-engineering-productivity) has never been straightforward, and [AI has made it harder](https://leaddev.com/ai/the-struggle-to-prove-ai-productivity-gains). At OpenAI, engineers who lean heavily on Codex open roughly 70% more pull requests than colleagues who don’t – and the gap keeps widening, according to [Sherwin Wu](https://www.linkedin.com/in/sherwinwu1), who leads engineering for OpenAI’s API platform. Read that number as a productivity [metric](https://leaddev.com/reporting/introducing-engineering-metrics-your-organization) on its own and you’d conclude the heavy [Codex](https://leaddev.com/ai/openais-5-codex-here) users are your best engineers. That conclusion might be right. It might also be exactly backwards. The number alone can’t tell you which. That’s the problem now sitting on every engineering leader’s desk. Quarterly business reviews used to be straightforward. [Engineering leaders ](https://leaddev.com/the-engineering-leadership-report-2026/)would open their dashboards, review familiar metrics like pull requests, commit counts, and story-point velocity, and compare them with the previous quarter. No single metric captured software engineering, but together they painted a reasonable picture of team performance. That picture is now blurring. Yet everything else tells a different story: the team has shipped more customer-facing features, incidents have dropped, engineers spend less time on repetitive implementation, customers are happier, and releases remain predictable. ## Your inbox, upgraded. Receive weekly engineering insights to level up your leadership approach. So which version of reality should leaders trust? Neither view is wrong. They’re measuring different things. For years, commits, pull requests, story points, and deployment frequency have served as proxies for productivity. They were never perfect, but they broadly tracked how software got built: engineers wrote code, reviewed it, tested it, shipped it. As the work evolved, so did the metrics. AI hasn’t broken engineering productivity. It has broken our proxies for measuring it. [Stack Overflow’s 2025 Developer Survey](https://survey.stackoverflow.co/2025/ai) found that 84% of developers now use or plan to use [AI tools](https://leaddev.com/ai/best-ai-coding-assistants), up from 76% the year before. However, only 52% say those tools have actually made them more productive, and trust in AI-generated output has fallen even as adoption climbed. Adoption and value are already decoupling at the individual developer level, not just the organizational level most enterprise AI surveys track. Software engineering sits at the center of that shift. [Coding assistants](https://leaddev.com/ai/your-ai-coding-tools-buying-checklist-for-2026) were the first step. Now teams are experimenting with agentic systems that plan, implement, test, and refine software with increasing autonomy. Engineers are spending less time producing code directly and more time reviewing AI-generated changes, defining architectural intent, setting guardrails, and improving the workflows that generate software in the first place. The work is changing faster than the metrics built to measure it. That doesn’t make productivity unmeasurable – it means leaders need to rethink what productivity looks like when the highest-value contributions are increasingly invisible to traditional dashboards. ## Engineering productivity was never easy to measure AI creates the impression that measurement has suddenly gotten hard. In reality, [software engineering](https://leaddev.com/ai/how-ai-will-change-software-engineering) has resisted simple measurement for decades. Unlike manufacturing or sales, engineering is knowledge work – creative, collaborative, contextual. Two engineers can spend identical time on different problems, produce wildly different amounts of code, and deliver comparable value. The industry has cycled through proxies as a result, and each one rewarded something slightly wrong: **Lines of code** rewarded volume, not judgment. The engineers who simplified systems or automated repetitive work looked unproductive by this measure.**Commit counts** had the same flaw. A senior engineer untangling a distributed-systems bug might produce a handful of commits over several days, while routine interface changes generate dozens, with no relationship to value delivered.**Story points** were built for planning, not performance review, and broke the moment organizations started treating velocity as a score instead of an estimate. Even today’s best-regarded metrics require context. The [DORA metrics](https://dora.dev/) – deployment frequency, lead time for changes, change failure rate, and mean time to recovery – are the industry benchmark for delivery system performance. They say little about individual developer productivity. That’s not a gap; it’s by design. [DORA](https://leaddev.com/reporting/are-dora-metrics-right-your-team) measures whether the delivery system performs well. The [SPACE framework](https://queue.acm.org/detail.cfm?id=3454124), introduced by Nicole Forsgren and colleagues in 2021, measures whether developers and teams are working effectively across satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Forsgren’s central argument was that productivity is inherently multidimensional and can’t be reduced to a single metric like commits or lines of code. The two frameworks complement each other, and together they give a far richer picture than activity metrics alone. A senior engineer [mentoring](https://leaddev.com/management/how-be-effective-mentor) juniors, simplifying an architecture, or preventing a production incident may leave almost no trace in Git history, yet that work often creates more long-term value than hundreds of commits. Traditional metrics were never built to capture it. They simply reflected a world where software was produced primarily by hand. That assumption is now breaking down. ## AI changed the work before it changed the metrics Most of the conversation about AI focuses on speed – coding assistants generating boilerplate, scaffolding services, automating repetitive tasks. Those gains are real, but they’re the least interesting part of the story. The bigger change isn’t that engineers write code faster. It’s that they spend their time on different kinds of engineering entirely: reviewing AI-generated implementations, comparing design approaches, refining specifications and constraints, validating security and compliance, evaluating generated tests, improving prompts and workflows, and making architectural calls that shape what gets built next. OpenAI’s own engineering org is a useful before/after example. Wu has said that [95% of OpenAI’s engineers](https://www.lennysnewsletter.com/p/engineers-are-becoming-sorcerers) now use Codex daily and effectively all merged pull requests get an AI review first. Human reviewers, meanwhile, have gone from reading nearly every line to skimming a much smaller share, catching what the model missed rather than re-deriving the change themselves. Some engineers now run 10 to 20 AI-coding threads in parallel. Read naively, OpenAI’s dashboard says: pull requests up 70%, review time down – huge productivity win. Read carefully, it says something narrower: these engineers got better at directing and checking AI output. Whether that’s actually valuable depends entirely on judgment the metric can’t see. The work that actually determines whether the 70% of extra pull requests are any good – deciding what to build, catching the change that looks right but isn’t, and knowing when to trust the model and when not to – happens upstream of the metric. Ironically, the more leverage AI provides, the less visible the resulting work becomes. A sharp architectural review can prevent months of rework. A well-scoped specification can let AI generate thousands of lines of production-ready code. A well-designed evaluation framework can lift the quality of every AI-generated change across an organization. None of that shows up in commit counts. The question is no longer* “how productive are our developers?” *It’s becoming:* *how do we measure an engineering organization where humans and AI increasingly operate as a single delivery system? ## More like this ## From product engineers to factory engineers Warp CEO [Zach Lloyd ](https://www.linkedin.com/in/zachlloyd/)captured this shift, [arguing ](https://www.warp.dev/blog/we-are-now-factory-engineers-not-product-engineers)that engineers are becoming “factory engineers” rather than “product engineers.” This means value increasingly comes from improving the system that produces software, not from manually shipping every feature. Success is measured by how well the factory runs, not by how much code any one engineer writes. [Matan Grinberg](https://www.linkedin.com/in/matan-grinberg/), CEO of an AI-coding agent Factory, makes a similar point in a conversation with [McKinsey](https://www.mckinsey.com/capabilities/mckinsey-technology/our-insights/paving-the-road-for-ai-agents-interview-with-factory-ceo-matan-grinberg). He compares AI agents to *“Ferraris on dirt roads”*– powerful tools that can only deliver value when organizations have invested in the underlying engineering foundations, including documentation, test coverage, CI/CD, and observability. The unglamorous infrastructure work that never shows up as a feature shipped. Role boundaries blur along with it –* *the live question stops being who wrote which line of code and becomes who owns the customer outcome. This isn’t engineering’s first identity shift. DevOps automated delivery pipelines. Site Reliability Engineering replaced firefighting with engineered reliability. Platform engineering replaced one-off infrastructure work with shared capability. Agentic AI extends the same pattern. Engineers now design workflows, define guardrails, validate AI output, and continuously improve the system that generates the software. ## When activity stops representing value Commits, pull requests, tickets closed, and story points remain useful signals of activity. They’re becoming weaker signals of value. [Laura Tacho](https://www.linkedin.com/in/lauratacho/), who researched how engineering organizations actually measure AI’s impact, ran into this at scale. Studying 18 companies including Google, GitHub, Dropbox, Microsoft, Atlassian, and Booking.com for a [guest piece in The Pragmatic Engineer](https://newsletter.pragmaticengineer.com/p/how-tech-companies-measure-the-impact-of-ai), she found that pull request count alone kept fooling people. The companies that got it right paired activity numbers with a quality check before trusting them: - Dropbox found engineers who regularly use AI ship 20% more pull requests, but only reports that number alongside change failure rate, which fell rather than rose. - Webflow found the pull request-volume gain wasn’t evenly spread. Engineers with three or more years at the company drove the real throughput increase, roughly 20%, while newer hires barely moved - Microsoft tracks something further upstream entirely – “Bad Developer Days,” a friction measure with nothing to do with output at all As CircleCI’s Shelly Stuart put it in Tacho’s research, output [metrics](https://leaddev.com/reporting/metrics-dont-tell-whole-story) show what’s happening while developer experience data is what tells you whether it’s actually sustainable. The same pattern shows up across teams generally. When AI frees up implementation time, strong teams rarely just ship more features with the extra capacity. They reduce technical debt, modernize legacy systems, and take on operational work that used to compete with feature delivery. A dashboard reading only activity will register that as a slowdown. It’s usually the opposite: an investment in the organization’s long-term ability to deliver. This is the real reason [productivity is getting harder to measure](https://leaddev.com/reporting/its-time-to-rethink-how-we-measure-engineering-productivity) – not because AI makes it unmeasurable, but because visible activity and business value are pulling apart. The question worth asking is shifting from “who produced the most code?” to “who improved the system the most?” ## Measuring what actually matters The fix isn’t a new set of vanity metrics. Counting AI-generated lines of code or tracking prompt usage just recreates the old problem in a new shape – activity, not impact. Instead, leaders should evaluate whether the engineering system as a whole is getting better at delivering customer value. DORA’s 2025 report gives a useful starting checklist for that. Its [AI Capabilities Model](https://dora.dev/dora-report-2025/) names seven organizational capabilities that determine whether AI adoption compounds or backfires: - A clear, communicated AI policy. - A healthy data ecosystem. - AI-accessible internal data. - Strong version-control discipline - Working in small batches. - A user-centric focus. - A quality internal platform. None of these are metrics you screenshot for a board deck; they’re closer to a maturity audit, and they’re a more actionable place to start than inventing another dashboard. Beyond that checklist, three shifts help: ### Individual output → team outcomes Software has always been a team sport, but autonomous development systems make an individual lens actively misleading. A team writing less code may be delivering more value because engineers are spending time on architecture, technical debt, and platform capability instead. [DORA’s delivery metrics](https://leaddev.com/ai/the-8-software-engineering-metrics-ai-broke) stay useful here, not as a scorecard for people but as a read on how well the system performs. ### Activity → business value As AI lowers the cost of implementation, the edge comes from solving the right problems, not producing more of them. Are customers adopting what gets built? Is reliability improving? Are engineers spending less time on repetitive work and more on complex problems? These questions don’t replace engineering metrics, they give the metrics meaning. ### Effort → leverage An engineer who builds a reusable workflow, automates a test pipeline, or writes solid guardrails for AI agents may show almost no visible code, yet improve the output of dozens of colleagues. This has always been true of staff and principal engineers, AI agents just extend the principle further down the org chart. That leverage is productivity, even when it never shows up in a commit graph. None of this works in isolation, either, and Dropbox’s discipline is the exception, not the norm. [Cortex’s 2026 Engineering in the Age of AI benchmark report](https://www.cortex.io/post/ai-is-making-engineering-faster-but-not-better-state-of-ai-benchmark-2026) – drawn from a survey of 50+ engineering leaders plus real development metrics across multiple organizations – found the same pull request-volume-vs-quality split playing out industry-wide, minus the pairing that made Dropbox’s number trustworthy: pull requests per author are up 20% year-over-year, but incidents per pull request have risen 23.5% and change failure rates are up roughly 30%. AI doesn’t remove the need for good engineering leadership. It raises the stakes on it. **Berlin** • **November 9 & 10, 2026** **Engineering leadership has never moved this fast. ** See how other leaders are keeping pace at **LeadDev Berlin**. ## Before your next QBR If you’re staring down a review where the old numbers look wrong, three moves are more useful than a new dashboard: **Sort your current metrics into two piles: activity and outcome.** Anything counting things produced (commits, pull requests, tickets, story points) is activity. Anything measuring what happened as a result (lead time, rework rate, customer adoption, incident rate) is outcome. If your board deck is mostly the first pile, that’s the actual problem, not AI.**Pick one activity metric to retire this quarter and replace it with an outcome metric.** The way Dropbox refuses to report pull request volume without change failure rate sitting right next to it. You don’t need to overhaul everything at once; one credible pairing changes the conversation.**Ask your senior engineers directly where their best work this quarter doesn’t show up in Git.** The answers – an incident prevented, a junior engineer leveled up, or an architecture simplified – are usually the clearest evidence that your current dashboard is undercounting them. The dashboard from the opening example wasn’t wrong. It was incomplete. The leaders who adjust it fastest – swapping in outcome metrics one at a time rather than waiting for a perfect AI-era framework – will spend next quarter’s QBR explaining real wins instead of explaining away a red number that never should have been red.