{"slug": "do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends", "title": "Do AI Coding Tools Actually Make Developers Faster? The Data Says It Depends", "summary": "A review of research on AI coding assistants finds that productivity gains depend heavily on context: field experiments with 4,867 developers at Microsoft, Accenture, and an anonymous Fortune 100 company found GitHub Copilot users completed 26.08% more tasks, with larger gains for less experienced developers, while a July 2025 METR randomized controlled trial of 16 experienced open-source developers working in familiar repositories found they took 19% longer with early-2025 AI tools even though they believed they were about 20% faster. METR later cautioned against treating its 2025 result as a universal statement about software development.", "body_md": "Artificial intelligence is changing the way software is developed. Today, developers can ask an AI coding assistant to generate a function, explain an error, write a test, refactor code, or suggest a solution to a problem that might otherwise take several searches and attempts.\n\nThis naturally raises an important question: Do AI coding tools actually make developers faster?\n\nThe simple answer is: sometimes.\n\nResearch on developer productivity presents a more complicated picture than the popular idea that AI automatically makes programmers more productive. Some studies have found significant productivity gains, while another controlled study found that experienced developers actually took longer when AI tools were available.\n\nThe difference suggests that the real question may not be whether AI makes developers faster, but when and under what conditions AI makes developers faster.\n\nEvidence That AI Can Increase Developer Productivity\n\nOne of the strongest pieces of evidence comes from a set of field experiments conducted at Microsoft, Accenture, and an anonymous Fortune 100 company.\n\nThe researchers studied 4,867 software developers and evaluated what happened when developers were given access to GitHub Copilot. Across the three experiments, developers with access to the AI coding assistant completed 26.08% more tasks than those without access. The researchers also found that less experienced developers experienced greater productivity gains.\n\nThis is important because the study was conducted in real workplace environments rather than only asking developers whether they felt more productive.\n\nIt provides evidence that AI coding assistants can improve measurable output in some professional software-development settings.\n\nBut that is not the whole story.\n\nWhen AI Made Developers Slower\n\nIn July 2025, METR published a randomised controlled trial involving experienced open-source developers. The researchers studied 16 developers completing 246 tasks in mature repositories that they already had experience working with.\n\nThe result was surprising.\n\nWhen developers were allowed to use early-2025 AI tools, they took 19% longer to complete the tasks compared with when AI tools were not allowed. Before the experiment, developers expected AI to reduce their completion time by 24%. Even after completing the tasks, they believed AI had made them about 20% faster.\n\nThis creates an interesting gap between perceived productivity and measured productivity.\n\nA developer can feel faster because an AI assistant quickly produces code, explanations or suggestions. But producing code faster is not necessarily the same as completing a software-development task faster.\n\nThe developer still has to understand the generated code, check whether it works, test it, identify mistakes, integrate it into an existing codebase and sometimes spend additional time correcting the AI's suggestions.\n\nSo Which Study Is Right?\n\nI don't think these findings necessarily contradict each other.\n\nThey studied different developers, environments, tasks and AI-tool contexts.\n\nThe Microsoft-led research examined thousands of developers working in company environments and found a substantial increase in completed tasks. The METR study focused on experienced open-source developers working on repositories they already knew well.\n\nThe difference is important.\n\nA developer working on repetitive tasks may benefit greatly from AI-generated boilerplate, documentation, tests or routine code. On the other hand, a developer working inside a complex and familiar codebase may spend considerable time reviewing, correcting and integrating AI-generated suggestions.\n\nIn other words, the value of AI may depend as much on the task as on the tool.\n\nMETR itself later cautioned against treating its 2025 result as a universal statement about software development. In February 2026, the organisation reported that a newer experiment was affected by selection effects and concluded that its newer data was not reliable enough to estimate the current productivity effect precisely. It also noted that AI may be speeding developers up more in early 2026 than its earlier study estimated.\n\nThat is a useful reminder: AI development is moving quickly, so productivity research can become outdated as tools and developer workflows change.\n\nWhat This Means for Developers\n\nFor me, the most useful lesson is that using AI is not the same as becoming a better developer.\n\nAI can reduce the time required to perform certain tasks, but developers still need the technical knowledge to judge whether the output is correct.\n\nFor example, an AI tool can generate a piece of code in seconds. But if I cannot explain what that code does, identify its assumptions, test it properly or recognise when it is wrong, then the speed I gained may come with a larger problem later.\n\nThis is especially important for developers who are still learning.\n\nThe temptation is to ask AI for the complete solution whenever we encounter a difficult problem. That can produce an immediate result, but it can also remove some of the thinking required to develop strong programming skills.\n\nA better approach is to use AI as a development partner rather than a replacement for development knowledge.\n\nI can use it to:\n\nexplain an unfamiliar error;\n\nsuggest different approaches to a problem;\n\ngenerate repetitive boilerplate;\n\ncreate initial test cases;\n\nhelp me understand documentation;\n\nreview or refactor code; and\n\nchallenge my existing solution.\n\nBut I should still be responsible for understanding, testing and validating the final code.\n\nThe Real Productivity Skill\n\nThe future of software development may therefore require a different kind of productivity skill.\n\nIt will not simply be about who can generate the most code with AI.\n\nIt will be about who knows when AI should be used, how it should be used, and when its output should be questioned.\n\nThe research does not give us one universal answer to whether AI makes developers faster. Instead, it gives us evidence that the impact varies across people, tasks and environments.\n\nFor developers, that means measuring productivity by more than the speed at which code appears on the screen.\n\nA faster developer is not necessarily the developer who writes code fastest.\n\nIt may be the developer who can move from problem → solution → tested and maintainable software efficiently.\n\nAI can help with that journey. But the developer still has to understand where the journey is going.\n\nReferences\n\nCui, Z., Demirer, M., Jaffe, S., Musolff, L., Peng, S., & Salz, T. (2025). The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers. Microsoft Research.\n\nBecker, J., Rush, N., Barnes, B., & Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR.\n\nBecker, J., Rush, N., Cunningham, T., Rein, D., & Mahamud, K. (2026). We are Changing our Developer Productivity Experiment Design. METR.", "url": "https://wpnews.pro/news/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends", "canonical_source": "https://dev.to/owusuaduomiflorencefl/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends-1e9c", "published_at": "2026-09-18 23:14:30+00:00", "updated_at": "2026-09-18 23:25:49.124583+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products", "ai-research"], "entities": ["GitHub Copilot", "Microsoft", "Accenture", "METR"], "alternates": {"html": "https://wpnews.pro/news/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends", "markdown": "https://wpnews.pro/news/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends.md", "text": "https://wpnews.pro/news/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends.txt", "jsonld": "https://wpnews.pro/news/do-ai-coding-tools-actually-make-developers-faster-the-data-says-it-depends.jsonld"}}