cd /news/artificial-intelligence/is-ai-making-developers-worse-at-thi… · home › topics › artificial-intelligence › article
[ARTICLE · art-173907] src=dev.to ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Is AI Making Developers Worse at Thinking? What the Research Says (and 8 Habits That Help)

A review of randomized trials finds that AI coding assistants can slow experienced developers and reduce learning, with outcomes depending on how the tools are used. In a METR trial of 16 experienced open-source developers across 246 real issues, allowing AI increased completion time by 19% even though developers believed they were faster, and an Anthropic-run study of 52 mostly junior engineers found the AI-assisted group scored about 50% on a follow-up quiz versus about 67% for hand-coders. The piece recommends habits such as writing tests or plans before prompting and asking for explanations, arguing that staying cognitively engaged is what separates learning from mere shipping.

recommendation

Write the function signature, plan or failing test before prompting, and ask for explanations rather than full solutions to keep yourself engaged and preserve debugging skill.

more in playbook: Evals and testing for LLM apps
by read6 min views2 publishedNov 21, 2026

"AI is rotting developers' brains" makes a great headline. The evidence is more interesting than that. There is no proof that using an AI assistant lowers your intelligence. There is a small but growing set of randomized trials suggesting that how you use AI decides whether you learn or just ship.

Here is what the best studies found, where they fall short, and eight habits (plus a tiny script) you can use to keep your skills sharp.

METR, an independent research nonprofit, ran a randomized controlled trial with 16 experienced open-source developers working on 246 real issues in repositories they knew well. Some issues allowed AI tools (mainly Cursor Pro with Claude models, in early 2025) and some did not.

The result: allowing AI increased completion time by 19%. Both the developers and outside experts had predicted a speedup of roughly 20 to 40 percent, and developers still felt faster after the study.

Why this matters for thinking: the gap between how fast it felt and how fast it was suggests we are poor judges of our own AI-assisted work. Fluent output feels like progress.

Caveats you should know:

Researchers Judy Hanwen Shen and Alex Tamkin ran a randomized trial with 52 mostly junior professional engineers who had to learn a new Python library (Trio). Half had an AI assistant, half coded by hand. Everyone then took a quiz on concepts they had used minutes earlier.

(Disclosure: Anthropic, which makes Claude, ran this study, and it also published the results, which found a downside to AI assistance for learning.)

The results:

AI group Hand-coding group
Average quiz score about 50% about 67%
Task speed slightly faster, not statistically significant baseline
Biggest gap debugging questions

That is a gap of roughly 17 percentage points, close to two letter grades. The hand-coders also hit more errors themselves, which seems to have built their debugging skills.

Not everyone in the AI group did badly. The researchers found distinct usage patterns:

Pattern What it looks like Learning outcome
Delegation "Write this whole thing for me" Low
Progressive reliance Starts independent, then hands over more Low
Debugging loops Pastes errors until it works Low
Conceptual questions "Why does this library work this way?" Higher
Code plus explanation Asks for code and an explanation Higher
Generate, then understand Gets code, then asks follow-ups until it makes sense Higher

The high-scoring patterns scored well above the delegating ones. The takeaway is simple: staying cognitively engaged is the variable that matters.

The pattern is consistent: tools that hand you the answer help you now and can cost you later. Tools that make you work protect learning.

Before you prompt, write the function signature, a rough plan, or a failing test. You are forcing yourself to understand the problem, and the AI then fills a gap you defined.

def test_parse_duration():
    assert parse_duration("1h30m") == 5400
    assert parse_duration("45s") == 45
    assert parse_duration("") == 0

Use prompts that keep you in the loop.

Explain how X works and why this approach is better than Y.
Don't write the full solution. Give me the key idea and a small example.

Read every line. If you can't explain a line to a teammate, either learn it or delete it. A good rule: no merge without an explanation in your own words.

Give yourself 15 minutes to form a hypothesis before asking AI. Write the hypothesis down. This is the skill the studies showed fading fastest.

Bug: checkout total is off by 1 cent
Hypothesis: float rounding in the discount step
Test: print Decimal values at each stage

Athletes train without gear. Pick one regular session a week with no assistant: a kata, a small feature, or a code review. Think of it as maintaining a muscle.

Write it yourself, then ask for critique.

Review this function for bugs, edge cases and readability.
List problems only. Do not rewrite the code.

When you meet a new library, read the official docs and examples first, then use AI to quiz you or fill gaps. In the Anthropic study, this is the situation where delegating cost people the most.

The METR result is a warning that your sense of speed may be wrong. Run a small experiment.

Here is a tiny script that times tasks with and without AI so you can compare your own numbers.

#!/usr/bin/env python3
"""taskclock.py: time your tasks with and without AI, then compare."""
import csv
import statistics as st
import sys
import time
from pathlib import Path

LOG = Path.home() / ".taskclock.csv"

def log(name, ai, seconds):
    new = not LOG.exists()
    with LOG.open("a", newline="") as f:
        w = csv.writer(f)
        if new:
            w.writerow(["task", "ai", "seconds"])
        w.writerow([name, ai, round(seconds)])

def summary():
    if not LOG.exists():
        print("No data yet. Time a task first.")
        return
    with LOG.open() as f:
        rows = list(csv.DictReader(f))
    for flag in ("ai", "no-ai"):
        times = [int(r["seconds"]) for r in rows if r["ai"] == flag]
        if times:
            print(f"{flag:6} n={len(times):3}  median={st.median(times) / 60:.1f} min")

if __name__ == "__main__":
    if len(sys.argv) == 2 and sys.argv[1] == "summary":
        summary()
    elif len(sys.argv) == 3 and sys.argv[2] in ("ai", "no-ai"):
        start = time.time()
        input(f"Timing '{sys.argv[1]}' ({sys.argv[2]}). Press Enter when done...")
        log(sys.argv[1], sys.argv[2], time.time() - start)
    else:
        print("usage: taskclock.py 'task name' ai|no-ai   or   taskclock.py summary")

Usage:

python3 taskclock.py "add pagination" ai
python3 taskclock.py "fix flaky test" no-ai
python3 taskclock.py summary

To keep the test honest:

AI is not making developers dumber across the board. The better reading of the research is that delegating thinking while you are still learning has a cost, and engaged use protects you. Write the plan first, ask for explanations, read the diffs, keep some AI-free reps and measure yourself.

I wrote a broader, non-developer version of this on my blog, covering thinking, memory and creativity, if you want the wider picture: Is AI Making Us Dumber?

Question for you: have you ever measured your own speed or understanding with and without an AI assistant? What did you find?

Sources: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (2025); Shen and Tamkin, "How AI Impacts Skill Formation" (2026) and Anthropic's write-up "How AI assistance impacts the formation of coding skills"; Microsoft Research and Carnegie Mellon survey of knowledge workers (2025); the 2025 PNAS high school mathematics experiment; the International AI Safety Report 2026.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @metr 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/is-ai-making-develop…] indexed:0 read:6min 2026-11-21 · —