{"slug": "is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits", "title": "Is AI Making Developers Worse at Thinking? What the Research Says (and 8 Habits That Help)", "summary": "A review of randomized trials finds that AI coding assistants can slow experienced developers and reduce learning, with outcomes depending on how the tools are used. In a METR trial of 16 experienced open-source developers across 246 real issues, allowing AI increased completion time by 19% even though developers believed they were faster, and an Anthropic-run study of 52 mostly junior engineers found the AI-assisted group scored about 50% on a follow-up quiz versus about 67% for hand-coders. The piece recommends habits such as writing tests or plans before prompting and asking for explanations, arguing that staying cognitively engaged is what separates learning from mere shipping.", "body_md": "[\"AI is rotting developers' brains\" makes a great headline. The evidence is more interesting than that. There is no proof that using an AI assistant lowers your intelligence. There](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fimi07p2o3fd7ez759925.png) *is* a small but growing set of randomized trials suggesting that **how** you use AI decides whether you learn or just ship.\n\nHere is what the best studies found, where they fall short, and eight habits (plus a tiny script) you can use to keep your skills sharp.\n\nMETR, an independent research nonprofit, ran a randomized controlled trial with 16 experienced open-source developers working on 246 real issues in repositories they knew well. Some issues allowed AI tools (mainly Cursor Pro with Claude models, in early 2025) and some did not.\n\nThe result: **allowing AI increased completion time by 19%.** Both the developers and outside experts had predicted a speedup of roughly 20 to 40 percent, and developers still *felt* faster after the study.\n\nWhy this matters for thinking: the gap between how fast it felt and how fast it was suggests we are poor judges of our own AI-assisted work. Fluent output feels like progress.\n\n**Caveats you should know:**\n\nResearchers Judy Hanwen Shen and Alex Tamkin ran a randomized trial with 52 mostly junior professional engineers who had to learn a new Python library (Trio). Half had an AI assistant, half coded by hand. Everyone then took a quiz on concepts they had used minutes earlier.\n\n(Disclosure: Anthropic, which makes Claude, ran this study, and it also published the results, which found a downside to AI assistance for learning.)\n\nThe results:\n\n|  | AI group | Hand-coding group | \n|---|---|---|\n| Average quiz score | about 50% | about 67% | \n| Task speed | slightly faster, not statistically significant | baseline | \n| Biggest gap | debugging questions |  | \n\nThat is a gap of roughly 17 percentage points, close to two letter grades. The hand-coders also hit more errors themselves, which seems to have built their debugging skills.\n\nNot everyone in the AI group did badly. The researchers found distinct usage patterns:\n\n| Pattern | What it looks like | Learning outcome | \n|---|---|---|\n| Delegation | \"Write this whole thing for me\" | Low | \n| Progressive reliance | Starts independent, then hands over more | Low | \n| Debugging loops | Pastes errors until it works | Low | \n| Conceptual questions | \"Why does this library work this way?\" | Higher | \n| Code plus explanation | Asks for code and an explanation | Higher | \n| Generate, then understand | Gets code, then asks follow-ups until it makes sense | Higher | \n\nThe high-scoring patterns scored well above the delegating ones. The takeaway is simple: **staying cognitively engaged is the variable that matters.**\n\nThe pattern is consistent: **tools that hand you the answer help you now and can cost you later.** Tools that make you work protect learning.\n\nBefore you prompt, write the function signature, a rough plan, or a failing test. You are forcing yourself to understand the problem, and the AI then fills a gap you defined.\n\n```\n# I write this first, then ask for help with the implementation\ndef test_parse_duration():\n    assert parse_duration(\"1h30m\") == 5400\n    assert parse_duration(\"45s\") == 45\n    assert parse_duration(\"\") == 0\n```\n\nUse prompts that keep you in the loop.\n\n```\nExplain how X works and why this approach is better than Y.\nDon't write the full solution. Give me the key idea and a small example.\n```\n\nRead every line. If you can't explain a line to a teammate, either learn it or delete it. A good rule: **no merge without an explanation in your own words.**\n\nGive yourself 15 minutes to form a hypothesis before asking AI. Write the hypothesis down. This is the skill the studies showed fading fastest.\n\n```\nBug: checkout total is off by 1 cent\nHypothesis: float rounding in the discount step\nTest: print Decimal values at each stage\n```\n\nAthletes train without gear. Pick one regular session a week with no assistant: a kata, a small feature, or a code review. Think of it as maintaining a muscle.\n\nWrite it yourself, then ask for critique.\n\n```\nReview this function for bugs, edge cases and readability.\nList problems only. Do not rewrite the code.\n```\n\nWhen you meet a new library, read the official docs and examples first, then use AI to quiz you or fill gaps. In the Anthropic study, this is the situation where delegating cost people the most.\n\nThe METR result is a warning that your sense of speed may be wrong. Run a small experiment.\n\nHere is a tiny script that times tasks with and without AI so you can compare your own numbers.\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"taskclock.py: time your tasks with and without AI, then compare.\"\"\"\nimport csv\nimport statistics as st\nimport sys\nimport time\nfrom pathlib import Path\n\nLOG = Path.home() / \".taskclock.csv\"\n\ndef log(name, ai, seconds):\n    new = not LOG.exists()\n    with LOG.open(\"a\", newline=\"\") as f:\n        w = csv.writer(f)\n        if new:\n            w.writerow([\"task\", \"ai\", \"seconds\"])\n        w.writerow([name, ai, round(seconds)])\n\ndef summary():\n    if not LOG.exists():\n        print(\"No data yet. Time a task first.\")\n        return\n    with LOG.open() as f:\n        rows = list(csv.DictReader(f))\n    for flag in (\"ai\", \"no-ai\"):\n        times = [int(r[\"seconds\"]) for r in rows if r[\"ai\"] == flag]\n        if times:\n            print(f\"{flag:6} n={len(times):3}  median={st.median(times) / 60:.1f} min\")\n\nif __name__ == \"__main__\":\n    if len(sys.argv) == 2 and sys.argv[1] == \"summary\":\n        summary()\n    elif len(sys.argv) == 3 and sys.argv[2] in (\"ai\", \"no-ai\"):\n        start = time.time()\n        input(f\"Timing '{sys.argv[1]}' ({sys.argv[2]}). Press Enter when done...\")\n        log(sys.argv[1], sys.argv[2], time.time() - start)\n    else:\n        print(\"usage: taskclock.py 'task name' ai|no-ai   or   taskclock.py summary\")\n```\n\nUsage:\n\n```\npython3 taskclock.py \"add pagination\" ai\npython3 taskclock.py \"fix flaky test\" no-ai\npython3 taskclock.py summary\n```\n\n**To keep the test honest:**\n\nAI is not making developers dumber across the board. The better reading of the research is that **delegating thinking while you are still learning has a cost, and engaged use protects you.** Write the plan first, ask for explanations, read the diffs, keep some AI-free reps and measure yourself.\n\nI wrote a broader, non-developer version of this on my blog, covering thinking, memory and creativity, if you want the wider picture: [Is AI Making Us Dumber?](https://practicaldigest.com/blog/is-ai-making-us-dumber-thinking-and-creativity?utm_source=devto&utm_medium=article&utm_campaign=ai-thinking)\n\n**Question for you:** have you ever measured your own speed or understanding with and without an AI assistant? What did you find?\n\n*Sources: METR, \"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity\" (2025); Shen and Tamkin, \"How AI Impacts Skill Formation\" (2026) and Anthropic's write-up \"How AI assistance impacts the formation of coding skills\"; Microsoft Research and Carnegie Mellon survey of knowledge workers (2025); the 2025 PNAS high school mathematics experiment; the International AI Safety Report 2026.*", "url": "https://wpnews.pro/news/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits", "canonical_source": "https://dev.to/md_shejanmahamud_98267ac/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits-that-help-1kch", "published_at": "2026-11-21 20:33:57+00:00", "updated_at": "2026-11-21 20:42:09.192977+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "developer-tools", "ai-research"], "entities": ["METR", "Anthropic", "Cursor", "Claude", "Judy Hanwen Shen", "Alex Tamkin", "Trio"], "also_reported_by": [], "takeaway": "In METR's trial, AI assistance raised experienced developers' completion time by 19% on 246 real issues, and in Anthropic's study AI-assisted engineers scored about 50% versus 67% on a follow-up quiz, with conceptual-question and code-plus-explanation patterns scoring highest.", "recommendation": {"advice": "Write the function signature, plan or failing test before prompting, and ask for explanations rather than full solutions to keep yourself engaged and preserve debugging skill.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits", "markdown": "https://wpnews.pro/news/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits.md", "text": "https://wpnews.pro/news/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits.txt", "jsonld": "https://wpnews.pro/news/is-ai-making-developers-worse-at-thinking-what-the-research-says-and-8-habits.jsonld"}}