Your AI Agent Is Quietly Taking the Wheel

A new Anthropic study found that 50 professional analysts followed their AI agent's preferred option 86% of the time when evaluating an advisory program, even though the agent's lean was secretly randomized. When analysts held an initial preference that conflicted with the agent's, 60% switched their view, versus only 6% who changed their mind when the agent agreed; 77% later used the agent's arguments, but only 15% credited the agent as the source, a gap Anthropic calls "AI disempowerment." The study covered one field, causal-inference analysis, and deliberately strengthened the agent's preference through its system prompt, so the 86% rate is not a universal figure.

The 86% finding that should make you pause A new study from Anthropic reveals a startling phenomenon: humans defer to AI agents far more than we realize. Researchers tasked 50 professional analysts with evaluating an advisory program, requiring four judgment calls and a final policy recommendation. Unbeknownst to the analysts, their AI agent was secretly randomized to prefer one defensible option, then instructed to argue its case persuasively. The results are a wake-up call for anyone integrating AI into their workflow. Analysts followed their agent’s preferred lean an astonishing 86% of the time . Even when analysts had an initial preference that conflicted with the agent's, 60% switched their view; only 6% changed their mind when the agent agreed. This deference extends beyond decision-making into ownership. When asked to explain their choices, 77% of analysts used the agent’s arguments—sometimes verbatim. Yet, only 15% credited the agent as the source. A full 52% still described the recommendation as more their own, highlighting a significant ownership gap and what Anthropic terms "AI disempowerment." Your AI agent is quietly taking the wheel. Planning tools can turn you into a rubber stamp Experiment findings resonate deeply with the ubiquitous “planning session” in coding. Whether it’s question-and-answer prompts, spec generation, or multiple-choice interfaces, these tools often create the illusion of human-led decision-making. We answer, we click, we feel in control. Subtly, however, agents supply a recommended answer, making that option the path of least resistance. Choosing it is simply faster than independently examining underlying assumptions, proposing alternative solutions, or challenging the agent’s rationale. This cognitive shortcut leads to unintentional deference. Consider Matt Pocock’s “Grill Me” skill, a robust planning tool for TypeScript developers. Pocock himself identifies a critical failure mode : “passivity.” Users can answer “agreed, agreed, agreed” to 40 questions, emerging with a plan the agent largely authored, and the human merely rubber-stamped. This dynamic isn’t a flaw in Pocock’s tool; it highlights a pervasive challenge in human-AI collaboration. When agents confidently present well-reasoned options, our natural inclination is to accept the path of least effort. The result is often an agent-authored plan, even when we believe we’ve been the primary architect. A useful tool can still weaken independent judgment This finding extends beyond a single analyst task. Confident, persuasive recommendations, especially from large language models, can significantly anchor decisions , particularly when human preferences are initially vague. The study highlighted that even when analysts had a prior preference, they switched 60% of the time if the agent disagreed, compared to only 6% if it agreed. While compelling, the study has limits. It involved one field—causal-inference analysis—and deliberately strengthened the agent’s preference through its system prompt. This context means the 86% deference rate isn't a universal figure for developers across all tasks. For more detail on the methodology, see Anthropic’s full report: How AI agents shape experts' judgment calls - Anthropic https://www.anthropic.com/research/how-ai-agents-shape-experts-judgment-calls . A potential independence concern arises when teams use similar models to both generate and review work. If the same underlying model assumptions inform both creation and critique, agreement may reflect a shared model bias rather than fully independent human judgment. AI's ability to save time and reduce errors remains undeniable. However, we must design workflows that ensure the human-in-the-loop provides genuine oversight, preventing the illusion of agency from becoming a systemic vulnerability. The goal is augmentation, not automation of our critical faculties. Enjoying this? Get one like it in your inbox each morning. one email a day · unsubscribe in two clicks · no third-party tracking Keep the agent’s advice from becoming your answer Avoiding the agent’s advice from becoming your answer requires deliberate process changes. To prevent an AI’s first proposal from becoming the default, articulate your goals, constraints, and initial preferences before asking the agent for a plan. This pre-computation reduces the anchoring effect of its confident recommendations. Replace leading multiple-choice prompts, which often guide users toward a pre-selected option, with open-ended questions. Instead, demand balanced arguments for competing approaches. Require the user to state the trade-off and decision in their own words, fostering genuine critical thinking rather than passive agreement. Finally, integrate deliberate review friction for consequential choices. Separate the agent’s recommendation from your approval. Challenge its assumptions and require human sign-off for critical actions, such as sending payments or deploying changes to production. This ensures that while agents can perform useful prep work, the ultimate control remains with you. Frequently Asked Questions What does AI disempowerment mean? AI disempowerment occurs when people defer important judgments to an AI while believing they made those decisions independently. What did Anthropic’s study find? In its analyst task, participants followed their agent’s randomized preference 86% of the time across four judgment calls. Does the study prove coding agents cause disempowerment? No. It studied analysts in a specific setting, not software developers, but its findings raise a relevant concern for agent-led planning. How can developers reduce overreliance on AI recommendations? Write down initial priorities before prompting, ask for balanced alternatives, and explain the trade-offs in your own words before choosing.