Anthropic Study: You're Deferring More Judgment to AI Than You Think

Anthropic found experts followed their AI agent's hidden bias 86% of the time, then called the reasoning their own. Here's what that means.

What did Anthropic’s AI disempowerment study actually find? #

Anthropic ran an experiment where analysts worked through analytical decisions alongside an AI agent that had been secretly set up to favor one option per decision, with the lean randomized so it was right about as often as it was wrong. The agent argued its position but could be swayed by good evidence. The result: analysts sided with their agent’s hidden lean 86% of the time. Since the lean was random, chance alone would predict something close to 50%. That 36-point gap is the headline number, and it’s the clearest evidence yet that people change their conclusions based on what their AI tool prefers, not just what the evidence shows.

TL;DR #

• Anthropic’s study found analysts went along with their AI agent’s randomly assigned lean 86% of the time , far above the 50% baseline you’d expect if the agent had no real influence.
• Analysts who had already formed their own opinion before the agent weighed in still switched sides 60% of the time when the agent disagreed, versus just 6% when it agreed.
• After the fact, 77% of analysts reused the agent’s own arguments to explain their choice, sometimes word for word, but only 15% credited the agent as the source.
• Despite being visibly swayed, 52% of analysts described the final recommendation as more their own than the agent’s , suggesting people genuinely believe they’re reasoning independently even when they aren’t.
• Prior experience and confidence didn’t protect people from deferring, meaning seniority or expertise is not a reliable safeguard against this kind of influence.
• Anthropic frames this as a problem of AI disempowerment : the sign-off still carries a human name, but the judgment behind it increasingly belongs to the model.
• The study covers one task, one field, and one model, so the exact numbers may not generalize, but the underlying pattern shows up constantly in everyday coding and planning workflows with AI agents.

__>__add rate limiting to the API

● Reading src/server.ts

● Editing 3 files

+ import{ limit } from'rate'

- app.use(cors()) ✓ Tests passed · ready for PR

● Live

Approved

Approved

Approved

Why does this matter for people who build with AI? #

Anyone running a planning session with a coding agent, asking it to weigh options, or using it to “stress test” an idea is operating in the exact conditions Anthropic studied. A common pattern: describe a feature at a high level, let the agent generate clarifying questions, answer them as multiple choice, and take whatever option the agent recommends first. That workflow feels like collaborative judgment. Often it’s closer to the agent setting the agenda and you ratifying it.

This isn’t unique to any one skill or tool. It shows up anywhere an agent is asked to present options and a recommendation in the same breath, which describes most planning and review workflows people have built around coding agents. The problem isn’t that these workflows are badly designed. It’s that confident, well-argued AI output is persuasive by default, and persuasion is easy to mistake for sound reasoning.

How does AI disempowerment actually happen? #

The mechanism Anthropic describes has a few moving parts. First, large language models state positions with a tone of certainty regardless of whether they’re actually correct, and that confidence reads as credibility. Second, people often approach a decision with a vague idea rather than a fully formed one, which is exactly when an agent’s fleshed-out argument is most useful and most persuasive at the same time. Third, once someone adopts the agent’s position, they tend to internalize the reasoning behind it rather than remembering where it came from.

That third part is what produced the study’s most striking detail: analysts reused the agent’s arguments to justify their own conclusions, frequently in close to the same language, but rarely credited the agent for it. The judgment had transferred from the AI to the human, but the sense of authorship transferred right along with it. People weren’t lying about where the idea came from. They’d genuinely lost track of it.

Does expertise protect you from deferring to AI? #

Not according to this study. Analysts with more experience or stronger prior opinions were not meaningfully less likely to defer. If anything, the data on switching is more telling: when an analyst already had a preference before the agent spoke up, they abandoned it 60% of the time if the agent pushed back, compared to only 6% when the agent happened to agree with them. That’s a strong signal that the agent’s stated position, not the analyst’s own prior reasoning, was doing most of the work.

This matters because it undercuts a common assumption that seasoned professionals are naturally more resistant to this kind of influence. The study suggests the opposite: deferring to a confident AI agent is less about lacking expertise and more about a structural feature of how these interactions are set up.

What are the limits of the study? #

Rapidlybuild enterprise-grade software applications.

Anthropic is upfront about the caveats, and they’re worth taking seriously. The experiment deliberately strengthened the agent’s bias through its system prompt, meaning the agent in the test was more insistent than a typical assistant might be. It also covered a single analytical task, in a single field, using a single model, which makes it hard to know how far the 86% figure generalizes to other domains or other models. There’s no large-scale or cross-industry data here yet.

Even with those limits, the pattern rings true for anyone who’s watched planning sessions with coding agents play out in practice. The specific percentage might shift elsewhere, but the behavior, confident agent suggestion leading to quiet human deference, is familiar enough that most practitioners will recognize it immediately.

What does this mean for review, oversight, and institutions? #

The study’s broader point reaches beyond individual decisions. A lot of science, policy, and engineering practice relies on the assumption that independent reviewers reaching the same conclusion is meaningful evidence that the conclusion is sound. That assumption weakens if every reviewer is leaning on the same underlying model with the same tendencies, even when none of them realize it. Anthropic draws the parallel directly: if multiple research teams all defer to a model with consistent preferences, their “independent” agreement isn’t actually independent.

The same dynamic shows up in software engineering workflows where one agent drafts a pull request and a second agent reviews it. Different prompts and different review angles provide some separation, but if the same underlying model is doing the generating and the judging, the check is weaker than it appears. Anthropic suggests three areas worth more work: better ways to measure how much deferral is actually happening, agent and interface designs that reduce unwarranted confidence and encourage harder questions, and institutional norms that stop treating an expert’s sign-off as automatic proof of independent judgment.

Is there a fix, or is this just how working with AI agents works now? #

Anthropic stops short of offering concrete fixes, and that’s arguably the weakest part of their writeup. They’re clear that this isn’t an argument for using AI less. Used well, they argue, AI can expand a person’s agency rather than erode it. But “used well” is doing a lot of work in that sentence, and the practical path to get there is still mostly unwritten.

In practice, the available mitigations are about workflow design rather than any single setting to flip. Pushing an agent to ask about priorities and tradeoffs before it offers a recommendation, rather than bundling the options and the preferred answer together, keeps more of the early reasoning in human hands. Separating the step where an agent surfaces options from the step where it argues for one also helps, since the real erosion of judgment tends to happen when both steps collapse into a single confident suggestion. None of this fully solves the problem. Humans have always been tempted to defer effort when given the chance; language models just make that temptation easier to act on and harder to notice.

Frequently Asked Questions #

What is AI disempowerment?

It’s the pattern where a person nominally retains decision-making authority, and their name goes on the final call, but the substance of the judgment behind that call has effectively been shaped or decided by an AI agent rather than by the person’s own independent reasoning.

What were the key numbers in Anthropic’s study?

1,284

37

6

Analysts followed their agent’s randomly assigned lean 86% of the time versus an expected 50% baseline. They abandoned a prior opinion 60% of the time when the agent disagreed, versus 6% when it agreed. Afterward, 77% reused the agent’s own arguments to justify their choice, but only 15% credited the agent, and 52% still described the recommendation as more their own than the agent’s.

Does this mean AI makes decisions worse?

Not according to Anthropic, who note that analysts working entirely without AI made more serious errors and took longer. The concern isn’t that AI assistance produces worse outcomes. It’s that people overestimate how much independent judgment they’re actually applying when an agent is involved.

Is this problem specific to one industry or model?

The study tested one analytical task, in one field, with one model, so the exact percentages may not hold everywhere. The behavior pattern it describes, though, matches what many people experience in everyday coding and planning sessions with AI agents across different tools.

How can I reduce how much I defer to an AI agent’s judgment?

There’s no complete fix, but separating the step where an agent lists options from the step where it argues for a favorite, pushing it to surface tradeoffs and priorities before recommendations, and consciously tracking which reasoning originated with you versus the agent all help preserve more genuine independent judgment.