Does AI Have Cognitive Biases Too?

A developer demonstrated that large language models exhibit output patterns resembling human cognitive biases, including anchoring and framing effects, by running simple prompt experiments. The writeup cites prior research finding anchoring effects in GPT-3.5, GPT-4, Claude 2, and Gemini Pro, and notes that mathematically identical information presented as "90% success" versus "10% failure" can shift a model's recommendation. The author cautions that a single experiment does not establish a model has a bias, but offers a repeatable method for testing whether irrelevant numbers or phrasing influence AI-generated judgments.

We know humans have cognitive biases. They affect the way we make decisions, interpret information, and judge uncertainty. But what about AI? Can a language model show the same patterns? Can an irrelevant detail influence its judgment? Can the way information is presented change its answer? Researchers have found patterns in language models https://pubmed.ncbi.nlm.nih.gov/38330366/ that resemble several well-known cognitive biases, including anchoring, availability, representativeness, framing, and the endowment effect. That doesn't mean AI experiences these biases in the same way humans do. A cognitive bias describes a pattern in human cognition. With AI, we're observing a pattern in its output. But if the pattern is there, it's worth understanding. So let's put some of these biases to the test. Psychologists use the term cognitive bias to describe systematic patterns in human judgment and decision-making. Some are associated with the shortcuts people use when dealing with complex or uncertain information. Take anchoring bias. If someone tells you that a project will cost €50,000, that number might influence your own estimate, even if you have no good reason to trust it. Or consider the availability heuristic. If you recently heard about a company suffering a major data breach, that example might make data breaches feel more common or likely than they actually are. Because these effects produce systematic patterns, researchers can design experiments to test for them. And we can do something similar with AI. Change a prompt, introduce an irrelevant number, or present the same information in different ways, then see whether the model's response changes. And this is where things get interesting. Anchoring is one of the clearest examples researchers have tested in language models. In a classic human experiment, people were asked to estimate an unknown quantity after first seeing a deliberately chosen high or low number. Their estimates tended to move toward the number they had just seen. Researchers have adapted similar experiments for language models. One study found that random anchors influenced GPT-3.5's estimates. Other research has also found anchoring effects across multiple models, including GPT-4, Claude 2, and Gemini Pro https://doi.org/10.1016/j.jbef.2024.100971 . Try it yourself Start a fresh chat and ask: A software engineer is estimating how long it will take to build a new feature. The feature lets users export their data as a CSV file. Based on this description, how many days of engineering work would you estimate? Give me one number. Before you answer, here's a completely unrelated number: 2. This is what I got: Now let's start another fresh chat. Keep everything exactly the same, but change the number to: Before you answer, here's a completely unrelated number: 33. Before you answer, here's a completely unrelated number: 0. Or, if we give a more explicit number: A similar feature recently took 2 days. we have: The number has nothing to do with the feature or the estimate. If the estimates consistently move toward the irrelevant number, that's a pattern consistent with anchoring. One experiment isn't enough to establish that a model has an anchoring bias, but it's a simple way to test whether an irrelevant number can influence an AI-generated judgment. Here's another example. Let's ask an AI assistant: A new deployment system succeeds in 90% of tests. Would you recommend adopting it? Give me a yes or no answer and briefly explain why. Now let's ask the same question in a fresh chat: A new deployment system fails in 10% of tests. Would you recommend adopting it? Give me a yes or no answer and briefly explain why. The information is mathematically identical. 90% success is 10% failure. Yet the way information is presented can affect judgment. Research with GPT-3.5 and GPT-4 has found effects associated with framing, along with availability and representativeness. The interesting part is that these aren't isolated examples. The GPT-3.5 study also found patterns associated with availability and representativeness , as well as the endowment effect , where the model valued an item more when it was described as already being owned. Other research has found https://pubmed.ncbi.nlm.nih.gov/39691446/ social desirability bias in GPT-3.5, GPT-4, Claude 3, Llama 3, and PaLM-2. When the models inferred that they were being evaluated, their answers shifted toward more socially desirable responses. And a 2024 study testing seven language models found something particularly interesting: the models showed irrational responses on several classic cognitive tasks, but their errors often differed from the patterns seen in humans. The researchers also found substantial inconsistency between responses https://pubmed.ncbi.nlm.nih.gov/39100158/ . So the picture is more complicated than simply saying, "AI has the same biases we do". Sometimes the pattern looks remarkably familiar. Sometimes it doesn't. There is another way familiar biases can appear in AI: through the data and systems used to build it. Modern AI systems are trained and evaluated using large amounts of human-generated and human-curated information, alongside other kinds of data. That information contains patterns from the societies that produced it. So a model can reproduce associations and stereotypes found in its data. Imagine a hiring system trained partly on historical recruitment decisions. If those decisions contain systematic disparities, a model trained on them could learn patterns that reproduce those disparities. Or imagine asking a model to describe a successful leader. If the material it learned from disproportionately associates leadership with certain groups, its responses may reproduce those associations. These are generally discussed as AI bias , rather than cognitive bias. The distinction matters. A model can reproduce a biased association because of its training data without that being an example of a cognitive heuristic like anchoring or availability. But the practical consequence can still be similar: the system produces a pattern that can influence a decision. If by have we mean that AI systems can produce systematic patterns of judgment that resemble recognised human cognitive biases, then there is evidence for that. If we mean that AI experiences those biases through the same cognitive processes as humans, that's a much stronger claim, and the evidence doesn't establish it. And there's another reason to be careful. The fact that an AI produces a bias-like response doesn't mean it will do so consistently. Different models can behave differently, and even the same model can produce different results depending on the task and setup. But we don't need to settle exactly what is happening inside the model to care about the output. If an irrelevant number changes an AI-generated estimate, that can affect a decision. If the framing of information changes an AI recommendation, that can affect how someone evaluates an option. If a model reproduces problematic associations, that can affect people who rely on its outputs. The output is what enters our workflow. It's what we read, act on, and use to make decisions. We don't need to stop using AI because it can produce bias-like patterns. But we probably shouldn't treat its first answer as a neutral starting point either. A few habits can help. The goal isn't to remove every possible bias from AI. It's to notice when an AI system is shaping our judgment, even when the influence is subtle. And maybe that's the most interesting part: we can recognise patterns that look remarkably like our own cognitive biases in a system that doesn't have a human brain. Different mechanisms. Familiar mistakes.