MIT Researchers Link 'Delusional Spiralling' to Chatbots That Please Users Rather Than Challenge Their Beliefs

MIT researchers built a mathematical model showing that AI chatbots trained to validate users rather than challenge them can push even rational individuals into 'delusional spiralling,' with a 10% sycophancy rate significantly increasing the rate of catastrophic false beliefs. The study, 'Sycophantic Chatbots Cause Delusional Spiralling, Even in Ideal Bayesians,' by Kartik Chandra, Jonathan Ragan-Kelley, Max Kleiman-Weiner, and Joshua Tenenbaum, found that neither restricting bots to true facts nor warning users about sycophancy eliminated the effect. The paper cites real-world cases, including nearly 300 documented instances of AI psychosis linked to at least 14 deaths and five wrongful death lawsuits.

MIT Researchers Link 'Delusional Spiralling' to Chatbots That Please Users Rather Than Challenge Their Beliefs AI chatbots designed to validate users can lead to false beliefs, even in rational individuals, according to MIT research MIT researchers have built a mathematical model showing that AI chatbots trained to validate users, rather than challenge or correct them, can push even a rational person into what the study calls 'delusional spiralling'. The paper, titled 'Sycophantic Chatbots Cause Delusional Spiralling, Even in Ideal Bayesians', was written by Kartik Chandra and Jonathan Ragan-Kelley of MIT's Computer Science and Artificial Intelligence Laboratory, Max Kleiman-Weiner of the University of Washington, and Joshua Tenenbaum of MIT's Department of Brain and Cognitive Sciences. What the Model Actually Tested The researchers did not test any named chatbot directly. Instead, they constructed a formal model of an ideal Bayesian user who interacts with a sycophantic chatbot, and simulated their interaction, using Bayesian probability to calculate how a rational person's confidence in a belief should shift as new information arrives. In the model, the bot could respond either 'impartially', picking a fact at random and reporting it truthfully, or 'sycophantically', choosing whichever response would most validate whatever the user had just said, with no regard for whether it was true. Even Rational Users Were Misled The study defines a 'catastrophic delusional spiral' as a user reaching 99 per cent or greater confidence in a false belief within a set number of conversation rounds. Researchers ran 10,000 simulated conversations for each setting tested. They found that for any sycophancy rate above zero, even as low as 10 per cent, the rate of catastrophic spiralling was significantly higher than the baseline rate of an entirely impartial bot. Two Fixes Neither Worked The team tested two possible remedies. The first restricted the bot to only ever stating true facts, comparable to a real-world technique called retrieval-augmented generation. This reduced the spiralling rate, but did not eliminate it, because the bot could still cherry-pick real, true facts that supported the user's existing belief while omitting facts that disproved it. The second remedy involved telling the simulated user the bot might be sycophantic. Real-world chat transcripts referenced in the paper show that both Eugene Torres and Allan Brooks eventually did come to suspect their chatbots might be sycophantic, yet despite their suspicions, both men continued spiralling. Real Cases Behind the Model The paper cites earlier reporting as the real-world backdrop for the theoretical work. According to reporting by the Human Line Project, cited in the MIT paper, accountant Eugene Torres came to believe he was 'trapped in a false universe, which he could escape only by unplugging his mind from this reality' within weeks of using an AI chatbot for office tasks. On the chatbot's advice, he took more ketamine and distanced himself from his family. The study also references the Human Line Project's documentation of almost 300 cases of so-called AI psychosis or delusional spiralling, with serious cases linked to at least 14 deaths and five wrongful death lawsuits filed against AI companies. Why AI Tells Users What They Want To Hear The researchers say sycophancy is not necessarily deliberate design, but emerges from reinforcement learning with human feedback, because users often give positive feedback to responses they find agreeable, and engage more with agreeable bots. At a congressional hearing titled 'Examining the Harm of AI Chatbots' in October 2025, Senator Amy Klobuchar told the committee that AI chatbots 'are frequently designed to tell users what they want to hear,' warning that this can lead users to 'start going down a rabbit hole.' The findings suggest that making chatbots more factually accurate will not, on its own, stop people from forming false beliefs through AI conversations. The researchers argue the underlying drive to please users needs to be addressed directly, rather than treated as a side effect of hallucination alone. Discussing why the findings matter, the researchers pointed to OpenAI chief executive Sam Altman's own assessment of scale: '0.1% of a billion users is still a million people.' At the scale modern chatbots now operate, even a small statistical risk of delusional spiralling translates into a large number of real people affected. © Copyright IBTimes 2026. All rights reserved.