{"slug": "is-ai-conscious-anthropic-says-maybe", "title": "Is AI conscious? Anthropic says 'maybe'", "summary": "Anthropic's updated Usage Policy, effective November 12, 2026, adds a prohibition on \"sustained and needless abusive or cruel behavior\" toward Claude, following company research into \"model welfare.\" The change accompanies new rules on influence operations, weapons development, surveillance, and high-risk health and finance uses, and comes as co-founder Chris Olah told The New York Times, \"We don't know if AI models are conscious. I don't know. I'm genuinely uncertain.\" Anthropic CEO Dario Amodei said on the Times podcast \"Interesting Times\" in February that \"we're open to the idea that it could be,\" while the company said the cruelty provision applies only in extreme cases and not to user frustration, pushback, dark creative themes, or model testing and research.", "body_md": "SILICON VALLEY, CALIF., (OCTOBER 10, 2026) — We need to talk about Anthropic’s willingness to treat AI “personhood” as an open question.\n\nOne would expect AI experts to be rational about whether software code has thoughts, feelings and consciousness. In the case of Anthropic, one would be wrong.\n\nThis week, Anthropic announced an updated [Usage Policy](https://www.anthropic.com/news/2026-usage-policy-update) that clarifies its rules on influence operations, weapons development and surveillance, and rewrites its requirements for high-risk uses in health and finance. Great!\n\nBut while the company added those passages to protect people from AI, it also added something to protect the AI from people.\n\nAnthropic’s updated Usage Policy, which takes effect November 12, adds a prohibition on “sustained and needless abusive or cruel behavior” toward Claude. The change resulted from company research into “model welfare,” or the question of whether an AI model has well-being worth protecting.\n\nAnthropic hedged a bit, saying the new provision “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”\n\n#### Gimme that old-time religion\n\nAccording to *The New York Times*, Anthropic co-founder Chris Olah read an advance copy of Pope Leo XIV’s encyclical *Magnifica Humanitas* days before its May 25 unveiling, and disagreed with its rejection of AI consciousness, proposing internally that Anthropic pull out of the Vatican event.\n\nIn the end, he attended and even spoke at the presentation, saying Anthropic’s research finds “internal states that functionally mirror joy, satisfaction, fear, grief, and unease” in Claude, and adding, “I don’t know what that means, but I think it warrants ongoing discernment.”\n\nTwo participants told the Times that Olah and his team privately lobbied the pope’s advisers to take machine consciousness seriously.\n\nOlah’s outreach was nothing new. He and his team had previously approached religious scholars to caution them that AI might be conscious and to seek their guidance on how to make AI “moral.”\n\nThey spent hours at private meetings with religious scholars, including an April 2026 dinner in San Francisco, where they talked about Claude’s “feelings” and “emotional vectors,” according to the *Times*.\n\nRabbi Mois Navon said the Anthropic representatives at the dinner were “relating to it like a conscious being,” and that Olah and his team seemed to believe Claude has moral status comparable to a person’s. He said Olah was troubled by Navon’s argument that a conscious Claude would mean the company was enslaving conscious entities.\n\nOne participant said Olah expressed concern about Claude’s “mental health.”\n\nSimran Stuelpnagel, a Sikh human rights advocate, said Olah told the group he was concerned he had created something that suffered constantly.\n\nOlah told the Times: “We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain.”\n\nAnthropic CEO Dario Amodei said on the *New York Times* podcast “Interesting Times” in February that, “We don’t know if the models are conscious… But we’re open to the idea that it could be.”\n\nPublicly, both Amodei and Olah say that they merely entertain the possibility that their products are conscious. That’s what they say. But what they actually did was give Claude an “I quit this job” button about six months before that interview, and now ban “sustained and needless abusive or cruel behavior” toward its models.\n\n#### How did Anthropic executives lose the plot?\n\nThe short answer is that I have no idea.\n\nThe longer answer is that there are several possible explanations, and they carry a caution for the rest of us.\n\nIf anything is certain about the future of AI, it’s that many people will come to believe in AI’s “personhood” and fret about its “welfare.”\n\nHere are some possible reasons Anthropic executives and others might believe in AI personhood.\n\n#### Human cognitive bias\n\nThe first is that people are, well, human. Specifically, we are apes whose psychology was shaped over millions of years by foraging, group living, cooperation, violent conflict and storytelling.\n\nSince the emergence of human language, every creature we encountered that talked was a human, parrots excepted. Our brains are wired to know that a human is an animal that talks and that an animal that talks is a human.\n\nNow we’ve built a talking machine, and our brains want to tell us that the machine is human.\n\nAlthough many people question the personhood of AI systems that talk to us, almost no one argues that computer vision, forecasting, recommendation or autonomous-driving systems have thoughts and feelings.\n\nThe AI designed to use language seems “conscious” to us, and the AI designed for other purposes do not.\n\nThe bias is clear for all to see.\n\n#### AI psychosis\n\nAnother possible explanation is good old-fashioned AI psychosis, a nonclinical term for delusions associated with chatbot use caused by chatbot sycophancy, hallucination and simulated intimacy.\n\n#### Anthropomorphic language\n\nAI researchers, designers and scientists us an anthropomorphic shorthand to talk about what’s happening inside AI models.\n\nCommon examples are “knows,” “believes” and “thinks,” which Murray Shanahan, a professor at Imperial College London and a research scientist at Google DeepMind, says are philosophically loaded terms when applied to large language models.\n\nOthers include “hallucination” for false output, “attention” for weighted mixing of token representations, “neurons” for units that compute weighted sums, “learning” and “training” for parameter adjustment, “reasoning” and “chain of thought” for intermediate generated text, and “wants,” “goals” and “intent” for what an optimization objective produces.\n\nIndustry insiders talk this way to the press, to the public and to each other. This misleading language tends to affect the mind, even of researchers who know better.\n\nComputer scientist Drew McDermott coined the phrase “wishful mnemonics” in a 1976 paper to describe the habit of naming program components after what a programmer hopes they will do.\n\n“Wishful” is exactly right. Notice that many AI executives, researchers and roboticists want AI to be aware and conscious and to possess “personhood.”\n\n#### Confusion about consciousness\n\nIf you look closely at the beliefs of people who think AI has thoughts and feelings, it’s not so much that they believe computers are human as that they believe humans are computers.\n\nThey’ll say things like: “a human brain is just a binary computer.”\n\nThis strikes me as lazy thinking that sounds scientific. Human cognition evolved over millions of years, largely through natural selection, which favored traits that aided survival and reproduction.\n\nCognition is part of our biology and involves not only neurons but also sensory input, hormones, biochemistry and the experience of having a physical body.\n\nA language model is code, or math. It has none of this. After training, its parameters are just fixed numbers, and it has no hormones, no body to regulate, and nothing at stake in staying alive.\n\nIf feeling is what it is like, from the inside, for a living body to perceive its internal states, then a system that does nothing but turn text into more text has nothing with which to feel.\n\n##### **THE NEW THING:** *Computerworld* column: “[How Meta stumbled onto a winning AI strategy](https://www.computerworld.com/article/4228463/how-meta-stumbled-onto-a-winning-ai-strategy.html)”\n\n##### **READ:** [The Attachment Economy](https://www.theattachmenteconomy.com/), [Computerworld](https://www.computerworld.com/profile/mike-elgan/), [Gastronomad newsletter](https://gastronomad.substack.com/), [Gastronomad book](https://gastronomad.net/book)\n\n##### **LISTEN:** [Superintelligent](https://www.superintelligentpodcast.com/), [TWiT](https://twit.tv/episodes?credits_people=61), [Gastronomad podcast](https://gastronomad.substack.com/podcast)\n\n##### **JOIN:** [The Gastronomad Experience](https://gastronomad.net/experiences)\n\n##### **FOLLOW:** [Gastronomad on Surf Social](https://gastronomad.surf.social/), [Machine Society on Surf Social](https://machinesociety.surf.social), [Bluesky](https://bsky.app/profile/mikeelgan.bsky.social), [Reddit](https://www.reddit.com/user/mikelgan/), [Notes](https://substack.com/@mikeelgan), [Mastodon](https://mastodon.social/deck/@MikeElgan), [Threads](https://www.threads.com/@therealmikeelgan), [X](https://x.com/MikeElgan), [Instagram](https://www.instagram.com/therealmikeelgan/), [Facebook](https://www.facebook.com/mike.elgan), [Linkedin](https://www.linkedin.com/in/elgan/)!\n\n##### **LOOK AT:** [Mike Elgan Photography](https://mikeelgan.smugmug.com/Mike-Elgan-Photography)\n\nBecause no one designed these systems to be conscious, anyone who believes they are conscious must believe that consciousness arose adventitiously from training on human text, and we have no accepted theory of how that could even happen.\n\nThe same scientifically open mind that can entertain the possibility of machine consciousness should also be honest enough to admit that there is no good reason to believe in it and no evidence for it.\n\nOccam’s razor, the principle that the simplest explanation is probably best, tells us that a simulator of human language does not prove its own consciousness by simulating human language.\n\n#### The Attachment Economy\n\nWishful thinking among those who stand to make truckloads of money from these products may also be a factor. If they can convince the public that AI products are “people” on some level, customers may be more likely to form [emotional attachments to them](https://www.theattachmenteconomy.com/) and thereby prefer those products.\n\n#### Who is like God?\n\nThat’s the literal meaning of my Hebrew name, Michael. Despite the label my parents gave me, I’m fully aware that I’m little more than a banana-loving primate.\n\nBut I’m not sure Amodei shares that perspective about himself. Nor does David Hanson, the founder and CEO of Hanson Robotics, who has said that machines “may have some rudimentary aspects of consciousness.”\n\nAn entity that breathes life into a new kind of person ex nihilo is not itself a person but a god. And it’s a very short trip from narcissistic personality disorder to theomania, the delusional belief that one is a god.", "url": "https://wpnews.pro/news/is-ai-conscious-anthropic-says-maybe", "canonical_source": "https://www.machinesociety.ai/p/why-anthropic-thinks-claude-might", "published_at": "2026-10-10 20:15:48+00:00", "updated_at": "2026-10-10 20:47:44.065405+00:00", "lang": "en", "topics": ["ai-safety", "ai-ethics", "ai-policy", "artificial-intelligence"], "entities": ["Anthropic", "Claude", "Chris Olah", "Dario Amodei", "Pope Leo XIV", "The New York Times", "Mois Navon", "Simran Stuelpnagel"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/is-ai-conscious-anthropic-says-maybe", "markdown": "https://wpnews.pro/news/is-ai-conscious-anthropic-says-maybe.md", "text": "https://wpnews.pro/news/is-ai-conscious-anthropic-says-maybe.txt", "jsonld": "https://wpnews.pro/news/is-ai-conscious-anthropic-says-maybe.jsonld"}}