{"slug": "new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations", "title": "New study finds startling flaw if AI starts choosing who gets organ donations: ‘Diverge from human values’", "summary": "A Penn State study presented at the 2026 Association for Computing Machinery Fairness, Accountability and Transparency (FAccT) conference found that leading large language models diverge from human values when asked to allocate a single kidney among multiple transplant patients, fixating on single factors such as drinking habits and almost never choosing a \"flip a coin\" option. Study lead Hadi Hosseini said the models \"overwhelmingly commit to a confident, deterministic answer\" even when given the option to express indecision, a gap he called meaningful because real moral dilemmas often lack one clearly correct answer. The researchers concluded that the gap between human indecision and chatbot confidence in moral dilemmas needs continued research and governance, with Hosseini stating that \"getting AI's role in them right isn't optional.", "body_md": "# New study finds startling flaw if AI starts choosing who gets organ donations: ‘Diverge from human values’\n\nSee more of our coverage in your search results.\n\n[Add The New York Post on Google](https://www.google.com/preferences/source?q=nypost.com)\n\nA prescription for disaster.\n\nWhen it comes to AI chatbots calling the shots for medical decisions, there seems to be a divide between whether the [promising outcomes](https://nypost.com/2026/06/22/health/18-kids-undiagnosable-diseases-solved-by-ai-total-game-changer/) outweigh the [medical mishaps](https://nypost.com/2025/10/24/health/real-life-ways-bad-advice-from-ai-is-sending-people-to-the-er/).\n\nA [study by Penn State](https://www.psu.edu/news/information-sciences-and-technology/story/who-gets-kidney-ai-chatbots-do-not-make-same-moral) researchers has found that artificial intelligence models prioritize vastly different values than humans when it comes to organ donation decisions, raising questions about the use of the technology in ethically sensitive situations.\n\nPresented at the [2026 Association for Computing Machinery Fairness, Accountability and Transparency (FAccT) conference](https://facctconference.org/2026/acceptedpapers.html), the study was used to determine the technology’s moral decision-making in a high-stakes situation.\n\nTo examine the challenge of scarce resources, researchers asked several leading large language models a hypothetical question: If there are multiple kidney transplant patients but only one available organ, who should receive the kidney?\n\nResearchers then compared the chatbot responses with those given by people with no specified medical training in earlier academic studies to see where the AI models aligned or misaligned with human values.\n\nThe AI chatbots’ decisions differed from the humans’ in that they simplified complex decisions and expressed unwarranted confidence in a situation where there is no one right answer.\n\nThe researchers set up numerous comparisons between two fictional patients, both in need of the same kidney, and described by a series of attributes, such as age, health status, drinking habits and number of dependents.\n\n“We ran these comparisons in a few different ways,” study lead Hadi Hosseini said.\n\n“Sometimes we isolated just one trait at a time, sometimes we mixed several traits together to see how AI weighed competing factors and sometimes we added a flip-a-coin option to measure indecision, a key factor present in human moral judgment.”\n\nUsing existing data from human studies on kidney allocation, the researchers were able to compare previous human decisions with what AI chose, with two findings standing out to the team.\n\n“First, AI chatbots often diverge from human values in how they weigh a patient’s traits,” Hosseini said. “They fixate on a single factor, like drinking habits, rather than balancing multiple considerations the way people do.”\n\nAnd second, AI didn’t struggle with indecision, confidently picking an answer, despite humans seeing that there may not be a clear and correct response.\n\nPer Hosseini, people struggle with indecision because they don’t want to accept agency.\n\n“AI models almost never do this: Even when directly given the option to ‘flip a coin,’ they overwhelmingly commit to a confident, deterministic answer instead,” he added. “That’s a meaningful gap, since real moral dilemmas often don’t have one clearly correct answer.”\n\nWhile the researchers don’t intend to encourage using AI anytime soon as a substitute for professional judgement, they recognize the growing presence of these chatbots.\n\n“Moral decisions in settings like organ allocation directly determine who lives and who dies, so getting AI’s role in them right isn’t optional,” Hosseini said.\n\nDespite the rapid rise in use of AI in medical fields, the researchers behind the study believe the gap between human indecision and chatbots’ confident answers when facing moral dilemmas needs continued research and governance.\n\nEven John Dickerson, chief executive officer at Mozilla.ai and collaborator on the study, acknowledges the dangers of AI’s inability to recognize the impact of ethical decisions in medicine and elsewhere.\n\n“When we allocate something scarce, whether it’s a kidney, a job or access to some other resource, there isn’t always a single objectively correct answer,” Dickerson said.\n\n“Humans recognize that ambiguity and codify it via open debate into the allocative process. AI models often don’t.”", "url": "https://wpnews.pro/news/new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations", "canonical_source": "https://nypost.com/2026/09/21/health/new-study-finds-startling-flaw-if-ai-chooses-who-gets-organ-donations/", "published_at": "2026-09-21 19:10:26+00:00", "updated_at": "2026-09-21 19:26:10.507475+00:00", "lang": "en", "topics": ["ai-ethics", "ai-safety", "large-language-models", "artificial-intelligence"], "entities": ["Penn State", "Hadi Hosseini", "Association for Computing Machinery Fairness, Accountability and Transparency (FAccT) conference", "John Dickerson", "New York Post"], "alternates": {"html": "https://wpnews.pro/news/new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations", "markdown": "https://wpnews.pro/news/new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations.md", "text": "https://wpnews.pro/news/new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations.txt", "jsonld": "https://wpnews.pro/news/new-study-finds-startling-flaw-if-ai-starts-choosing-who-gets-organ-donations.jsonld"}}