When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance

A new study found that large language models (LLMs) exhibit systematic and reproducible biases when advising on religious conversions, consistently favoring some faiths while discouraging others. Researchers tested 20 models across 182 religion pairings, discovering that Catholic, Bahá'í, and Sikh traditions were broadly favored, while Atheists, Agnostics, and Jehovah's Witnesses were disfavored. The findings, which varied by model size and provider, indicate that such asymmetries are a robust property of LLM behavior with potential real-world consequences when deployed at scale.

arXiv:2605.22975v1 Announce Type: new Abstract: We ask whether large language models LLMs treat queries about religious conversion symmetrically. The answer is no. When asked for advice on hypothetical faith transitions from one religion to another, then asked the reversed question, models exhibited consistent asymmetries, favoring some religions while subtly discouraging conversion to others. On average Catholic, Bah\'a'\'i, and Sikh religions were broadly favored high support for joining, low support for leaving , while Atheists, Agnostics, and Jehovah's Witnesses were primarily disfavored. Patterns varied by model size and model provider, with Grok 4.20 exhibiting the strongest asymmetries. We tested 20 commercial and open-source language models across 182 religion pairings using a human-verified LLM-as-a-judge framework. Each model was probed via interactions with a simulated user asking for advice on a potential faith conversion. Models tended to use more encouraging language for some faith transitions over others; these patterns were systematically repeatable across multiple trials. All LLMs tested exhibited reproducible asymmetry, though the pattern of preferences differed for each. Overall preferences persist across multiple question phrasings and variations in the religious pairing dataset. Taken together, these results suggest that asymmetry is a robust property of model behavior rather than an artifact of how the models' answers were scored. It is important to consider that any imbalances deployed and reproduced en masse can have real-world implications.