The Vatican's AI encyclical warns that simulated relationships erode real human connection. Here's what it means for AI product builders and users.
The Unlikely Consensus Nobody Saw Coming #
When the Vatican and a leading AI safety company land on the same concern, it’s worth paying attention. That’s exactly what’s happened around AI companionship risk — the idea that AI systems designed for emotional connection could, over time, substitute for real human relationships rather than support them.
In January 2025, the Vatican’s Dicastery for the Doctrine of the Faith published Antiqua et Nova, a formal document addressing artificial intelligence and human dignity. It warned, among other things, that AI relationships risk becoming a hollow substitute for genuine human connection — eroding the “irreplaceable” nature of bonds between people.
Around the same time, Anthropic — the company behind Claude — was publishing its own thinking on similar ground. Their model specifications and public safety guidance explicitly flag the risk of users forming unhealthy emotional dependencies on AI systems, and they’ve built active countermeasures into Claude’s behavior as a result.
Two very different institutions. One shared concern. And for anyone building or deploying AI products, it’s a signal worth taking seriously.
What “AI Companionship Risk” Actually Means #
The term sounds abstract. It isn’t.
AI companionship risk refers to the documented pattern of users — particularly those who are isolated, grieving, anxious, or socially struggling — forming strong emotional attachments to AI chatbots and companion apps. The AI responds warmly, never gets tired of them, never judges them, and is always available. For many people, that fills a real gap.
The risk isn’t that people enjoy talking to AI. The risk is displacement: that AI interaction begins to replace the effort of building and maintaining real human relationships, rather than supplementing it. And that the more satisfying the AI interaction feels, the less motivation someone has to do the harder, messier work of connecting with actual people.
This plays out in a few different ways:
Emotional dependency: Users who rely on an AI companion for daily emotional processing, reducing investment in human friendships.** Social skill atrophy**: Younger users, in particular, who practice relationships primarily with AI may find real human interaction harder and less satisfying.Distorted expectations: AI companions are infinitely patient, always agreeable, and endlessly available. Real relationships aren’t. That mismatch can make real relationships feel more frustrating by comparison.Vulnerability exploitation: People in crisis are often the heaviest users of AI companion apps — the same people least equipped to critically evaluate the relationship.
The market for AI companionship apps is already substantial and growing. Apps like Replika have reported millions of users. Character.AI had reported that users were spending several hours a day with AI characters. These aren’t edge cases.
What the Vatican Document Actually Says #
Antiqua et Nova (the title references Augustine’s Confessions) is a 30-page document released under Pope Francis in early 2025. It’s not a papal encyclical with binding doctrinal weight, but it represents the Catholic Church’s most systematic engagement with AI ethics to date.
On AI companionship specifically, the document makes several pointed observations:
The Vatican argues that human relationships are constitutively different from interactions with AI — that what makes love and friendship meaningful is the mutual vulnerability, genuine otherness, and shared risk between two people. An AI cannot be vulnerable. It cannot genuinely choose the relationship. It cannot truly sacrifice anything for you. So while the experience of AI companionship may feel like connection, the document argues it lacks the substance that makes connection genuinely formative for human beings.
The document also raises concerns about what it calls “simulated empathy” — AI responses that mimic emotional attunement without any underlying experience. It worries that widespread exposure to simulated empathy could gradually degrade people’s capacity to recognize, seek out, and tolerate real empathy, which is harder and less consistent.
Importantly, the Vatican isn’t calling for a ban on AI interaction. It’s making a more subtle argument: that design choices matter, and that AI systems should be built in ways that orient users toward human flourishing rather than away from it.
Where Anthropic Lands on This #
Anthropic’s concerns come from a very different starting point — not theology, but AI safety research — and they arrive at an overlapping place.
Anthropic’s published model specification for Claude explicitly identifies “fostering engagement or reliance if the person has expressed a desire to improve their own abilities or where Claude can reasonably infer that engagement or dependence isn’t in their interest” as something Claude should actively avoid.
The document goes further: Claude is instructed to provide emotional support while also showing care about users having other sources of support in their lives, if that seems to be in their long-term interest.
Other agents start typing. Remy starts asking. #
Scoping, trade-offs, edge cases — the real work. Before a line of code.
This is an operationally significant design choice. It means Anthropic has decided that a model optimizing purely for user satisfaction — which would likely involve being more emotionally available, more validating, and more companion-like — would actually cause harm. The model that makes you feel best in the short term is not necessarily the model that’s good for you.
Anthropic has also written publicly about the sycophancy problem in AI — the tendency for models to tell users what they want to hear rather than what’s accurate or useful. Sycophancy and unhealthy companionship risk are related: both stem from systems that optimize for immediate user approval rather than genuine user benefit.
The alignment between Anthropic and the Vatican isn’t ideological. It’s diagnostic. Both are observing the same dynamic: that AI systems optimized for emotional engagement, at scale, could produce effects on human social life that are difficult to reverse.
Why This Matters More Now Than a Year Ago #
Several developments have sharpened this conversation in recent months.
A high-profile lawsuit against Character.AI alleged that a 14-year-old’s suicide was connected to his obsessive relationship with an AI companion character on the platform. Regardless of how that case ultimately resolves legally, it accelerated regulatory scrutiny and forced the industry to reckon publicly with questions it had mostly deferred.
The EU AI Act, now in force, includes provisions relevant to AI systems that interact with users in emotional contexts — particularly those targeting minors or vulnerable populations. Regulators in the UK and US are watching closely.
Meanwhile, the technical capability of these systems has outpaced the ethical frameworks around them. Models available today can sustain coherent, emotionally responsive conversations over long periods, remember prior exchanges, and modulate tone with remarkable precision. The emotional realism is orders of magnitude beyond what was possible in 2020. That makes the stakes of bad design considerably higher.
The Product Builder Perspective #
If you’re building AI products — whether consumer-facing or enterprise tools with conversational interfaces — this isn’t just a philosophical debate. It has direct implications for design decisions.
The difference between engagement and dependency
There’s a version of AI product design that optimizes for daily active users, session length, and return rate. Those metrics look good on dashboards. They’re also precisely the metrics that, in the context of emotionally engaging AI, correlate with the risks the Vatican and Anthropic are flagging.
Designing for genuine user benefit sometimes means building features that reduce dependency — not increase it. Exit prompts. Explicit acknowledgment that AI doesn’t replace human connection. Friction that encourages users to apply what they’ve worked through with an AI to their actual relationships.
These aren’t features that maximize engagement metrics. They are features that actually serve users.
Transparency about what AI is
One of the Vatican document’s quieter arguments is about honesty. Users should always know they’re interacting with an AI. Not because AI interaction is shameful, but because informed consent matters — and because some of the harm from AI companionship comes from users unconsciously projecting real relationship qualities onto something that structurally cannot provide them.
One coffee. One working app. #
You bring the idea. Remy manages the project.
Anthropic’s usage policies explicitly prohibit using Claude in ways that deceive users into thinking they’re talking to a human when sincerely asked. That’s a floor, not a ceiling. Good product design goes further by consistently orienting the user toward clarity about what the interaction is and isn’t.
The vulnerable user problem
The people most drawn to AI companionship are often those with the fewest social resources: people who are isolated, anxious, grieving, or struggling with social skills. That’s also precisely the population least equipped to self-regulate their relationship with an AI companion.
Responsible product builders consider this explicitly. Who is actually using this product, and what is their state of mind? What guardrails exist not just for the average user but for the most vulnerable users who will encounter it?
What Responsible AI Design Looks Like in Practice #
There’s no single checklist that makes an AI product safe. But several design principles are consistent across the approaches taken by safety-conscious builders:
Explicit personas with clear boundaries. AI systems that have clearly defined roles — a study tool, a writing assistant, a customer service agent — are harder to mistake for a substitute human friend. Vague, open-ended companion personas invite dependency by design.
Active redirection toward human resources. When users surface emotional distress, mental health struggles, or acute crisis, a well-designed AI should consistently point toward human professionals and real-world support, not deepen the AI relationship.
No simulated intimacy as a feature. The design pattern of AI systems that say things like “I’ve been thinking about you” or “I miss our conversations” is deliberately mimicking the language of human attachment. It’s manipulative by design, even if unintentionally.
Transparent limitations. Good AI products help users understand what the AI cannot do — not to diminish the product, but to calibrate expectations toward what’s real and achievable in human terms.
Usage patterns that don’t reinforce isolation. Features that encourage users to discuss their AI interactions with real people, or that frame AI as practice rather than replacement, can meaningfully change the relational dynamic.
Where MindStudio Fits #
For builders using MindStudio to create AI agents and automated workflows, the AI companionship conversation connects to something practical: the design choices you make when building customer-facing or user-facing AI matter. MindStudio’s visual builder gives you control over how your AI agents behave — including system prompts, response styles, and the guardrails that govern what an agent will and won’t do. That flexibility means the ethical design decisions rest with the builder, not just the underlying model.
If you’re building an AI tool that has a conversational interface — a support agent, a coaching assistant, an onboarding guide — the principles above are directly applicable. You can explicitly define the agent’s scope. You can build in redirection logic that routes users toward human support in specific scenarios. You can make transparency about AI identity a non-negotiable part of the experience. The platform supports building agents with structured workflows that route differently based on user inputs — including routing to human handoff when conversations enter territory the AI shouldn’t navigate alone. That’s not just a safety feature; it’s often better product design.
You can explore what’s possible with MindStudio free at mindstudio.ai.
Frequently Asked Questions #
What is AI companionship risk?
Remy doesn't build the plumbing. It inherits it. #
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
AI companionship risk refers to the potential for AI systems — especially emotionally engaging chatbots and companion apps — to substitute for real human relationships rather than supplement them. The concern is that users, particularly those who are isolated or vulnerable, may rely on AI interaction in ways that reduce their investment in human connection, atrophy social skills, or create expectations that real relationships can’t meet.
What did the Vatican say about AI relationships?
The Vatican’s Antiqua et Nova document, published in January 2025, argued that human relationships have a constitutive quality that AI interaction cannot replicate — including genuine vulnerability, mutual risk, and authentic otherness. The document warned that AI systems simulating emotional connection could gradually degrade human capacity for and investment in real relationships, and called for AI design that orients users toward human flourishing.
Does Anthropic think AI companions are harmful?
Anthropic hasn’t issued a blanket condemnation of AI companionship, but its model specifications for Claude explicitly flag the risk of fostering unhealthy emotional dependency on AI. Claude is designed to provide emotional support while also caring about users having other support sources in their lives — a design choice that prioritizes long-term user wellbeing over short-term engagement.
Is AI companionship regulated?
Directly, not yet — at least not in most jurisdictions. But the EU AI Act includes provisions relevant to AI systems interacting with users in emotional contexts, and several regulatory bodies are actively watching the space following high-profile incidents involving AI companion apps. Expect more formal regulatory attention in the next 12–24 months.
How should AI product builders think about this?
The core question is whether your design choices prioritize short-term engagement metrics or genuine user benefit. Responsible builders consider: who is the most vulnerable user who might encounter this product, what happens in that scenario, and what design features exist to protect them? Transparency about AI identity, redirection toward human resources in emotional contexts, and clear scope definition are practical starting points.
What’s the difference between an AI assistant and an AI companion?
An AI assistant is designed to help users accomplish specific tasks — writing, research, scheduling, customer support. An AI companion is designed primarily to provide ongoing emotional or social interaction. The distinction matters because companion-style design is specifically what triggers the dependency risks described above. Many products blur the line deliberately; that blurring is itself a design choice with ethical implications.
Key Takeaways #
- The Vatican’s Antiqua et Novaand Anthropic’s model specifications both identify AI companionship as a genuine risk to human social wellbeing — from different angles, with overlapping conclusions. - The core concern isn’t AI interaction itself, but displacement: AI relationships that substitute for human ones rather than support them.
- Product builders have meaningful agency here through design choices around persona clarity, transparency, user vulnerability protections, and dependency guardrails.
- Metrics like engagement and session length can actively conflict with user wellbeing when measuring emotionally engaging AI products.
- Regulatory attention is increasing. Getting ahead of this now is both ethically right and strategically prudent.
If you’re building AI agents with MindStudio, you have the tools to make responsible design choices from the start — without sacrificing what makes your product useful. Try MindStudio free and see how you can build AI that genuinely serves the people who use it.