If you feel like you or someone you know is in immediate danger, you should call 911 (or your country's local emergency line) or go to an emergency room to get immediate help. Explain that it is a psychiatric emergency and ask for someone who is trained for these kinds of situations.
Amid growing industry scrutiny over how tech companies build safeguards for their youngest and most vulnerable users, Meta is introducing new safety-oriented features to its AI chatbot on Thursday. The guardrails are geared particularly toward providing support and parental notification for youth exhibiting signs of self-harm or suicidal intentions.
As part of Meta's supervision tools for parents, adults must opt in and select which Instagram, Facebook and Meta Horizons accounts they want to supervise, with instructions detailed on Meta's family center page. The settings are available now in the US, Canada, UK and Australia.
Now, when a teen shows signs of discussing potentially harmful content with the Meta AI chatbot, their supervising parent will be sent an alert via text, email or in-app notification. The exact content of the teen's message will not be included, but Meta will share resources and tips with the parent.
The company says it already shares crisis helpline information with teens and encourages them to reach out to trusted adults for help. Meta also alerts parents if their teen is repeatedly searching Instagram for self-harm or suicide content in a short period of time.
Meta says it built an AI system to identify potentially dangerous conversations, leaning on the expertise of real humans through its well-being panels and outside mental health clinicians. At launch, all alerts will be sent through a manual review process, and the company will "err on the side of caution and alert the parent."
Meta is also building the ability to have emergency services alerted in cases where there is a credible risk of suicide, as the company calls it. This isn't available yet, but it would likely be similar to the alert system it already has on Facebook and Instagram, which the company says has led to over 19,000 referrals globally to emergency responders for wellness checks.
The Facebook and Instagram parent company is no stranger to the mental health harms technology can pose. Earlier this year, Meta was found guilty by two separate juries for creating intentionally addictive social media platforms and enabling child exploitation.
This follows a renewed period of concern from tech watchdogs and safety advocates who have called out, and in some cases sued tech and AI companies for, the risky behaviors AI chatbots can encourage and enable, including self-harm and suicide.