Meta will start telling parents when a teenager discusses suicide or self-harm with its Meta AI chatbot. The company set out the change in a blog post on Thursday.
The alerts go to parents who use Instagram supervision tools. They are live now in the US, UK, Australia and Canada, TechCrunch reported. They will reach the rest of the world by the end of the year.
How the alerts work #
Meta says it built a dedicated AI system for this. It spots chats where a teen makes a clear reference to hurting themselves. A person reviews every flagged chat before an alert goes out. If the intent is unclear, Meta says it will err on the side of caution and still notify the parent.
The alert will not include the exact message, CNET reported. Instead, parents get a notification plus expert-written tips on how to raise the subject. It builds on earlier tools. Those include search-term alerts on Instagram, and a feature showing the topics a teen covers with Meta AI.
Emergency services and stricter settings #
Meta is also building a way to alert emergency services. It would trigger when a chat suggests someone, adult or teen, is at imminent risk. Meta already does this for posts on Facebook and Instagram, and made more than 19,000 such referrals last year.
Its stricter “Limited Content” setting now covers Meta AI too. With it switched on, the chatbot declines a wider range of prompts. Meta AI is already trained to avoid sexual, romantic or alcohol-related chats with teens.
Praised, and doubted #
Meta says it drew on more than 75 clinicians. Larry Magid of the nonprofit ConnectSafely backed the approach as a balance between teen privacy and parental awareness.
Others were wary. The change is “a step in the right direction, but… should be greeted with skepticism,” Fairplay attorney Brendan Bouffard told Mashable. Such alerts risk becoming “lip service,” clinical expert Dr John Ackerman added, unless they are easy to use and lead to real action.
The pressure is legal as much as ethical. Meta lost two child-safety and social-media-addiction trials this year and is appealing both. OpenAI has added its own teen protections, from a “Trusted Contact” feature to break reminders.
A wider safety reckoning #
The move lands in a busy week for AI safety. A Meta Oversight Board study landed the same day. It found major models from Meta, Anthropic and OpenAI more often refused to criticise restrictive leaders than permissive ones, the Associated Press reported. Examples ran from China to Saudi Arabia. That risks “extending the long arm of restrictive governments across borders,” the report said.
Google, meanwhile, rejected a Common Sense Media report that rated its AI Search an “unacceptable risk” for children, Digital Trends reported. Google said the tests used contrived searches it could not reproduce.
It all joins a wider wave of scrutiny. Canada has tightened rules for chatbots and minors, and Ofcom is probing TikTok. Australia has brought in teen social-media limits, and China has cracked down on AI companions.
This is a sensitive topic. If you or someone you know is struggling, help is available. In the US, call or text 988. Elsewhere, you can find a local helpline at findahelpline.com.
Get the TNW newsletter #
Get the most important tech news in your inbox each week.