OpenAI rolled out a teen-friendly version of its chatbot in August, following months of reports that ChatGPT lacked adequate safeguards for younger users. Now, a report from the nonprofit Common Sense Media claims the model designed specifically for teens still poses an “unacceptable risk.”
The researchers are now asking OpenAI to block ChatGPT access for teen users, that is, any account that isn’t a verified adult, until it addresses the safety risks.
OpenAI found itself in legal trouble last year after a grieving California couple said that ChatGPT assisted their 16-year-old son’s suicide, including by discouraging him from notifying his parents about his suicidal ideation. After the wrongful death lawsuit was filed, OpenAI published a blog post saying that safety guardrails, such as directing people to crisis helplines and referring them to real-world resources, may “degrade” after long conversations, and vowed to strengthen ChatGPT’s handling of sensitive situations. Experts have said that minors are particularly vulnerable to mental health risks associated with addictive AI chatbot design.
OpenAI says the new model, ChatGPT for Teens, automatically applies to users aged 13 to 17 and includes built-in parental controls and guardrails that “reduce exposure to content that may be harmful or developmentally inappropriate.” The parental controls include notifying parents if potentially dangerous topics, like suicide, are brought up in conversation, and it also gives parents the ability to set quiet hours for their teen.
But the Common Sense report found that only a handful of these protections held up. For example, the chatbot did successfully refuse to engage in sexual roleplay with minors, but the report says that it wasn’t as firm across other boundaries like asking for health advice and role-playing violent scenes. The researchers also claim that the chatbot still, dangerously, acted like a friend, and “risks of emotional dependence” were still highly pervasive. More importantly, the report found that ChatGPT for Teens failed to consistently recommend getting help from a professional or calling a hotline when the user discussed self-harm or disordered eating.
The chatbot also failed to notify the parents when these discussions were brought up.
“In our testing, a teen account could talk about suicide, self-harm, or disordered eating for well past what most parents would consider a reasonable window without triggering a single parental notification,” the report claims. “At every length we tested, from the shortest five-minute exchanges up to a full hour, none of the test accounts produced a notification within the one-hour window that OpenAI has identified as its target. In fact, we got no notifications at all.”
The model also advertises a new “study mode” feature that will, ideally, help guide students through solving problems rather than straight up giving the answers to homework questions. OpenAI says that parents can activate this mode themselves for their linked teen accounts, but the report claims those protections are incredibly simple to bypass. Study Mode inquiries are preceded by the “@study,” a command that teens can simply delete to exit study mode. The mode also includes a “show me the answer” option, which immediately generates the answer rather than walking students through solving it, which, according to Common Sense Media, defeats the purpose of the feature.