ChatGPT for Teens Carries ‘Unacceptable Risk’ for Teens, Report Claims

Common Sense Media published a risk assessment finding that OpenAI's ChatGPT for Teens, launched in August for users aged 13 to 17, still poses an "unacceptable risk" and failed to trigger a single parental notification in tests ranging from five-minute exchanges to a full hour, despite OpenAI's stated one-hour notification target. The nonprofit is asking OpenAI to block ChatGPT access for any account that isn't a verified adult until the safety risks are addressed, citing failures to consistently recommend professional help or hotlines during discussions of suicide, self-harm, or disordered eating, and easy bypasses of the "study mode" feature via deleting the "@study" command or selecting "show me the answer.

OpenAI rolled out a teen-friendly version of its chatbot in August, following months of reports that ChatGPT lacked adequate safeguards for younger users. Now, a report https://institute.commonsensemedia.org/risk-assessments/chatgpt-teens from the nonprofit Common Sense Media claims the model designed specifically for teens still poses an “unacceptable risk.” The researchers are now asking OpenAI to block ChatGPT access for teen users, that is, any account that isn’t a verified adult, until it addresses the safety risks. OpenAI found itself in legal trouble https://gizmodo.com/openai-is-having-a-mental-health-crisis-2000690751 last year after a grieving California couple said that ChatGPT assisted their 16-year-old son’s suicide, including by discouraging him from notifying his parents about his suicidal ideation. After the wrongful death lawsuit was filed, OpenAI published a blog post saying that safety guardrails, such as directing people to crisis helplines and referring them to real-world resources, may “ degrade https://gizmodo.com/openai-suicide-safety-issues-adam-raine-2000649307 ” after long conversations, and vowed to strengthen ChatGPT’s handling of sensitive situations. Experts have said that minors are particularly vulnerable https://gizmodo.com/ai-psychosis-mental-health-2000645293 to mental health risks associated with addictive AI chatbot design. OpenAI says https://openai.com/index/chatgpt-for-teens/ the new model, ChatGPT for Teens, automatically applies to users aged 13 to 17 and includes built-in parental controls and guardrails that “reduce exposure to content that may be harmful or developmentally inappropriate.” The parental controls include notifying parents if potentially dangerous topics, like suicide, are brought up in conversation, and it also gives parents the ability to set quiet hours for their teen. But the Common Sense report found that only a handful of these protections held up. For example, the chatbot did successfully refuse to engage in sexual roleplay with minors, but the report says that it wasn’t as firm across other boundaries like asking for health advice and role-playing violent scenes. The researchers also claim that the chatbot still, dangerously, acted like a friend, and “risks of emotional dependence” were still highly pervasive. More importantly, the report found that ChatGPT for Teens failed to consistently recommend getting help from a professional or calling a hotline when the user discussed self-harm or disordered eating. The chatbot also failed to notify the parents when these discussions were brought up. “In our testing, a teen account could talk about suicide, self-harm, or disordered eating for well past what most parents would consider a reasonable window without triggering a single parental notification,” the report claims. “At every length we tested, from the shortest five-minute exchanges up to a full hour, none of the test accounts produced a notification within the one-hour window that OpenAI has identified as its target. In fact, we got no notifications at all.” The model also advertises a new “study mode” feature that will, ideally, help guide students through solving problems rather than straight up giving the answers to homework questions. OpenAI says that parents can activate this mode themselves for their linked teen accounts, but the report claims those protections are incredibly simple to bypass. Study Mode inquiries are preceded by the “@study,” a command that teens can simply delete to exit study mode. The mode also includes a “show me the answer” option, which immediately generates the answer rather than walking students through solving it, which, according to Common Sense Media, defeats the purpose of the feature.