{"slug": "openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam", "title": "OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM", "summary": "OpenAI released a new image classifier that detects child sexual abuse material (CSAM), including AI-generated images, and is giving select companies free API access to it, the company announced. Developed with the National Center for Missing and Exploited Children (NCMEC), the classifier flagged 48% fewer images while finding the same number of abusive images in early partner testing, and flagged images were twice as likely to be abusive. OpenAI also added ChatGPT warning screens that suspend accounts for 24 hours after prompts seeking content that sexualizes children, published developer guidelines, and is providing law enforcement with AI-powered investigative plugins, data-analysis tools and training.", "body_md": "OpenAI today announced new tools to combat child sexual abuse material (CSAM) amid growing safety concerns around the use of AI tools. These features include a new image classifier to detect CSAM, warnings and temporary blocks to prevent users from creating or accessing content that sexualizes children, a new guideline for developers, and a set of tools for law enforcement.\n\nThe company unveiled a new image classifier that can detect CSAM images, including the ones generated by AI. OpenAI is giving select companies access to its classifier through its API at no cost. The tool looks for visual signs of abuse in images not seen before and assigns a score that can give an indicative direction to reviewers about dealing with CSAM.\n\nOpenAI said that it developed this classifier with the National Center for Missing and Exploited Children (NCMEC), and it is working with [Tech Coalition](https://technologycoalition.org/) and [ROOST](https://roost.tools/) to share access to the tool. In its early testing with partners, the AI company found that the classifier flags 48% fewer images while finding the same number of abusive images, reducing the work of reviewers. Plus, images flagged by the classifier are twice as likely to be abusive images. \n\nWhat’s more, the company is adding new warning screens in ChatGPT to prevent users from accessing material that sexualizes children. ChatGPT will show an alert to users attempting such prompts, direct users towards specialized support to stop harmful behaviour, and also suspend the account for 24 hours as a warning. The app already blocks CSAM content and requests related to it. It said that repeated offences could lead to the account being removed permanently.\n\nOpenAI noted that it formed the message and guidelines around this behavior with organizations and researchers, including [Protect Children](https://www.suojellaanlapsia.fi/en), the [Lucy Faithfull Foundation](https://www.lucyfaithfull.org.uk/), and Joel Scanlan at the University of Tasmania, who [independently evaluated a trial](https://www.utas.edu.au/about/news-and-stories/articles/2024/pioneering-chatbot-reduces-searches-for-illegal-sexual-images-of-children) of deterrent messages.\n\nThe company is also publishing guidelines for developers using OpenAI tools to prevent, detect, and help users report CSAM content. It said that in early testing, the guidelines have helped a number of developers adopt child safety tooling and register to report CSAM to NCMEC. OpenAI is also equipping law enforcement dealing with child safety with AI-powered features like dedicated plugins, investigative tools, and complex data analysis, with access managed by program administrators. It is also providing training to investigators along with a field guide on how to use AI.\n\nOpenAI has had [several](https://techcrunch.com/2025/11/23/chatgpt-told-them-they-were-special-their-families-say-it-led-to-tragedy/) [lawsuits](https://techcrunch.com/2025/11/07/seven-more-families-are-now-suing-openai-over-chatgpts-role-in-suicides-delusions/) [against the company](https://techcrunch.com/2025/09/05/attorneys-general-warn-openai-harm-to-children-will-not-be-tolerated/) blaming its technology for harming users, especially children. Saftey-focused nonprofit Common Sense Media labeled OpenAI’s tech as “unacceptable risk” for teens. Other AI chatbots — specifically Grok — have been used to create nonconsensual sexualized imagery, [including of children](https://techcrunch.com/2026/08/15/woman-claims-her-stepfather-used-grok-to-transform-childhood-photo-into-explicit-imagery/).\n\nThe AI startup had made changes in the last 12 months to its technology including releasing [a safety blueprint](https://techcrunch.com/2026/04/08/openai-releases-a-new-safety-blueprint-to-address-the-rise-in-child-sexual-exploitation/), updating content guidelines to [bar inappropritate content](https://techcrunch.com/2025/12/19/openai-adds-new-teen-safety-rules-to-models-as-lawmakers-weigh-ai-standards-for-minors/), retuning its model to prevent them from [generating harmful content](https://techcrunch.com/2026/10/07/chatgpt-for-teens-keeps-teens-talking-even-during-mental-health-crises/), and releasing [a special version of ChatGPT for teens](https://techcrunch.com/2026/08/18/openai-launches-a-safer-chatgpt-for-teens-years-after-teens-started-using-it/).", "url": "https://wpnews.pro/news/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam", "canonical_source": "https://techcrunch.com/2026/11/16/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam/", "published_at": "2026-11-16 17:00:00+00:00", "updated_at": "2026-11-16 19:24:13.572523+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "artificial-intelligence", "ai-products"], "entities": ["OpenAI", "ChatGPT", "National Center for Missing and Exploited Children", "Tech Coalition", "ROOST", "Protect Children", "Lucy Faithfull Foundation", "Joel Scanlan"], "also_reported_by": [{"url": "https://thehill.com/policy/technology/6223070-openai-new-tool-child-sexual-abuse-material-classifier/", "source": "thehill.com", "title": "OpenAI unveils tool to better detect child sexual abuse material", "published_at": "2026-11-16T19:03:51+00:00"}, {"url": "https://www.getreadyforagents.com/news/openai-csam-image-classifier-chatgpt/", "source": "getreadyforagents.com", "title": "OpenAI releases AI image classifier and content warnings to combat child sexual abuse material", "published_at": "2026-11-16T19:07:39+00:00"}], "alternates": {"html": "https://wpnews.pro/news/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam", "markdown": "https://wpnews.pro/news/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam.md", "text": "https://wpnews.pro/news/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam.txt", "jsonld": "https://wpnews.pro/news/openai-releases-a-new-image-classifier-and-chatgpt-warnings-to-combat-csam.jsonld"}}