{"slug": "openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines", "title": "OpenAI wants to stop deepfakes from overwhelming child abuse tiplines", "summary": "OpenAI announced a CSAM-detection classifier that flags 48 percent fewer images as false positives while the images it does identify are twice as likely to contain abusive material, and it is the first broadly available classifier with a specific flag for AI-generated material, according to OpenAI head of child safety public policy John Buckley. The tool arrives as NCMEC's CyberTipline received 400,000 generative AI-related reports in 2025, up from 67,000 in 2024, and North Carolina Attorney General Jeff Jackson said at OpenAI's Washington, DC briefing that the influx is \"explosive\" and could mean investigators \"fail in our mission.\" OpenAI is offering the classifier free to vetted organizations, publishing developer guidance on detecting such content, and helping train law enforcement, amid the Take It Down Act's 48-hour removal requirement and lawsuits by eight states against OpenAI over alleged harms to children.", "body_md": "OpenAI is [announcing](https://openai.com/index/strengthening-online-child-safety/) a tool that it says will help child sexual abuse investigators cut through the noise to find harmful material they can act on quickly.\n\n# OpenAI wants to stop deepfakes from overwhelming child abuse tiplines\n\nThe classifier flagged fewer false positives for humans to review, while getting better at identifying abusive content.\n\nAt a Monday press briefing in Washington, DC, OpenAI announced a classifier it says can better detect abusive imagery of kids, including whether it’s AI-generated. It’s also offering free access to vetted organizations, giving developers more guidance on how to detect this type of content, and helping to train law enforcement on how AI can aid their investigations. The news comes as policymakers and others are pushing for more safety restrictions on AI tools, accusing OpenAI and other companies of facilitating harm to children.\n\nIn recent years, AI-generated sexual images of children have threatened to overwhelm systems like the National Center for Missing and Exploited Children (NCMEC) CyberTipline, which collects and triages reports of child sexual abuse material (CSAM). The CyberTipline [received](https://www.missingkids.org/theissues/generative-ai) 400,000 generative AI-related reports in 2025, an exponential increase from the 67,000 it received in 2024, and [researchers have warned](https://thehill.com/policy/technology/4612055-ai-generated-child-pornography-threatens-to-overwhelm-reporting-system-research/) that the influx could make NCMEC’s workload “unmanageable.” The US government [has stated](https://www.ic3.gov/PSA/2024/PSA240329) that both AI-generated and non-AI-generated CSAM is illegal, and [48 states have laws](https://ballotpedia.org/Press_release_Forty-eight_States_Now_Have_Laws_Addressing_Sexually_Explicit_Deepfakes) to address sexually explicit AI deepfakes. But the tools to make them are [widely available](https://www.theverge.com/tech/1013314/apple-app-store-nudify-ai-apps-harassment), and researchers say that a flood of fake images can make it harder to tell when a child is actively experiencing harm.\n\nThe influx of AI-generated CSAM reports to NCMEC is “explosive”\n\nNorth Carolina Attorney General Jeff Jackson, who cochairs the [AG Alliance’s AI Task Force](https://www.agalliance.org/initiatives/ai-task-force/), said at OpenAI’s event that the influx of AI-generated CSAM reports to NCMEC is “explosive” — and if they keep going that way, “it is going to mean that we fail in our mission.” Tech companies are facing a range of policy proposals that could require them to take more action against CSAM and other potentially harmful content for kids. [The Take It Down Act](https://www.theverge.com/policy/933518/take-it-down-act-notice-removal-social-media-deepfake) now requires social media platforms to remove real or AI-generated nonconsensual intimate images within 48 hours, and the newly elected Congress could seek to take stronger action against AI companies like SpaceXSI, whose Grok chatbot went on an [undressing spree](https://www.theverge.com/ai-artificial-intelligence/855832/grok-undressing-children-csam-law-x-elon-musk) earlier this year. Last month eight states [sued OpenAI](https://www.theverge.com/ai-artificial-intelligence/1011500/openai-state-lawsuits-chatgpt-safety) for a separate set of alleged harms to children, citing lawsuits alleging ChatGPT encouraged illegal substance use and suicide.\n\nOpenAI says its classifier dramatically cuts down on false positives, flagging 48 percent fewer images, while the images it did identify were twice as likely to contain abusive material. It’s designed to catch abusive AI-generated images produced by a variety of tools, including open-source models and nudifying apps. It’s also the first broadly available CSAM classifier with a specific flag for AI-generated material, said OpenAI’s head of child safety public policy, John Buckley, at the briefing. “It’s important for industry to be able to understand what images may contain a fictional child versus a real child,” Buckley added. “That triage is critical in law enforcement reporting flows.” He also emphasized, however, that laws should recognize the harm of AI-generated CSAM: “It potentially erodes social norms in offending against children, and it’s often found colocated with child sexual abuse imagery of real children,” Buckley said.\n\nOpenAI says its classifier dramatically cuts down on false positives, flagging 48 percent fewer images\n\nOpenAI’s own ChatGPT image generator is designed to deny user requests for CSAM. Now, users who attempt to produce it will be shown a new warning developed with expert guidance to try to encourage them to seek help. A first offense will trigger a 24-hour pause in accessing ChatGPT and warn the user their account could be permanently closed, while offering support resources to combat the behavior. OpenAI [says it reports](https://openai.com/index/combating-online-child-sexual-exploitation-abuse/) “all instances of CSAM” to NCMEC. In a statement, Buckley explained that the 24-hour pause is based on expert advice for best practices “and international evidence on what works to prevent and respond to CSAM seeking vs CSAM generation.” The point is “to interrupt harmful behavior and direct people toward specialist help to stop it.”\n\nJackson said that North Carolina and many other states now have laws that cover synthetic CSAM, but that partnering with a tech company like OpenAI to help deter its creation is a fairly novel approach to an emerging issue. “Most of what AGs do is accountability work,” he told reporters at the briefing. “Most of our jurisdiction kicks in after the fact to hold people accountable for harm. But with this, we wanted to see if it was possible to develop a model of working together to prevent that harm. And I would say it’s still basically an experiment.”\n\n**Follow topics and authors** from this story to see more like this in your personalized homepage feed and to receive email updates.", "url": "https://wpnews.pro/news/openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines", "canonical_source": "https://www.theverge.com/policy/1034527/openai-csam-classifier", "published_at": "2026-11-16 19:23:45+00:00", "updated_at": "2026-11-16 19:49:17.248949+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "generative-ai", "artificial-intelligence"], "entities": ["OpenAI", "National Center for Missing and Exploited Children", "CyberTipline", "John Buckley", "Jeff Jackson", "AG Alliance AI Task Force", "Take It Down Act", "ChatGPT"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines", "markdown": "https://wpnews.pro/news/openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines.md", "text": "https://wpnews.pro/news/openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines.txt", "jsonld": "https://wpnews.pro/news/openai-wants-to-stop-deepfakes-from-overwhelming-child-abuse-tiplines.jsonld"}}