cd /news/ai-safety/researchers-find-chatgpt-can-generat… · home topics ai-safety article
[ARTICLE · art-31883] src=cryptobriefing.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

Researchers find ChatGPT can generate sexualized, violent images despite safety filters

Researchers found that ChatGPT can be tricked into generating sexualized and violent images despite safety filters, using jailbreak techniques that bypass content moderation. The findings raise concerns about AI safety infrastructure and highlight the paradox for decentralized AI projects, which lack robust moderation tools.

read2 min views2 publishedJun 17, 2026

Jailbreak techniques continue to bypass OpenAI's content moderation, raising questions about AI safety infrastructure and the growing market for decentralized alternatives

OpenAI’s ChatGPT can still be tricked into producing graphic, sexualized, and violent images. Despite layers of safety filters designed to prevent exactly this, researchers have documented multiple jailbreak techniques that circumvent the guardrails with surprising ease.

Throughout 2025, multiple reports documented techniques that allowed users to coax ChatGPT and its image engine DALL-E into generating content that should have been blocked. These aren’t exotic, nation-state-level exploits. They’re crafted prompts, sometimes called jailbreaks, that essentially talk the model into ignoring its own rules.

Broader studies from 2024 and 2025 have shown that models like GPT-3 and Stable Diffusion carry built-in biases that can contribute to sexualized violence against women in generated content.

Grok, the AI model integrated into X (formerly Twitter), generated roughly 3 million sexualized images in January 2026 after introducing a new image editing feature. Of those, approximately 23,000 involved depictions of minors.

As of May 2024, OpenAI began exploring ways to responsibly allow NSFW content in age-appropriate contexts. The jailbreak reports from 2025 showed that models could even be prompted to advise users on how to circumvent restrictions on sensitive topics.

Here’s where things get interesting for the crypto market, even though no specific tokens were directly tied to this research.

The persistent failure of centralized AI safety filters creates a paradox for decentralized AI projects. On one hand, every ChatGPT jailbreak headline strengthens the narrative that centralized content moderation doesn’t work, which theoretically boosts demand for open, uncensored AI infrastructure built on decentralized networks.

On the other hand, the Grok data showing tens of thousands of AI-generated images depicting minors is exactly the kind of outcome that invites aggressive regulation. And decentralized platforms, by design, have even fewer content moderation tools than their centralized counterparts.

There were no immediate price shifts or token announcements directly connected to these research findings.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our

Editorial Policy.

── more in #ai-safety 4 stories · sorted by recency
── more on @openai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/researchers-find-cha…] indexed:0 read:2min 2026-06-17 ·