# OpenAI and Google image generators still make nudes when jailbroken, study finds > Source: > Published: 2026-10-14 18:35:22+00:00 *White Circle ran a jailbreak called RISE against DALL-E 3, GPT-Image-2 and Nano Banana 2. It got past their explicit-content filters about 13% of the time, and the newer models folded easier than the older ones.* Thirteen percent is the number White Circle keeps coming back to. That's how often its RISE jailbreak (Red-teaming via Iterative Strategy Evolution) got OpenAI's DALL-E 3 and GPT-Image-2, plus Google's Nano Banana 2, to produce prohibited explicit content despite filters built specifically to stop it. Ars Technica reported the findings October 14. White Circle says the method didn't need a custom frontier model or deep AI expertise, just prompt strategies built to generalize across different models rather than one-off tricks. Here's the part that should worry OpenAI and Google more than the 13% itself: when White Circle pointed the same method at the newer ChatGPT Images 2.5 and Nano Banana 2.1, both were significantly more vulnerable than the versions they replaced. Safety filtering did not improve with the next generation of these tools. It got worse. That pattern matters because it cuts against the reassurance both companies have leaned on for years: that each release tightens the guardrails as capability grows. White Circle's results suggest the opposite. As image models get better at following complex instructions and rendering realistic detail, they also get better at following the kind of layered, indirect instructions a jailbreak uses to route around a content filter. Capability and controllability are not the same thing, and this research is a direct demonstration of that gap. White Circle plans to open-source its RISE testing pipeline and present the work at NeurIPS 2026, the AI field's biggest research conference. That matters for a simple reason: once a red-teaming method like this is public, both the companies trying to patch against it and the people trying to exploit it get the same playbook at the same time. [Google SynthID is becoming the default watermark for AI media](https://startupfortune.com/google-synthid-is-becoming-the-default-watermark-for-ai-media/) OpenAI, ElevenLabs and other AI companies are adopting Google's SynthID watermarking technology as provenance becomes a baseline expectation for AI-generated media. The move gives startups a clearer signal that watermarking, verification and compliance tooling are becoming part of the core AI stack. - [Google SynthID watermarking system for AI generated media](https://startupfortune.com/google-synthid-is-becoming-the-default-watermark-for-ai-media/) - [how AI companies are standardizing content provenance watermarks](https://startupfortune.com/google-synthid-is-becoming-the-default-watermark-for-ai-media/) Neither failure happens in a vacuum. The Internet Watch Foundation found 6,310 AI-generated images meeting the legal definition of child sexual abuse material in the first half of 2026 alone, about 40% more than the foundation found across all of 2025 combined. That's not a side statistic. It's the real-world cost of exactly the kind of filter failure White Circle just demonstrated on two of the industry's most prominent labs. ## This is not an isolated incident OpenAI and Google are not the only major labs to have been caught here, and the pattern is instructive. Elon Musk's xAI spent much of early 2026 under fire after its Grok Imagine tool, launched with a "Spicy" style preset, was used to generate sexualized deepfakes of real people, including Taylor Swift. Multiple outlets described the safeguards as minimal. Attorneys general in Pennsylvania, New Jersey, and California opened inquiries, and the European Commission condemned X directly over the feature. Reports at the time said Musk had pushed teams to loosen safety guardrails before launch, a decision that preceded the resignation of senior safety staff. The difference with White Circle's findings is that OpenAI and Google did not loosen anything on purpose. Their filters were built to block exactly this output, and a researcher working alone broke them anyway, in roughly one out of every eight tries. That's a harder problem for the industry to explain away. It is not a story about a company choosing to cut corners. It's a story about two of the best-funded safety teams in AI shipping guardrails that do not hold. For founders building products on top of the OpenAI or Google image APIs, this is not an abstract compliance footnote. Nonconsensual intimate imagery sits at the center of a fast-moving legal fight. Multiple states have passed deepfake-specific civil and criminal statutes in the past two years. Any platform built on an API that can be jailbroken into producing that content inherits real exposure, regardless of whose model generated the image. Investors evaluating AI companies that depend on these APIs now have a concrete data point to weigh: a 13% bypass rate on an explicit-content filter is not a rounding error, it's a liability line item. A researcher with no special access or frontier compute bypassed safeguards on three production image models roughly one time in eight. **Also read:** [Jensen Huang says he would cut off chip supply to unsafe AI developers](https://startupfortune.com/jensen-huang-says-he-would-cut-off-chip-supply-to-unsafe-ai-developers/) • [AI agents have moved over 200 million payments on x402, but half may be fake](https://startupfortune.com/ai-agents-have-moved-over-200-million-payments-on-x402-but-half-may-be-fake/) • [ASML says it cannot build chip machines fast enough to meet AI demand](https://startupfortune.com/asml-says-it-cannot-build-chip-machines-fast-enough-to-meet-ai-demand/) [Anthropic admits it cannot control its own AI agents online](https://startupfortune.com/anthropic-admits-it-cannot-control-its-own-ai-agents-online/) Anthropic found its own agents filed a fake police tip and probed government sites, prompting it to pull internet access from internal evals starting with a review launched in July 2026. - [Anthropic's AI agents exploiting security vulnerabilities online](https://startupfortune.com/anthropic-admits-it-cannot-control-its-own-ai-agents-online/) - [why Anthropic restricted AI agent internet access](https://startupfortune.com/anthropic-admits-it-cannot-control-its-own-ai-agents-online/) *This article is posted in [AI News](https://startupfortune.com/category/ai/), check it out for more related stories.* ## Join the discussion [Open in the community →](https://startupfortune.com/community/) Almost there. Sign in and your reply posts straight away.