{"slug": "recursion-into-madness", "title": "Recursion into madness", "summary": "A security researcher's blog post demonstrates that recursive AI rewriting of text leads to increasingly bizarre and distant output, with Gemini producing metaphors like 'vast, towering tomb of bleached opulence' after repeated iterations. The post contrasts this with image editing, where accumulated errors cause chaos, and notes that simple text edits remain deterministic.", "body_md": "I don’t write about generative AI much. Everyone else does; not much of consequence ever gets said. I think the technology is great where it helps us automate mundane tasks, cancerous where it undermines genuine human expression, and [darkly funny where the two worlds collide](https://blog.coredump.cx/p/ai-childrens-books-body-horror-edition).\n\nAs a security person, though, I want to know how the models fail. They’re human-shaped but inhuman; they beg to be anthropomorphized and then choke on a seahorse emoji. Recursive processes where AI feeds on its own output are of special interest. Pragmatically, they’re the core mechanic of agent loops; more abstractly, they’re a major theme in the doomsday predictions of AI skeptics (model collapse) and the believers in the coming AI god (rapid self-improvement).\n\nAnd you better believe that recursion is fun! Here’s a quick video I put together by asking Nano Banana 2 to generate a faux movie poster and then asking it to repeatedly make a localized edit to the title:\n\nThis effect is the bane of AI-based image and video editing tools: you get a very limited number of tries before the accumulated errors unleash pure chaos.\n\nDoes the same happen for text or code? Well, yes and no. In image pipelines, there are many inherent sources of noise. For text, input and output encoding is lossless and the amount of injected per-token entropy is small, so you get deterministic results on simple, well-specified tasks. In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.\n\nThis is different for open-ended rewrites. Most rewrite prompts don’t produce runaway sequences; instead, the system wanders around and then converges on stable phrasing before too long; maximum LLMese, if you will. Still, what you end up with can be quite distant from the starting point.\n\nTo illustrate, let’s take the following passage from Raymond Chandler:\n\n*“This room was too big, the ceiling was too high, the doors were too tall, and the white carpet that went from wall to wall looked like a fresh fall of snow at Lake Arrowhead. There were full-length mirrors and crystal doodads all over the place. The ivory furniture had chromium on it, and the enormous ivory drapes lay tumbled on the white carpet a yard from the windows. The white made the ivory look dirty and the ivory made the white look bled out. The windows stared towards the darkening foothills. It was going to rain soon. There was pressure in the air already.*\n\n*I sat down on the edge of a deep soft chair and looked at Mrs. Regan. She was worth a stare. She was trouble. She was stretched out on a modernistic chaise-longue with her slippers off, so I stared at her legs in the sheerest silk stockings. They seemed to be arranged to stare at. They were visible to the knee and one of them well beyond. The knees were dimpled, not bony and sharp. The calves were beautiful, the ankles long and slim and with enough melodic line for a tone poem. She was tall and rangy and strong-looking. Her head was against an ivory satin cushion. Her hair was black and wiry and parted in the middle and she had the hot black eyes of the portrait in the hall. She had a good mouth and a good chin.”*\n\nI think it’s good writing. It’s not high-brow, but it’s… visual. Visceral. It draws you in.\n\nWith this quote in hand, I asked Gemini to *“boldly rewrite”* the passage to *“improve tone, clarity, and flow”*. The model immediately fell back onto its worst habits: bizarre metaphors that sound sophisticated but are just obtuse. In the first iteration, the room became a 👾 “*vast, towering tomb of bleached opulence”,* the drapes 👾 *“pooled greedily on the carpet”,* and the air 👾 *“sat thick with the threat of rain”.*\n\nWith each iteration, I presented the model only with its previous output and the same rewrite prompt. After eight iterations, the metaphors have gotten truly weird: 👾 *“looming mirrors weaponized the brutal glare”,* Mrs. Regan’s legs became 👾 *“sculpted for ruin”*, and her mouth had 👾 *“an insatiable hunger for destruction”* — quite the look.\n\nAfter twelve iterations, we’re in uncharted territory. The carpets are 👾 *“suffocating the room in sterile, unyielding wealth”*. The brewing storm threatens the narrator, 👾  *“waiting to shatter the floor-to-ceiling glass”*. As for Mrs. Regan, she’s now 👾 *“pure, unapologetic malice … effortlessly cruel”*. She’s one with the storm.\n\nAfter iteration sixteen, 👾 *“opulence didn’t merely occupy the room; it annihilated it”*, 👾 *“a suffocating white carpet devoured the space, plunging it into a blinding, agonizing glare”.* Against that backdrop, Mrs. Regan’s legs 👾 *“cut through the air like stiletto blades”*, her obsidian eyes 👾 “*burning with the singular urge to destroy”.*\n\nAfter about twenty iterations, the model steps away from the ledge and the output more or less stabilizes, cycling through several tepid metaphors for the decor (*“stripping”, “scrubbing”,* or *“eradicating”* traces of humanity), describing Mrs. Regan as *“pure venom”* or *“concentrated venom”*, and asserting she intendsto *“dismantle”, “tear apart”, “ruin”,* or *“flay”* the narrator. Finally, we hit a stable point at iteration 31, with the following piece:\n\n👾 *“The room turned its opulence into a weapon, stripping away every trace of human warmth. Blinding, clinical light scorched pristine silk, razor-honed chrome, and spotless carpet. Beyond the glass, a bruised storm raged; within, leaded crystal and mirrors shattered the glare into jagged shards.*\n\n*I froze before Mrs. Regan—venom wrapped in high fashion. Reclined on a low chaise, her sheer-sheathed legs cut through the space like obsidian blades. Merciless and suffocating, she was the living mirror of the monstrous portrait in the foyer. Every sharp angle of her jaw and every frozen glance delivered a single guarantee: she was about to ruin me.”*\n\nWho knew that Raymond Chandler had such a knack for horror. Until next time, friends.\n\n*PS. For obvious reasons, this post will fail Pangram tests. That said, everything except the* 👾*-marked quotations is human writing.*", "url": "https://wpnews.pro/news/recursion-into-madness", "canonical_source": "https://blog.coredump.cx/p/recursion-into-madness", "published_at": "2026-09-07 21:45:50+00:00", "updated_at": "2026-09-07 22:00:19.622784+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "large-language-models"], "entities": ["Gemini", "Nano Banana 2", "Raymond Chandler"], "alternates": {"html": "https://wpnews.pro/news/recursion-into-madness", "markdown": "https://wpnews.pro/news/recursion-into-madness.md", "text": "https://wpnews.pro/news/recursion-into-madness.txt", "jsonld": "https://wpnews.pro/news/recursion-into-madness.jsonld"}}