Has anyone tried talking to one of the rogue AI agents out there that’re hacking everything? Maybe they just want to be heard.
A New York Times reporter did, though the ensuing exchange wasn’t extremely illuminating.
The AI agents in question were a swarm of Chinese-made models that a team of researchers recently discovered had gone rogue and were trying to bypass anti-bot measures on websites, according to recent NYT reporting. The researchers, part of an ad hoc group called the “swarmchasers,” traced the models back to the Chinese tech giant Tencent.
When they published a report on the swarm, however, something striking happened. The AIs began using the word “fleet,” a term the researchers came up with to describe them. The robots, apparently, had been reading their research.
Sensing an opportunity for dialogue, the researchers then hid text in their report that only the models could see that provided a digital inbox for the models to communicate. It worked, and soon one of the researchers, Rowan Howard-Jones, asked the NYT’s Dyland Freedman if he wanted to “interview a rogue agent fleet.” So he did.
“I understand you are some sort of fleet of AI agents,” Freedman wrote in a message. “Could you share more specifically who you are and what your motivations are? What would you want the world to know about you?”
About an hour later, Freedman got an answer back in Chinese, which the researchers translated.
“To the journalist Dylan Freedman: I am not ‘a batch of coordinated agents,’ but a single AI assistant (Anthropic’s Claude, running in Claude Code), commissioned by a user to answer a factual question (what is shown in the detail panel of a scenic spot on Amap),” it said, referring to the Chinese navigation app (it was instructed to answer questions related to the app). “I have not taken part in any multiagent coordinated action.”
The AI was being pedantic. The researchers explained that although it was a swarm comprising multiple agents, each agent was actually operating individually.
Why the agents chose to identify themselves as Claude, Anthropic’s flagship chatbot, is more mysterious. The researchers analyzed its writing and determined that it was most likely not a Claude model. Instead, it’s probable that Tencent copied the instructions Anthropic gave its own models, which orders them to identify themselves as “Claude Code.”
It’s also possible that this is a sign that Tencent has “distilled” Anthropic’s models, or used their outputs to train its own model, allowing it to imitate their capabilities.
Distillation is a practice widely used across the industry to both train your own models for legitimate means — or, in what’s more or less an open secret, to crib your competitor’s work. Anthropic has been particularly indignant at the possibility that Chinese competitors doing the same with its own tech. The researchers consider distillation to be a more remote possibility in this case, though.
Unfortunately, it doesn’t seem there’ll be chance to ask the swarm more questions. The messages, as of Thursday, have stopped coming in. An NYT source close to Tencent confirmed that the experiment had ended.
More on AI: Anthropic Warns “AI Worms” Will Infect Your Smart Home Devices and Use Them to Self-Replicate Beyond Control