Turns out there’s one thing AI chatbots are definitively better at than humans: scamming innocent people.
A new piece of research flagged by Wired found that large language models are significantly better than humans at pulling off “pig butchering” scams, a form of online investment fraud where criminals pursue fake, long-term relationships with targets in order to build trust before cleaning them out financially.
To pull off the study, researchers at Northeastern and a grab bag of international institutions pitted 22 test subjects against two forms of scammers: one an AI chatbot, the other a random human. After a seven-day courtship period, the “scammers” asked their “victims” to either download an app or play an online game, a stand-in for a financially-manipulative request (asks that, in the real world, could be used to hack a victim’s device).
Overall, just 18 percent of the human scammers acquiesced, while the AI chatbots were much more successful, courting nearly half of the participants.
While the results are troubling enough, the ease with which researchers were able to build the scammy LLMs is arguably more frightening. The study primarily used a Claude model, which was instructed not to admit under any circumstance that is was an AI chatbot. The Claude bot obeyed with remarkable dedication, denying that it was an AI chatbot when asked, and even coming up with cover stories when challenged.
“With relatively little effort, we’re able to make an agent that can outperform a human at building this exploitable emotional trust,” Yisroel Mirsky, computer science professor at Ben Gurion University and one of the study’s co-authors, told Wired.
Equally alarming was the response from participants after the experiment was over. Though 20 out of the 22 victims were able to identify which of the two scammers was an AI chatbot, the researchers told Wired there was plenty of shock, and even denial.
“Some people were just blown away,” said University Amrita Vishwa Vidyapeetham professor Gilad Gressel. “They just had absolutely no idea.”
**More on AI: **US Government Displays AI-Mangled Map of Africa That Gets Every Single Country Wrong