The notion that scammers can use AI to sharpen their deceptions, polish their language, and lubricate their banter with victims is now a reality for anyone fighting the fraud operations that steal tens of billions of dollars a year worldwide. But can AI fully replace a human scammer, autonomously building the web of deception leading up to the fake investment that defrauds the mark? One study’s experiment suggests that it can—and may even be able to carry out the majority of that long con more effectively than humans.
Researchers from four universities—Amrita Vishwa Vidyapeetham in India, Foscari University of Venice, the University of Melbourne, and Ben Gurion University of the Negev—carried out a broad study on the use and potential of generative AI chatbots in the growing scam industry centered around a form of fraud known as “pig butchering,” text-based romance scams that eventually shift to fake crypto investments that steal as much as six-figure sums from victims. In their study, the researchers pitted AI chatbots directly against humans in a simulation of the scamming process—or more specifically, the long, trust-building conversations that eventually lead up to soliciting a fake investment from the scam’s target.
They found that for the relationship-establishing stages of the scam—the stage that in real-world scams typically represents the longest part of the interactions with the victim, often stretching to months—an AI chatbot performed remarkably effectively, successfully impersonating a human and by some measures outperforming the real human “scammers” in their experiment.
After a week of talking to 22 test subjects who were recruited to unwittingly serve as “victims,” the chatbots and human scammers were assigned to ask the victim to either download an app or play an online game as a proxy for their willingness to fulfill the scammer’s request. Nearly half of the test subjects fulfilled that request for the AI chatbot, while fewer than one in five took the bait when talking to a human. The subjects also graded their level of trust with each “person” they were texting with and gave significantly higher scores to the AI bot.