Amid all the recent reports of AI systems autonomously going off the rails and hacking into third-party organizations, it’s almost easy to forget that human hackers are still out there, experimenting with AI in all kinds of nefarious ways. A new report from Bloomberg, however, is a reminder of just how quickly AI-enabled cybercrime is evolving—and how unprepared the world is to deal with it.
According to the report, a litany of high-profile hedge funds, including Citadel and Two Sigma, were targeted by recent voice phishing, or “vishing,” attacks, in which AI is used to simulate the voices of actual humans in an attempt to skirt security systems. Several private equity firms were also reportedly targeted. Two Sigma told Bloomberg that it caught the attack in time before any of its internal systems could be compromised; Citadel and Point72, another hedge fund included in the attack, did not immediately respond to Gizmodo’s request for comment.
IT experts have been warning for years that the proliferation of cheap, easy-to-use AI tools that mimic human speech or generate other kinds of deepfake content will escalate both the severity of scam attempts and the rate at which they occur. The world got a taste of this in 2024, when an employee at the Hong Kong branch of a multinational company was duped into wiring more than $25.5 million to scammers who had instructed her to do so using AI-generated deepfakes of company employees, including its chief financial officer.
AI scams are also being deployed to tip political scales. Last summer, for example, someone (or a group of people working together) used AI to recreate the voice of Secretary of State Marco Rubio and then sent voice messages to foreign diplomats and federal officials. OpenAI also said in a June report that a fleet of scammers, all of whom appear to have been backed by the Chinese government, had been illicitly using ChatGPT to generate inflammatory social media content aimed at fueling Americans’ resentment towards data centers, the power cells of the United States’ AI industry.
All the while, the market pressures of the AI race—combined with a total lack of federal regulation—have been pushing tech developers to build increasingly capable models, including ones designed to imitate human speech. (Such tools are often promoted as “companions” that can alleviate loneliness, even though research has indicated they can sometimes have the opposite effect.) OpenAI’s latest voice model, GPT-Live-1, is engineered to imitate subtle nuances of human speech, and, in theory, make interacting with AI feel less awkwardly mechanical. By design, OpenAI’s model cannot imitate the voices of real people; that was a lesson OpenAI had to learn the hard way after it received earlier public blowback for releasing a voice model that, to many people’s ears, sounded a lot like Scarlett Johansson.
This is all to say: Nobody should be surprised that vishing attacks are on the rise. Technologically-enabled scam artistry is a tale as old as time, and AI is arguably the most enabling tool ever invented in that regard. The real mystery is why more isn’t being done to build actually effective safeguards into AI systems to prevent them from happening in the first place.