AI has supercharged scammers with an entirely new arsenal of tools. Google is cracking down.
On Friday, the company filed a lawsuit to dismantle a Chinese cybercrime group known as "Outsider Enterprise," which it alleges has used Gemini and other AI tools to generate thousands of fake text campaigns. According to Google, the group has used AI-generated government and brand websites to obtain people's credit card numbers and personal information, using the targets' trademarks and logos in these phishing campaigns.
"You’ve seen the texts: fake package alerts, urgent bank warnings, panicked messages about your compromised account," the company said in its announcement. "Behind them is an AI-powered cybercrime network built to steal your passwords and credit cards."
The tech giant claims that more than 100,000 Americans have been impacted, and losses are "estimated in the millions." Additionally, more than 9,000 fake websites and one million fraudulent URLs are connected to this group.
But the lawsuit isn't the only action Google is taking against these scams:
- The company said it is coordinating with the FBI to go after Outsider Enterprise, as well as working with telecom providers like AT&T, T-Mobile and Verizon to block texts before they reach users. Halimah DeLaine Prado, Google's general counsel, told The New York Timesthat it represents the company's first coordinated effort and lawsuit of this nature. "That speaks to the breadth of impact that this particular scam has," she said. - The company is also backing seven bipartisan bills that aim to fight back against these scams, including legislation that specifically protects elders against these kinds of attacks, and a bill for public awareness and education around AI.
It's also not Google's first attempt at warding off scammers: In early June, Google launched fake call detection for Android, which flagged suspected spoofed calls. According to Google, 55,000 spam texts were flagged by Android users over a two-week period in May, and more than 2.5 million messages by this organization were sent to Android users during that same period.
Our Deeper View #
AI has given both security defenders and attackers way more ammo. Through this effort, Google is taking accountability for its part in that, tracking down the wide-scale misuse of its models. Though Google's attempt to contain the problem is a noble one, and some of the damage could be mitigated, it's still a reactionary effort rather than a proactive one. It begs the question of whether the power that these models provide should be so widely available for anyone to use or abuse. Additionally, if the genie is already out of the bottle, should the industry heed the slowdown warnings from companies like Anthropic before more harm is done?