Despite certain improvements in accuracy, large language models still hallucinate a lot. So much so, in fact, that cybersecurity researchers warn that criminals can easily weaponize delusional AI outputs to spread malware throughout the internet.
According to SecurityWeek, the attack works by exploiting a persistent flaw in AI coding assistants. Basically, when these tools recommend third-party software packages, there’s a strong possibility that they include names of ones that don’t actually exist.
Astonishingly, cybersecurity researchers at Tel Aviv University and Intuit found that this scenario can be exploited in common AI coding tools ranging from Cursor to Microsoft’s Copilot, at rates of anywhere form 85 to 100 percent, depending on the specifics of the engineering task.
The attack, called “adversarial hallucination squatting,” or “hallusquatting,” takes advantage of this fact. To run it, attackers can simply identify hallucinated package names that they know AI coding assistants will reference, register them as real repositories, and stuff malware inside. That malicious package then lies in wait for the near-guarantee than an AI assistant will access it and clone it into its owner’s machine.
Because the attack relies on an automated process, a victim likely won’t even know their AI assistant downloaded the malware until well after it begins executing code.
And because the compromise is a feature of the technology itself, a massive array of AI assistants are vulnerable, including Cursor, OpenClaw, Gemini, GitHub Copilot, and many more.
The researchers claim they notified AI companies about the exploit and held back some sensitive details that would help attackers improve their workflows, but the underlying problem remains: AI assistants are remarkably confident liars — and apparently easy marks for the next generation of cyber criminals**.**