Hackers could alter AI model infrastructure and change chatbot responses, potentially bypassing safeguards and spreading misinformation. #
New research suggests that hackers could turn AI chatbots into misinformation machines. Eclypsium stated that the foundational hardware of AI models was vulnerable to hacking, which could alter the answers chatbots give. The report details how AI can be compromised in a way that avoids safeguards, impacting the information that is delivered.
"Once model weights, adapters, system prompts, and runtime state are loaded into host or GPU memory, they become part of a live execution environment," the report states. "A sufficiently privileged component can alter that state after the normal verification steps have completed. The result is an integrity gap between the trusted artifact that was loaded and the runtime state that is actually producing answers."
Eclypsium specifically focuses on "hidden threats to infrastructure code, and the devices they power." The research was conducted in partnership with NVIDIA.
"It is our mission to provide organizations with actionable awareness and practical tools to build trust in critical hardware, firmware and software in their infrastructure and in their digital supply chains," the company states.
CNN reported that, in one example, the underlying "weights" were altered and researchers found that the resulting information was impacted. The specific example cited by the network involved a Chinese invasion of Taiwan.
"We can't trust the [AI] model if we can't trust the infrastructure it's running on," Yuriy Bulygin, chief executive of Eclypsium, the Portland-based firm that conducted the research, told CNN.
"Everybody's talking about model safety, which is perfectly fine. Please do that," Eclypsium senior vice president John Loucaides told the outlet. "I just am trying to make sure that we don't forget about all the computers it's running on."
The report also included suggestions for making AI models more secure and protecting against such tampering. For example, the report lays out a strategy for monitoring and verifying the underlying infrastructure of AI models.
"The broader industry goal should be straightforward: make advanced AI infrastructure easy to operate securely, difficult to compromise, and independently verifiable. Research, coordinated disclosure, secure defaults, dedicated hardware monitoring, and continuous verification are mutually reinforcing parts of the same architecture," the report states.
© Copyright IBTimes 2026. All rights reserved.