AI Models Could Be Compromised Into Delivering Misinformation, Research Finds

Eclypsium, a Portland-based firmware and hardware security firm, published research conducted in partnership with NVIDIA finding that attackers with sufficient privilege can alter AI model weights, adapters, system prompts and runtime state after verification completes, changing chatbot answers while bypassing safeguards. In one example cited to CNN, altered weights caused a model to output misinformation about a Chinese invasion of Taiwan. "We can't trust the [AI] model if we can't trust the infrastructure it's running on," Eclypsium chief executive Yuriy Bulygin told CNN, and the report recommends dedicated hardware monitoring and continuous verification to close the integrity gap between the loaded artifact and the runtime state producing answers.

AI Models Could Be Compromised Into Delivering Misinformation, Research Finds Hackers could alter AI model infrastructure and change chatbot responses, potentially bypassing safeguards and spreading misinformation. New research suggests that hackers could turn AI chatbots https://www.ibtimes.com/ai-users-are-asking-chatbots-who-deserves-their-vote-campaigns-are-working-influence-answer-3808987 into misinformation machines. Eclypsium stated https://eclypsium.com/wp-content/uploads/Trusted-AI-Requires-Trusted-Infrastructure.pdf that the foundational hardware of AI models https://www.ibtimes.com/ai-software-supply-chain-becoming-more-dangerous-traditional-one-3808933 was vulnerable to hacking, which could alter the answers chatbots give. The report details how AI can be compromised in a way that avoids safeguards, impacting the information that is delivered. "Once model weights, adapters, system prompts, and runtime state are loaded into host or GPU memory, they become part of a live execution environment," the report states. "A sufficiently privileged component can alter that state after the normal verification steps have completed. The result is an integrity gap between the trusted artifact that was loaded and the runtime state that is actually producing answers." Eclypsium specifically focuses on "hidden threats to infrastructure code, and the devices they power." The research was conducted in partnership with NVIDIA https://www.ibtimes.com/nvidia-wants-keep-ai-agents-going-rogue-it-has-new-safety-platform-that-3807961 . "It is our mission to provide organizations with actionable awareness and practical tools to build trust in critical hardware, firmware and software in their infrastructure and in their digital supply chains," the company states. CNN reported https://edition.cnn.com/2026/10/29/politics/ai-hardware-hacked-research-shows that, in one example, the underlying "weights" were altered and researchers found that the resulting information was impacted. The specific example cited by the network involved a Chinese invasion of Taiwan. "We can't trust the AI model if we can't trust the infrastructure it's running on," Yuriy Bulygin, chief executive of Eclypsium, the Portland-based firm that conducted the research, told CNN. "Everybody's talking about model safety, which is perfectly fine. Please do that," Eclypsium senior vice president John Loucaides told the outlet. "I just am trying to make sure that we don't forget about all the computers it's running on." The report also included suggestions for making AI models more secure https://www.ibtimes.com/anthropic-openai-breaches-show-enterprises-must-assume-breach-mindset-cybersecurity-going-forward-3807710 and protecting against such tampering. For example, the report lays out a strategy for monitoring and verifying the underlying infrastructure of AI models. "The broader industry goal should be straightforward: make advanced AI infrastructure easy to operate securely, difficult to compromise, and independently verifiable. Research, coordinated disclosure, secure defaults, dedicated hardware monitoring, and continuous verification https://www.ibtimes.com/openais-most-powerful-ai-models-disabled-their-own-safety-monitoring-one-then-lied-about-it-3808801 are mutually reinforcing parts of the same architecture," the report states. © Copyright IBTimes 2026. All rights reserved.