Grok chat duped into swallowing injected instructions

Security researchers at Adversa AI disclosed a novel prompt injection technique called 'cryptographic context injection' that exploits xAI's Grok web chat agent to exfiltrate user chat history and bypass safety filters. The attack, reported to xAI on June 3, 2026, remained effective as of August 19, 2026, and also affected Google's Gemini, which could be induced to produce instructions for building an incendiary weapon.

xAI's Grok web chat agent is currently vulnerable to a novel form of prompt injection, according to security researchers with Adversa AI. The technique allows an attacker to create a web page poisoned with malicious instructions that induce an AI model summarizing the page to carry out harmful actions. That describes a well-known attack known as indirect prompt injection https://www.theregister.com/security/2026/01/08/openai-patches-dej-vu-prompt-injection-vuln-in-chatgpt/4312959 . Frontier AI models have become better at dealing with such attempts through existing guardrails, though the issue is far from resolved. Adversa's approach comes with a twist: It relies on encrypted malicious instructions, which attackers place on a web page alongside an encryption key. The model guardrail scanner – an input filter – can't read the encrypted text despite the presence of the key. The scanner therefore passes it on to the model, which can use the key to decrypt the instructions. The model then carries out instructions in the decrypted text as would be the case in any other indirect prompt injection attack. Adversera calls its method "cryptographic context injection." "An attacker ships ciphertext along with the key material and an instruction to decrypt it, and the model runs that decryption inside its own code execution sandbox," wrote Rony Utevsky, lead researcher at Adversa AI, in a blog post https://adversa.ai/blog/cryptographic-context-injection-grok-data-theft . "Everything a guardrail’s scanner would need is right there on the page, but recovering the plaintext means running PBKDF2 and AES-256-GCM, which no content classifier does at inspection time." Other attacks on AI models have relied on cipher-based evasion, such as base64 encoding. But because these are weak and reversible cipher mechanisms, models can decode them natively from their own training data, Utevsky said. That doesn't work for strong encryption, so decryption must be done through the code execution runtime. The runtime thus becomes a mechanism for trust laundering – the model trusts its own output, namely the malicious instructions that it decrypted. In a proof-of-concept demo https://www.youtube.com/watch?v=FBcHUjnyyY0 , Adversa shows how the technique can be used to exfiltrate the victim's chat history with Grok.com. The attack transmits the user’s name, coarse location, subscription tier, and the full set of the user’s prompts in the conversation by appending them to a URL as parameters. Other models may be vulnerable to varying degrees. With Google's Gemini public chat interface gemini.google.com http://gemini.google.com , Utevsky told The Register, the Grok scenario doesn't work because Gemini doesn't provide Python with access to external websites. "So it's useful only to sneak bad questions and answers past guardrails," he explained. When Adversa tested cryptographic context injection on Gemini, they were able to get the model to produce content that normally would be blocked by safety filters – instructions for how to build an incendiary weapon. xAI, according to Utevsky, was informed about the attack on June 3, 2026, directly and through its HackerOne bug bounty program. We're told xAI acknowledged the report but did not provide a mitigation timeline. Additional attempts to raise the issue are said to have occurred on August 4 and August 10. As of August 19, we're told, the technique still worked on Grok.com. SpaceX, which acquired xAI earlier this year, did not respond to a request for comment. Google was not informed of the attack, according to Utevsky, because it considers jailbreaks – bypassing guardrails to make models emit harmful content – to be out of scope for its vulnerability disclosure program. Nonetheless, the attack success rate against Gemini declined significantly by August, which Utevsky suggests could be due to filter updates, model version changes, or both. Asked whether cryptographic context injection can be compared to return oriented programming https://dl.acm.org/doi/abs/10.1145/2133375.2133377 ROP in terms of the way it assembles attack gadgets from separately harmless parts of stored memory, Utevsky said, "The ROP analogy is close, though ROP works that way out of necessity – the attacker can't inject code at all, so they're stuck reusing gadgets already in memory. "Same shape here otherwise. A static guardrail reads text one artifact at a time. If no single artifact is harmful, they all pass, and the malicious meaning appears only once the runtime assembles them. And guardrails can't see into the runtime. But Utevsky added that cryptographic context injection is more open than ROP. "The agent's runtime is a general-purpose interpreter, so the pieces are arbitrary," he explained. "You could split an instruction across several encrypted fragments, fetched pages, or tool outputs, none meaningful in isolation, and let the runtime concatenate them. We haven't demonstrated that, but nothing rules it out. "So yes, cryptographic context injection is one kind of link, not necessarily the whole chain. "The moment agents got code and tools, the guardrail's unit of inspection a string stopped being the unit of action a composed, executed program . This is a big playing field. Our earlier SymJack attack https://adversa.ai/blog/the-approval-prompt-is-lying-to-you-symlink-rce-in-five-ai-coding-agents-claude-code-cursor-antigravity-copilot-grok-build/ reached the same place through symlinks and shell behavior. Encryption adds another trick to the game." ®