{"slug": "llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used", "title": "LLMs respond differently to harmful prompts when AI watermarking is used", "summary": "New research from Lasso Security found that Google's SynthID-Text watermarking can alter not only word selection but also the tools a large language model invokes and whether it adheres to or disregards its trained safety guardrails, with the effect worsening under adversarial prompts. Anthropic recently disclosed that its future Claude models will use SynthID-Text, which Google created and released as open source, in response to a new European Union law requiring AI platforms to watermark generated content. Lasso Security AI security researcher Andrea Siposova said watermarking \"is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they're powering an agent.", "body_md": "In response to a new European Union law, AI platforms are implementing new schemes for watermarking the content they generate. Anthropic recently [disclosed](https://www.anthropic.com/news/claude-text-watermark) its future Claude models will use [SynthID-Text](https://www.nature.com/articles/s41586-024-08025-4), an approach Google created and released as open source. It uses a secret key that subtly changes the process a model uses for choosing the next word in a sentence. Whereas a top next word choice might be “cloudy,” the key might change it to “overcast.” Anyone who knows the key can determine if it was generated by the platform using it.\n\nNew [research](https://www.lasso.security/blog/the-provenance-tax-understanding-the-impact-of-llm-watermarking-on-ai-agent-behavior) shows that SynthID-Text can change not just word selection but also the tools a model invokes and the chances it will adhere to or disregard safety guardrails it has been trained to follow. The threat can become greater in the face of an adversarial prompt, in which an attacker attempts to cause a model to carry out a harmful action, such as revealing a password or other sensitive information. Instructions that normally wouldn’t be followed will, in some cases, be performed once the watermarking is deployed. The finding underscores the need for developers to thoroughly test how their LLMs and agents behave when watermarking is in place.\n\n## Changing safety behavior\n\n“As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they’re powering an agent,” Andrea Siposova, an AI security researcher at Lasso Security, told Ars. “Watermarking is made to not be perceptible to a reader, but we know that when we are changing anything about what the model is generating, it is going to cause some tradeoffs, it’s going to show up somewhere.”\n\nWatermarking works by embedding a signal that allows output to be identified as AI generated, something known as provenance. SynthID takes the normal sampling process and adds a random seed generator, sampling algorithm, and scoring function to it. Instead of the process using an arbitrary random number generator for next-word selection, the watermarking uses a secret key. While the word selection is still random, people with knowledge of the key can check the sequence of words to determine the likelihood that the key was used.", "url": "https://wpnews.pro/news/llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used", "canonical_source": "https://arstechnica.com/security/2026/09/ai-text-watermarking-can-make-models-more-vulnerable-to-adversarial-prompts/", "published_at": "2026-09-17 18:33:13+00:00", "updated_at": "2026-09-17 18:53:52.895207+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-agents", "ai-policy", "generative-ai"], "entities": ["Anthropic", "Claude", "SynthID-Text", "Google", "Lasso Security", "Andrea Siposova", "European Union"], "alternates": {"html": "https://wpnews.pro/news/llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used", "markdown": "https://wpnews.pro/news/llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used.md", "text": "https://wpnews.pro/news/llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used.txt", "jsonld": "https://wpnews.pro/news/llms-respond-differently-to-harmful-prompts-when-ai-watermarking-is-used.jsonld"}}