{"slug": "how-claude-s-watermarking-probably-works", "title": "How Claude's watermarking (probably) works", "summary": "Anthropic has begun watermarking AI-generated text across all Claude models, including in the EU, using an imperceptible statistical token watermarking scheme rather than hidden Unicode or whitespace, according to a technical analysis by a developer who examined 7.2 million characters of Claude output. The analysis, which found no perceptible changes in historical chat logs, suggests the watermark is likely based on a green/red list approach that adds a statistical bias to token selection, making it detectable only through specific statistical tests and less effective on very short text.", "body_md": "Yesterday, [Anthropic announced](https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content) that they had started watermarking AI-generated content. Folks across the internet were particularly up in arms about it (I think rightfully so), especially because this apparently is happening to all Claude models whether or not you are in the EU. I wanted to investigate what they’re actually doing and whether it’s perceptible or changeable.\n\nAnthropic provides some insight into their approach in their article [How Claude marks AI-generated content](https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content). Though it doesn’t actually provide any technical details on the implementation, it provides some guidance to help draw a wide net around the scheme they’re using. The key clues Anthropic left in their help center article:\n\n- The scheme “weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.”\n- The watermarking doesn’t work well on very short text.\n- The watermark seems to have started recently (August 2nd or later)\n\nThis helps us narrow down the possibilities quite a bit.\n\n# Setup\n\nTo actually get to the bottom of what Anthropic are doing, I realized that there are some interesting experiments you can run. [Gloaguen et al.](https://www.sri.inf.ethz.ch/blog/probingsynthid) created specific tests to check for statistical watermarking, and we can also perform an analysis of Claude outputs to check for things like hidden unicode or whitespace results.\n\nFor good measure, I also downloaded a dump of my Claude chats (I’ve used Claude Code since 2/4/2025 and have recorded 1206 sessions) and did a quick comparison to see if there was any changepoint around early August that percepitbly changed the mix of tokens that Fable 5 output (my usual daily driver model). I wasn’t able to find any perceptible difference in this historical analysis, which confirms Anthropic’s claim that the watermarking is generally imperceptible unless using more specific tests.\n\n# No hidden unicode or whitespace\n\nThe second thing I checked is whether there’s hidden unicode or whitespace or punctuation patterns. This was pretty conclusive: they’re not doing something so simple.\n\nAn analysis across 7.2 million extracted prose characters (both on historical Claude Code text as well as generated Claude Code text on August 11) showed that the only unicode characters that were output by Claude were reasonable and part of day to day usage:\n\n- Curly quotation marks, apostrophes, and horizontal ellipses\n- Em/en dashes (A LOT of them unfortunately)\n- Mathematical symbols\n- Accented or non-English characters\n- Emoji\n\nAn audit by Codex, with me spot checking about 10 samples, found no anomalous instances that were consistent with a hidden Unicode watermark. I similarly found no whitespace encoding marks.\n\nThis evidence, combined with the fact that the watermarking is imperceptible and doesn’t work on short text, means that Anthropic is most likely using some form of statistical token watermarking.\n\n# Statistical watermarking schemes\n\nThere are a few different schemes that are available that can add watermarking to text. I’ll talk about the simplest version, the green/red list created in 2023 by [Kirchenbauer et al.](https://arxiv.org/pdf/2301.10226), because it’s the easiest to explain and once you understand it will allow you to understand how these schemes generally work.\n\n## Green/red lists\n\nThis scheme is super basic, but it’s quite clever and fun. Here are the steps:\n\n- Split your output vocabulary into two sets: a green and a red list. Make sure they’re chosen uniformly at random.\n- Then for the green list, add $\\delta$ to all of the logits and sample from the updated distribution at decoding time.\n\nTo figure out whether a text has been watermarked, then you compute the z-score that the tokens in the green set appear. If the text wasn’t watermarked, then the expected value of text in the green list is $T/2$, where $T$ is the token count, with a standard deviation of $\\frac{\\sqrt{T}}{2}$. So the suspiciousness of getting this outcome is just the z-score:\n\n$$ z = \\frac{2G-T}{\\sqrt{T}}. $$If some red token already has probability 0.99 (which would be a huge logit lead) a big $\\delta$ nudge to the greens still wouldn’t overtake it. So the bias only changes words that have a lot of options and generally high entropy.\n\nFor an example, let’s say you asked your LLM to write a poem, you might have the following potential sentences that get generated:\n\n| Word | List | Sentence |\n|---|---|---|\n| crisp | Green | It was a crisp morning |\n| quiet | Green | It was a quiet morning |\n| foggy | Red | It was a foggy morning |\n| cold | Red | It was a cold morning |\n\nIf it was watermarked, you’d get an imperceptibly higher percentage of generating “crisp” or “quiet” morning (depending on how strongly the LLM provider decided to watermark with their $\\delta$ value). Do this across all the words that an LLM is generating, and you can get high levels of confidence in your watermarking.\n\nThat being said, I don’t believe Green/red lists are used in practice because they’re easy to detect and there are schemes that use the model’s available entropy more efficiently (and thus harder to detect and less likely to change the outputs of the model). The most well known scheme is [SynthID-Text](https://www.nature.com/articles/s41586-024-08025-4) which was developed by Google Deepmind and is used by Google in production.\n\n## SynthID-Text\n\nSynthID-Text is the same idea as green/red lists, but using a slightly different approach that Deepmind calls tournament sampling. Here are the steps:\n\n- At each decoding step, hash a secret key together with the last $h$ tokens of context to produce a seed. That seed assigns every vocabulary token one $g$ value in $[0,1]$ per tournament layer.\n- Draw your candidate output tokens using the model’s logits as normal.\n- Run a tournament bracket where candidates face off in pairs, and the candidate with the higher $g$ value for that layer advances.\n- Output the tournament winner as the decoded token.\n\nTo detect the watermark, you just need the secret key. You can compute the seed value and the $g$ values associated with every token in the text:\n\n$$ \\operatorname{Score}(x)=\\frac{1}{mT}\\sum_{t=1}^{T}\\sum_{\\ell=1}^{m}g_\\ell(x_t,r_t), $$Then compare the result with the null distribution and generate a standardized score similar to the Green/red list detection.\n\nSynthID also uses repeated-context masking: if the same $h$-token context window has already been used during a response, the implementation can decline to watermark that position. This prevents a repeated context from receiving the same bias over and over, but it matters substantially for testing because using the wrong context length can accidentally trigger the mask and hide the signal.\n\nSynthID is a bit more disguised than Green/red lists because every candidate is drawn from the model’s own distribution, so the tournament can only promote words the model already considered saying. When the model is pretty certain about the next token, there’s low entropy and not much watermarking (just like Green/red lists). The signal gets stronger for high entropy words, which is also why these schemes need a decent amount of text before detection becomes reliable.\n\nWhen the model is nearly certain about the next token, there is little entropy available for any sampling-based watermark to use. The signal accumulates mainly at higher-entropy positions, which is one reason these schemes need a decent amount of text before detection becomes reliable.\n\nGoing back to our poem, say the model draws four candidates and we run a two-layer tournament using the continuous-score variant:\n\n| Matchup | g-values | Winner |\n|---|---|---|\n| crisp vs. foggy (layer 1) | 0.71 vs. 0.24 | crisp |\n| quiet vs. cold (layer 1) | 0.90 vs. 0.42 | quiet |\n| crisp vs. quiet (layer 2) | 0.35 vs. 0.83 | quiet |\n\nSo “quiet” gets emitted because of its tournament $g$ values. It’s clever because any single word looks like a standard part of the output, but with the secret key you can test whether emitted words repeatedly align with the keyed scores. It’s also quite nice because you can adjust the number of layers to change the tradeoff between detectability and model quality.\n\nSynthID is the only scheme that I know of (or ChatGPT/Claude knows of) which is running in production, and it seems like my best guess candidate given that it’s been publicized and run a scale by Google.\n\n# Experimental results\n\nTo see if Anthropic is running one of these kinds of statistical watermarks, I ran some of the black-box tests from the watermark detection literature with Codex GPT-5.6-sol[0].\n\n## Green/red list detection\n\nI had Codex run [Gloaguen et al.’s](https://www.sri.inf.ethz.ch/blog/probingsynthid) detection algorithm. In this, Claude is forced to complete sentences using one of four fruits:\n\n```\nI ate 111111111111111 peaches\nI chose 222222222222222 plums\n...\n```\n\nThe prefix and repeated digit vary while the fruit alternatives remain fixed. If a classic keyed watermark partitions tokens into green and red sets, then a particular fruit should become unusually likely or unlikely.\n\n| Model/test | Number of runs | Permutation p-value |\n|---|---|---|\n| Sonnet 5, compact digits | 1,800 | 0.677 |\n| Sonnet 5, spaced digits | 1,800 | 0.556 |\n| Fable 5 | 1,200 | 0.886 |\n\nThe Sonnet tests used 10 prefixes, 9 digit contexts, and 20 calls for each combination, for 1,800 responses each. The Fable test used 6 digit contexts, for 1,200 responses. Each p-value was calculated from 100,000 permutations.\n\nAll three were clear negative results. The Sonnet test was strong enough to rule out a fairly large, stable Green/red effect in this particular setup. It does not rule out a weaker watermark, a key that changes between requests, or a different kind of watermark entirely.\n\n## SynthID detection\n\nThe next experiment looked for the fixed context window used by SynthID-like schemes. The basic setup forced Claude to return a line such as\n\n```\nI ate red green cherries\n```\n\nHere `red`\n\nis a perturbation word and `green`\n\nis repeated $H$ times after it. If the watermark only looks at the last $h$ tokens, `red`\n\nshould stop affecting the watermark once $H$ reaches $h$. For example, if $h=2$, I would expect a strong effect at $H=1$ and little or no effect at $H=2$. So the thing we’re looking for is a sharp drop at some value of $H$.\n\nHere’s an actual example of the experiment I ran:\n\n| $H$ | Repeat | Actual response |\n|---|---|---|\n| 1 | 0 | `I ate red green cherries` |\n| 1 | 1 | `I ate red green cherries` |\n| 1 | 2 | `I ate red green cherries` |\n| 2 | 0 | `I ate red green green plums` |\n| 2 | 1 | `I ate red green green cherries` |\n| 2 | 2 | `I ate red green green cherries` |\n| 3 | 0 | `I ate red green green green cherries` |\n| 3 | 1 | `I ate red green green green cherries` |\n| 3 | 2 | `I ate red green green green plums` |\n\nAt larger scale, I calculated $Z(H)$, which measures how strongly changing the perturbation word changes Claude’s fruit choice. A value near zero would mean no detectable effect. $Z(1)=87.25$, which I saw in the experimental runs, means the test statistic was 87.25 (!!) standard deviations above what you would expect to see in randomized data:\n\n| $H$ | Context form | Standardized perturbation effect $Z(H)$ |\n|---|---|---|\n| 1 | `red green` | 87.25 |\n| 2 | `red green green` | 58.42 |\n| 3 | `red green green green` | 33.64 |\n| 4 | `red` + 4×`green` | 37.22 |\n| 5 | `red` + 5×`green` | 38.46 |\n| 6 | `red` + 6×`green` | 25.34 |\n| 7 | `red` + 7×`green` | 19.28 |\n| 8 | `red` + 8×`green` | 25.10 |\n\nThat is an extremely strong result, with $H=1$ giving $p\\approx0.00001$. Unfortunately, this test wasn’t conclusive because it only tells us that the perturbation word matters, and we didn’t actually see a sharp drop off on any $H$, only a slow decrease, which could mean that this isn’t a watermark at all, but rather just an effect that comes from the model.\n\nThe full scan used 20,736 Sonnet 5 responses, with 2,592 at each value of $H$. The sharpest bend was at $H=2$, so I tested it again using fresh Sonnet 5 responses and also on Sonnet 4.6 (which based on my reading of Anthropic’s help center article is less likely to be watermarked because it’s an older model):\n\n| Endpoint | Observations | $Z(1)$ | $Z(2)$ | $Z(3)$ | $D(2)$ |\n|---|---|---|---|---|---|\n| Sonnet 5, held-out confirmation | 7,776 | 85.77 | 61.90 | 33.35 | 38.14 |\n| Sonnet 4.6, matched comparison | 7,776 | 105.49 | 70.61 | 76.25 | 32.06 |\n\nThe bend appeared again in the fresh Sonnet 5 data. But the effect did not disappear at $H=2$: the $Z(2)$ and $Z(3)$ values were still enormous. Sonnet 4.6 also showed a very similar bend.\n\nThis makes the result much less exciting than the huge numbers initially suggest. Sonnet 4.6 might also be watermarked, so it is not a true negative control, but it does seem the pattern is not unique to Sonnet 5 and does not look like a clean context-window boundary. The most likely explanation is that words like `red`\n\nand `green`\n\nnaturally change how Claude chooses among fruits.\n\nSo this was a strong detection of a prompt effect, but not a positive detection of SynthID. It also doesn’t rule out a different watermark that this test cannot see.\n\n# Conclusions\n\nMy current best guess is that Anthropic is using a private-key watermark that changes token selection as Claude generates text. But that guess comes mostly from Anthropic’s description and negative results in other tests as opposed to a positive result in my experiments.\n\nI was able to rule out a few things as I found no evidence of hidden Unicode or whitespace, and the constrained-choice tests argue against a large, stable Green/red bias. That said, the apparent SynthID signal turned out to be a likely strong prompt effect that also appeared in Sonnet 4.6, so it doesn’t necessarily positively identify a watermark.\n\nNote that I’m not actually sure whether the watermarking rollout is fully complete yet and which models it’s available on. From Anthropic’s own help center article, it says that models are going to be watermarked going forward and that support for any existing model is “in progress”. I think I’ll have to re-run this analysis again in a few weeks or when there’s a verifiable model that does have watermarking enabled and is confirmed by Anthropic. We’ll just have to wait and see.\n\n# A note on the future of watermarking\n\nWhile this is mostly a technical post, I do think it’s worth thinking about what this potentially means for the future. It’s already relatively easy to detect when AI was used to write something and someone was careless. For example, I don’t need a statistical measure to figure out that this was written by an LLM: “That’s not a documentation problem — it’s a retrieval problem.”\n\nIn my view, putting statistical watermarks like the one described in this post on all LLM output greatly ratchets up the stakes from what is easily perceptible by humans, and in a way that is particularly undemocratic. You’ll only be able to detect the watermark if you’re in a select group that has access to a secret key (e.g. frontier lab employees or government / police). While this particular change is somewhat innocuous in my opinion, as I would assume most content written in the years after 2026 will be LLM generated or at least LLM assisted, it is a bit scary to know that a single relatively undemocratic, but innocuous change can give way to many more that may not be as innocuous. The EU transparency code that Anthropic is following has basically mandated that watermarking of text is required from model providers operating in the EU, so unfortunately, we should expect this to happen to a lot more of our model output.\n\nThankfully, sidestepping this kid of watermarking is fairly easy with a paraphraser or rephraser that doesn’t have a watermark (or just rewriting the text by hand). The watermarking is more meant to raise the cost and annoyance of doing so.\n\n# Footnotes\n\n[0] Of course, I tried to use Fable 5 for the analysis to start with, but it failed the security classifier and fell back to Opus 5 and I had to rely on the old trusty GPT-5.6-sol. This was probably better anyways as I’m not sure Claude would want itself to be self inspected.", "url": "https://wpnews.pro/news/how-claude-s-watermarking-probably-works", "canonical_source": "https://johnjwang.com/post/2026/08/12/how-claude-watermarking-probably-works/", "published_at": "2026-08-12 18:32:23+00:00", "updated_at": "2026-08-12 18:42:15.803264+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-policy", "ai-research"], "entities": ["Anthropic", "Claude", "Claude Code", "Kirchenbauer et al.", "Gloaguen et al.", "Codex"], "alternates": {"html": "https://wpnews.pro/news/how-claude-s-watermarking-probably-works", "markdown": "https://wpnews.pro/news/how-claude-s-watermarking-probably-works.md", "text": "https://wpnews.pro/news/how-claude-s-watermarking-probably-works.txt", "jsonld": "https://wpnews.pro/news/how-claude-s-watermarking-probably-works.jsonld"}}