Anthropic just explained how Claude’s text watermark works, and the approach is pretty neat. Here is how it works…
When Claude announced this, I was super keen to understand how they would implement this, especially for text. Images are pretty straightforward to watermark, but text was an interesting problem.
Every time an LLM generates a token, it is choosing from a probability distribution. Often, several candidates are nearly tied. For example…
“The weather today was cold and…” could just as easily continue with “overcast” or “grey”. Neither is more correct. Normally, a plain random number generator breaks that tie.
The watermark changes only one thing - where that randomness comes from.
Instead of an arbitrary random number, the model derives its random value from a secret key plus the tokens that came right before. The token picked still looks like an ordinary random choice to us.
But anyone holding the key can replay that same derivation on a piece of text and check whether the sequence of choices matches what the key would produce. The better the match, the higher the probability that Claude wrote it.
Pretty neat.