# LLM Watermarking Detection is a Statistical Problem: Hypothesis-Testing

> Source: <https://pub.towardsai.net/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing-e113c5c680e6?source=rss----98111c9905da---4>
> Published: 2026-09-03 03:08:30+00:00

At the time of writing this LLMs watermarking is becoming significantly important. All major language models, including Claude, Gemini, ChatGPT are now adding a text watermark to comply with the EU AI Act’s transparency rules.

In short, LLM watermarking works by softly changing the token-generation process in a way that text produced by the model will have a detectable statistical signature, while keeping the text essence and meaning.

The LLM generation process works by estimating the next-token probability distribution using previous tokens:

A watermark works by modifying the sampling process (over the probability distribution) in a way that a regular reader can’t tell but is statistically detectable.

Let us suppose we have a vocabulary that is randomly divided:

This division can depend on a secret rule and the previous context.

In a normal setting, maybe:

P(green token) ≈ 0.5.

However, using a watermark, the generator might modify the probabilities in a way that green tokens are now more likely, like:

P(green token) ≈ 0.55.

Then, if we have a watermark detector that knows the secret key, we can make it then ask (using hypothesis testing):

“Does this text contain statistically significantly more green tokens than we would expect?”

So, let’s say the detector gets a document with 100 tokens and it contains 55 green tokens. Would that be evidence of a watermark in the text? Is that enough to belive that the text was generated by an LLM model?

The key is that a watermark cannot (or should not) be detected by the human eye. Instead, it is a **controlled statistical deviation in the distribution of the generated tokens that can be detected.**

So, we can formulate the detection problem like this:

Under H0, the number of green tokens G, in a document of n tokens, might be modeled as

where p0 is the probability of observing one green token in a document without watermark.

Then the detector can compute the test statistic:

A positive Z value might indicate that our document under study has more green tokens that we would expect on a text, which goes in favor of beliving H1: the text has a watermark.

Thus we can finish our detection process by computing the **p-value**: the probability, under the null hypothesis, of observing the test statistic at elast as the one obsererved in the real data:

where

under H0. The standardized statistic converges in distribution to a standard normal by the Central Limit Theorem (the sum of green tokens can be modeled as a sum of Bernoulli random variables).

We reject the null hypothests H0 if p-value < alpha. Where alpha is our significance level.

To finish this post I want to give a list of statistical concepts and some sources to learn about them:

[LLM Watermarking Detection is a Statistical Problem: Hypothesis-Testing](https://pub.towardsai.net/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing-e113c5c680e6) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.