{"slug": "llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing", "title": "LLM Watermarking Detection is a Statistical Problem: Hypothesis-Testing", "summary": "LLM watermarking detection is fundamentally a statistical hypothesis-testing problem, according to a technical post on Towards AI. The article explains that watermarking modifies token-generation probabilities so that watermarked text contains a statistically detectable excess of 'green' tokens, and detection relies on computing a p-value and rejecting the null hypothesis when p < alpha. Major language models including Claude, Gemini, and ChatGPT are adopting watermarks to comply with the EU AI Act's transparency rules.", "body_md": "At the time of writing this LLMs watermarking is becoming significantly important. All major language models, including Claude, Gemini, ChatGPT are now adding a text watermark to comply with the EU AI Act’s transparency rules.\n\nIn short, LLM watermarking works by softly changing the token-generation process in a way that text produced by the model will have a detectable statistical signature, while keeping the text essence and meaning.\n\nThe LLM generation process works by estimating the next-token probability distribution using previous tokens:\n\nA watermark works by modifying the sampling process (over the probability distribution) in a way that a regular reader can’t tell but is statistically detectable.\n\nLet us suppose we have a vocabulary that is randomly divided:\n\nThis division can depend on a secret rule and the previous context.\n\nIn a normal setting, maybe:\n\nP(green token) ≈ 0.5.\n\nHowever, using a watermark, the generator might modify the probabilities in a way that green tokens are now more likely, like:\n\nP(green token) ≈ 0.55.\n\nThen, if we have a watermark detector that knows the secret key, we can make it then ask (using hypothesis testing):\n\n“Does this text contain statistically significantly more green tokens than we would expect?”\n\nSo, let’s say the detector gets a document with 100 tokens and it contains 55 green tokens. Would that be evidence of a watermark in the text? Is that enough to belive that the text was generated by an LLM model?\n\nThe key is that a watermark cannot (or should not) be detected by the human eye. Instead, it is a **controlled statistical deviation in the distribution of the generated tokens that can be detected.**\n\nSo, we can formulate the detection problem like this:\n\nUnder H0, the number of green tokens G, in a document of n tokens, might be modeled as\n\nwhere p0 is the probability of observing one green token in a document without watermark.\n\nThen the detector can compute the test statistic:\n\nA positive Z value might indicate that our document under study has more green tokens that we would expect on a text, which goes in favor of beliving H1: the text has a watermark.\n\nThus we can finish our detection process by computing the **p-value**: the probability, under the null hypothesis, of observing the test statistic at elast as the one obsererved in the real data:\n\nwhere\n\nunder H0. The standardized statistic converges in distribution to a standard normal by the Central Limit Theorem (the sum of green tokens can be modeled as a sum of Bernoulli random variables).\n\nWe reject the null hypothests H0 if p-value < alpha. Where alpha is our significance level.\n\nTo finish this post I want to give a list of statistical concepts and some sources to learn about them:\n\n[LLM Watermarking Detection is a Statistical Problem: Hypothesis-Testing](https://pub.towardsai.net/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing-e113c5c680e6) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing", "canonical_source": "https://pub.towardsai.net/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing-e113c5c680e6?source=rss----98111c9905da---4", "published_at": "2026-09-03 03:08:30+00:00", "updated_at": "2026-09-03 03:21:45.836126+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-policy", "ai-research"], "entities": ["Claude", "Gemini", "ChatGPT", "EU AI Act", "Towards AI"], "alternates": {"html": "https://wpnews.pro/news/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing", "markdown": "https://wpnews.pro/news/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing.md", "text": "https://wpnews.pro/news/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing.txt", "jsonld": "https://wpnews.pro/news/llm-watermarking-detection-is-a-statistical-problem-hypothesis-testing.jsonld"}}