{"slug": "llm-watermarking-isn-t-about-visible-text-or-hidden-ads", "title": "LLM watermarking isn't about visible text or hidden ads", "summary": "A developer built a minimal, educational version of Google's SynthID-Text style watermarking system, demonstrating how token probabilities are manipulated to embed a detectable signal in LLM output. The implementation, available on GitHub, splits the vocabulary into green and red lists based on a hash of the previous token, boosting green token probabilities during generation, and uses a hypothesis test to detect statistically improbable green token concentrations. The project illustrates the trade-off between watermark strength and model utility, where excessive bias degrades text quality and insufficient bias makes the watermark easy to strip.", "body_md": "# LLM watermarking isn't about visible text or hidden ads\n\nI decided to build a minimal, educational version of a SynthID-Text style watermarking system to see if I could actually wrap my head around the math. It isn't a 1:1 replica of Google's proprietary SynthID system, but the core logic of how you manipulate token probabilities to embed a signal is all there.\n\n## The core concept: Red/Green lists\n\nThe fundamental trick involves splitting the vocabulary into two sets—let's call them \"green\" and \"red\" lists—based on a pseudo-random hash of the previous token. When the model is deciding which token to generate next, you artificially boost the probability of tokens from the \"green\" list.\n\nIf a piece of text is purely human-written, the distribution of tokens should be relatively \"natural.\" However, if the text was generated by a watermarked model, you will see a statistically impossible concentration of \"green\" tokens. A detector can then run a hypothesis test to see if the frequency of these green tokens deviates significantly from what we'd expect by chance.\n\n## A simplified implementation approach\n\nTo make this work in a practical tutorial style, you basically need to intercept the logits (the raw scores before softmax) during the inference step. Here is a high-level breakdown of the workflow:\n\n1. **Generate a pseudo-random sequence:** Use the previous token's ID as a seed for a hash function to determine which tokens in the current vocabulary belong to the \"green\" list.\n\n2. **Modify the logits:** Add a small constant (a \"bias\" value) to the logits of all tokens in the green list.\n\n3. **Sample the token:** Perform the standard softmax and sampling. Because the green tokens now have higher scores, the model is much more likely to pick them.\n\n4. **Detection:** To verify, you don't look at the words; you look at the ratio of green tokens to red tokens across a long string of text.\n\nI put together a repository that demonstrates this logic from scratch. It’s a great way to understand the tension between \"watermark strength\" (how easy it is to detect) and \"model utility\" (how much the watermark ruins the actual quality of the text). If you push the bias too high, the model starts talking nonsense; if you keep it too low, the watermark is too easy to strip away with simple rephrasing.\n\nIf you want to poke around the code and see the actual math in action, you can find the implementation here:\n\n```\nhttps://github.com/Saad1926Q/llm-watermark\n```\n\n[Anthropic might list AI backlash as a major risk in their IPO 22h ago](/en/news/7337/)\n\n[Is the AI rally a genuine productivity boom or a 1d ago](/en/news/7282/)\n\n[Anthropic quietly rewrites its enterprise data retention rules 1d ago](/en/news/7199/)\n\n[The consciousness debate feels like a distraction tactic 1d ago](/en/news/7184/)\n\n[Stripe buying OpenRouter puts payments under every model call 2d ago](/en/news/7128/)\n\n[Why I still lose sleep over alignment even though I build with 2d ago](/en/news/7118/)\n\n[Next Stop letting your old TVs collect dust when they can act as a →](/en/news/7415/)\n\n[these real-world AI monetization case studies](https://tanyan888.com/), with plenty of directly applicable cases.", "url": "https://wpnews.pro/news/llm-watermarking-isn-t-about-visible-text-or-hidden-ads", "canonical_source": "https://promptcube3.com/en/news/7418/", "published_at": "2026-08-23 14:51:57+00:00", "updated_at": "2026-08-23 15:12:40.853876+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools"], "entities": ["Google", "SynthID", "GitHub", "Saad1926Q"], "alternates": {"html": "https://wpnews.pro/news/llm-watermarking-isn-t-about-visible-text-or-hidden-ads", "markdown": "https://wpnews.pro/news/llm-watermarking-isn-t-about-visible-text-or-hidden-ads.md", "text": "https://wpnews.pro/news/llm-watermarking-isn-t-about-visible-text-or-hidden-ads.txt", "jsonld": "https://wpnews.pro/news/llm-watermarking-isn-t-about-visible-text-or-hidden-ads.jsonld"}}