{"slug": "a-minimal-implementation-of-llm-output-watermarking", "title": "A minimal implementation of LLM output watermarking", "summary": "Andrej Karpathy's minimal microgpt has been extended with a secret-keyed Gumbel sampler that watermarks LLM output without altering token probabilities, achieving a p-value of 6.844e-53 for watermarked text versus ~0.6-0.75 for normal or wrong-key text. The pure-Python implementation, released by developer berba-q on GitHub, requires no dependencies and demonstrates statistically detectable watermarking in 91 lines of code.", "body_md": "Experiment gpt watermarking based on Karpathy's microgpt\n\nBuilt on Andrej Karpathy's minimal `microgpt`\n\nto show exactly how a secret-keyed sampler can create a statistically detectable watermark without changing the model's underlying token probabilities.\n\n- Pure python\n- No dependency libraries\n- Few lines of code\n\nAn LLM generates text by sampling the next token from a probability distribution.\n\nNormal generation:\n\nGPT → probabilities → random sampling → next token\n\nWatermarked generation:\n\nGPT → same probabilities → keyed Gumbel sampling → next token\n\nThe watermark is not a visible marker in the text. It emerges statistically across many token choices and can be detected using the same secret key.\n\n```\ngit clone https://github.com/berba-q/gpt-watermark\ncd gpt-watermark\npython3 microgpt_watermark.py\n```\n\nNo dependencies beyond the Python 3 standard library.\n\nSame tiny GPT, same probabilities, two samplers — the text looks equally natural either way:\n\n```\nyuh | normal: yuha           | watermarked: yuhan\nxav | normal: xavinn         | watermarked: xavia\njua | normal: juan           | watermarked: juale\n```\n\nBut the detector, holding the secret key, tells them apart with overwhelming confidence:\n\n```\nnormal       n=308  mean=0.962  p=7.459e-01\nwatermarked  n=322  mean=2.105  p=6.844e-53\nwrong key    n=322  mean=0.984  p=6.094e-01\n```\n\nWatermarked text scores a p-value of ~6.8×10⁻⁵³; ordinary text and text checked with the wrong key both land around p≈0.6-0.75 indistinguishable from chance.\n\nWant the full explanation? [Read: Watermarking a tiny GPT: 91 lines, NO Frameworks](https://berba-q.github.io/blog/llm-watermark-microgpt.html).\n\nThis project builds on Andrej Karpathy's excellent [ microgpt](https://karpathy.github.io/2026/02/12/microgpt/), which provides the minimal GPT implementation used here. The watermarking experiment is inspired by\n\n[Scott Aaronson's work](https://scottaaronson.blog/?p=9333)on keyed Gumbel sampling for LLM output watermarking.", "url": "https://wpnews.pro/news/a-minimal-implementation-of-llm-output-watermarking", "canonical_source": "https://github.com/berba-q/gpt-watermark", "published_at": "2026-08-25 07:38:36+00:00", "updated_at": "2026-08-25 08:14:34.165889+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-safety", "ai-tools"], "entities": ["Andrej Karpathy", "microgpt", "berba-q", "gpt-watermark", "Scott Aaronson", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/a-minimal-implementation-of-llm-output-watermarking", "markdown": "https://wpnews.pro/news/a-minimal-implementation-of-llm-output-watermarking.md", "text": "https://wpnews.pro/news/a-minimal-implementation-of-llm-output-watermarking.txt", "jsonld": "https://wpnews.pro/news/a-minimal-implementation-of-llm-output-watermarking.jsonld"}}