Show HN: WMTrace – See how LLM text watermarks work (Anthropic just adopted one)

WMTrace, a new open-source forensic workbench for LLM text watermark detection, launched on Hacker News, offering scheme-aware analysis with evidence grades and abstention rather than opaque AI scores. The tool, which Anthropic recently adopted a watermark scheme from, includes detectors for KGW-style, unigram, zero-width, and lexical-codebook watermarks, and reports calibrated statistics such as z = 16.1 and Holm-adjusted p = 3.5e-48 in its demo.

Forensic detection, benchmarking, and explainable analysis of LLM text watermarks, provenance signals, and hidden-payload channels. WMTrace is a scheme-aware forensic workbench, not a monolithic "AI detector". It answers narrow, honest questions — does this text carry calibrated evidence for a declared watermark scheme, key, and configuration? — and it states, with every result, exactly what that result does not prove. Important A negative watermark result does not mean text was human-written. A recovered payload does not prove who embedded it. No result here supports provider attribution without trusted key provenance. pip install -e ". dev " from a clone; Python 3.10+ wmtrace serve polished web UI at http://127.0.0.1:8177 The Analyze tab on a generated watermarked sample: summary pills per detector, calibrated statistics z = 16.1, Holm-adjusted p = 3.5e-48 , and the keyed green-token heatmap. Reproduce it locally with wmtrace serve → http://127.0.0.1:8177/?demo=1. Evidence, not verdicts. Results are versioned evidence bundles with categorical grades E0–E5 and mandatory scope statements — never an opaque "97% AI" score. Four problems, four namespaces. Watermark detection, generic AI-text detection, model attribution, and provenance verification stay separate in code, API, and UI, and are never averaged together. Abstention is an answer. Detectors declare unsupported E0 or insufficient E1 before scoring rather than guessing on short, low-entropy, or incompatible input. No guessed provider secrets. There is no anthropic or openai detector here and never will be until an official verification API or reproducible public specification exists. All keys in this build are published research keys. Executable documentation. Every worked example in the research spec /gtesei/llm-watermark/blob/main/docs/RESEARCH.md is reproduced exactly by golden tests — the docs cannot drift from the code. | Detector | Scheme | Claim status | Keyed? | Max grade | Round-trip demo | |---|---|---|---|---|---| kgw-toy | Contextual green-list KGW-style , γ=0.25, 2-word context | ILLUSTRATIVE | ✔ published demo key | E3 | ✔ green-biased generator | unigram-toy | Fixed keyed green-list | ILLUSTRATIVE | ✔ published demo key | E3 | ✔ green-biased generator | zero-width | Invisible Unicode payload U+200B=0, U+200C=1 | ILLUSTRATIVE | ✘ unkeyed | E2 | ✔ encoder | lexical-codebook | Synonym-pair payload bits | ILLUSTRATIVE | ✘ public codebook | E2 | ✔ encoder | unicode-inspector | Zero-width / bidi / unusual-space / mixed-script indicators | IMPLEMENTATION | ✘ keyless | E2 | — | Model-based detectors SynthID-Text reference, Fast-DetectGPT, Binoculars , C2PA verification, and provider adapters are roadmap items — see the research spec /gtesei/llm-watermark/blob/main/docs/RESEARCH.md for the full plan and the reasoning. wmtrace serve opens a responsive, dark/light, mobile-friendly interface: Analyze — paste text, get evidence cards per detector: grade badge, decision, exact statistics N, green count, z, exact binomial p, Holm-adjusted p , a per-token green heatmap, Unicode findings, and a "what this result means" scope statement. Embed — generate green-biased sample text under the demo key, or hide zero-width / lexical payloads in your own text, then round-trip it into the analyzer with one click. Invisible characters can be revealed inline. About — evidence-grade table, detector manifests, and responsible-interpretation guidance. Query parameters: ?demo=1 auto-loads a watermarked sample and analyzes it; ?theme=light|dark forces a theme used for the screenshots above . Run all detectors Holm-corrected across the run wmtrace scan document.txt wmtrace scan document.txt --json full evidence bundles One declared scheme wmtrace detect --scheme kgw-toy document.txt Unicode and formatting inspection only wmtrace inspect document.txt Demo embedding: generate a watermarked sample, reproducibly wmtrace embed --scheme kgw-toy --length 150 --seed 11 marked.txt wmtrace detect --scheme kgw-toy marked.txt Hide an ASCII payload in invisible Unicode echo "Meet at the usual place." | wmtrace embed --scheme zero-width --payload Hi --ascii php flowchart LR I Input text -- H Hash + explicit representations<br/ raw / NFC / NFKC H -- C Capability check<br/ E0 unsupported · E1 insufficient C -- D Detector plugins<br/ score → calibrate → explain D -- S Statistics<br/ exact binomial · z · Holm S -- E Evidence bundles<br/ grade + scope statement E -- O1 JSON & O2 Terminal & O3 Web UI Each detector implements the five-method plugin contract manifest / capabilities / score / calibrate / explain ; the shared statistics layer owns significance testing, so no detector rolls its own. Normalization is never silent — every bundle names the representation it analyzed. | Grade | Meaning | |---|---| E0 | Unsupported: required key, tokenizer, or configuration unavailable | E1 | Insufficient: compatible, but too little usable signal | E2 | Heuristic: indicator-level, unkeyed or uncalibrated channel | E3 | Scheme evidence: calibrated evidence for a declared scheme/key/config | E4 | Provider-bound: trusted key provenance binds the result no detector in this build can reach E4 | E5 | Cryptographically verified reserved for future provenance adapters | A grade is an evidence category, not a posterior probability of authorship. git clone https://github.com/gtesei/llm-watermark && cd llm-watermark python -m venv .venv && . .venv/bin/activate or: uv venv pip install -e ". dev " or: uv pip install -e ". dev " pytest 41 tests: golden examples, calibration, round-trips, API pytest tests/test golden examples.py -v the research spec's examples, executable wmtrace serve web UI Layout: src/wmtrace/{core,statistics,detectors,web} + cli.py ; tests in tests/ ; the full research spec taxonomy, provider status, benchmark methodology, roadmap in docs/RESEARCH.md /gtesei/llm-watermark/blob/main/docs/RESEARCH.md ; release and design history in . /gtesei/llm-watermark/blob/main/CHANGELOG.md CHANGELOG.md - Never use a detector score as sole evidence for punishment or accusation. - Watermark evidence indicates compatibility with a declared scheme — not authorship, intent, factual accuracy, or provider identity. - The demo keys in this repository are published and illustrative; results under them bind nothing to any provider. - Prefer abstention over a forced binary answer; report minimum length and effective sample size. Recommended report scope statement: This result provides evidence compatible with watermark scheme X under configuration Y. It does not by itself prove human or machine authorship, provider identity without trusted key provenance, factual accuracy, ownership, or intent. The repository began as a research design document; it is preserved in full at docs/RESEARCH.md /gtesei/llm-watermark/blob/main/docs/RESEARCH.md , covering the watermarking technique taxonomy KGW, unigram, unbiased reweighting, ITS/EXP/Gumbel, SynthID-Text, semantic and multi-bit schemes, cryptographic marks , provider status as of 2026-08-14 Anthropic, OpenAI, Google, Meta , attack and robustness analysis, benchmark methodology, and the P0–P6 implementation roadmap with citations. Apache-2.0.