# Show HN: WMTrace – See how LLM text watermarks work (Anthropic just adopted one)

> Source: <https://github.com/gtesei/llm-watermark>
> Published: 2026-08-15 02:15:26+00:00

Forensic detection, benchmarking, and explainable analysis of LLM text watermarks, provenance signals, and hidden-payload channels.

WMTrace is a scheme-aware forensic workbench, not a monolithic "AI detector". It answers narrow, honest questions — *does this text carry calibrated evidence for a declared watermark scheme, key, and configuration?* — and it states, with every result, exactly what that result does **not** prove.

Important

A negative watermark result does **not** mean text was human-written. A recovered payload does **not** prove who embedded it. No result here supports provider attribution without trusted key provenance.

```
pip install -e ".[dev]"     # from a clone; Python 3.10+
wmtrace serve               # polished web UI at http://127.0.0.1:8177
```

*The Analyze tab on a generated watermarked sample: summary pills per detector, calibrated statistics (z = 16.1, Holm-adjusted p = 3.5e-48), and the keyed green-token heatmap. Reproduce it locally with wmtrace serve → http://127.0.0.1:8177/?demo=1.*

**Evidence, not verdicts.** Results are versioned evidence bundles with categorical grades (E0–E5) and mandatory scope statements — never an opaque "97% AI" score.**Four problems, four namespaces.** Watermark detection, generic AI-text detection, model attribution, and provenance verification stay separate in code, API, and UI, and are never averaged together.**Abstention is an answer.** Detectors declare`unsupported`

(E0) or`insufficient`

(E1) before scoring rather than guessing on short, low-entropy, or incompatible input.**No guessed provider secrets.** There is no`anthropic`

or`openai`

detector here and never will be until an official verification API or reproducible public specification exists. All keys in this build are published research keys.**Executable documentation.** Every worked example in the[research spec](/gtesei/llm-watermark/blob/main/docs/RESEARCH.md)is reproduced exactly by golden tests — the docs cannot drift from the code.

| Detector | Scheme | Claim status | Keyed? | Max grade | Round-trip demo |
|---|---|---|---|---|---|
`kgw-toy` |
Contextual green-list (KGW-style), γ=0.25, 2-word context | `[ILLUSTRATIVE]` |
✔ published demo key | E3 | ✔ green-biased generator |
`unigram-toy` |
Fixed keyed green-list | `[ILLUSTRATIVE]` |
✔ published demo key | E3 | ✔ green-biased generator |
`zero-width` |
Invisible Unicode payload (U+200B=0, U+200C=1) | `[ILLUSTRATIVE]` |
✘ unkeyed | E2 | ✔ encoder |
`lexical-codebook` |
Synonym-pair payload bits | `[ILLUSTRATIVE]` |
✘ public codebook | E2 | ✔ encoder |
`unicode-inspector` |
Zero-width / bidi / unusual-space / mixed-script indicators | `[IMPLEMENTATION]` |
✘ keyless | E2 | — |

Model-based detectors (SynthID-Text reference, Fast-DetectGPT, Binoculars), C2PA verification, and provider adapters are roadmap items — see the [research spec](/gtesei/llm-watermark/blob/main/docs/RESEARCH.md) for the full plan and the reasoning.

`wmtrace serve`

opens a responsive, dark/light, mobile-friendly interface:

**Analyze**— paste text, get evidence cards per detector: grade badge, decision, exact statistics (N, green count, z, exact binomial p, Holm-adjusted p), a per-token green heatmap, Unicode findings, and a "what this result means" scope statement.**Embed**— generate green-biased sample text under the demo key, or hide zero-width / lexical payloads in your own text, then round-trip it into the analyzer with one click. Invisible characters can be revealed inline.**About**— evidence-grade table, detector manifests, and responsible-interpretation guidance.

Query parameters: `?demo=1`

auto-loads a watermarked sample and analyzes it; `?theme=light|dark`

forces a theme (used for the screenshots above).

```
# Run all detectors (Holm-corrected across the run)
wmtrace scan document.txt
wmtrace scan document.txt --json          # full evidence bundles

# One declared scheme
wmtrace detect --scheme kgw-toy document.txt

# Unicode and formatting inspection only
wmtrace inspect document.txt

# Demo embedding: generate a watermarked sample, reproducibly
wmtrace embed --scheme kgw-toy --length 150 --seed 11 > marked.txt
wmtrace detect --scheme kgw-toy marked.txt

# Hide an ASCII payload in invisible Unicode
echo "Meet at the usual place." | wmtrace embed --scheme zero-width --payload Hi --ascii
php
flowchart LR
    I[Input text] --> H[Hash + explicit representations<br/>raw / NFC / NFKC]
    H --> C[Capability check<br/>E0 unsupported · E1 insufficient]
    C --> D[Detector plugins<br/>score → calibrate → explain]
    D --> S[Statistics<br/>exact binomial · z · Holm]
    S --> E[Evidence bundles<br/>grade + scope statement]
    E --> O1[JSON] & O2[Terminal] & O3[Web UI]
```

Each detector implements the five-method plugin contract (`manifest / capabilities / score / calibrate / explain`

); the shared statistics layer owns significance testing, so no detector rolls its own. Normalization is never silent — every bundle names the representation it analyzed.

| Grade | Meaning |
|---|---|
E0 |
Unsupported: required key, tokenizer, or configuration unavailable |
E1 |
Insufficient: compatible, but too little usable signal |
E2 |
Heuristic: indicator-level, unkeyed or uncalibrated channel |
E3 |
Scheme evidence: calibrated evidence for a declared scheme/key/config |
E4 |
Provider-bound: trusted key provenance binds the result (no detector in this build can reach E4) |
E5 |
Cryptographically verified (reserved for future provenance adapters) |

A grade is an evidence category, **not** a posterior probability of authorship.

```
git clone https://github.com/gtesei/llm-watermark && cd llm-watermark
python -m venv .venv && . .venv/bin/activate   # or: uv venv
pip install -e ".[dev]"                        # or: uv pip install -e ".[dev]"

pytest                    # 41 tests: golden examples, calibration, round-trips, API
pytest tests/test_golden_examples.py -v   # the research spec's examples, executable

wmtrace serve             # web UI
```

Layout: `src/wmtrace/{core,statistics,detectors,web}`

+ `cli.py`

; tests in `tests/`

; the full research spec (taxonomy, provider status, benchmark methodology, roadmap) in [ docs/RESEARCH.md](/gtesei/llm-watermark/blob/main/docs/RESEARCH.md); release and design history in

[.](/gtesei/llm-watermark/blob/main/CHANGELOG.md)

`CHANGELOG.md`

- Never use a detector score as sole evidence for punishment or accusation.
- Watermark evidence indicates compatibility with a declared scheme — not authorship, intent, factual accuracy, or provider identity.
- The demo keys in this repository are published and illustrative; results under them bind nothing to any provider.
- Prefer abstention over a forced binary answer; report minimum length and effective sample size.

Recommended report scope statement:

This result provides evidence compatible with watermark scheme X under configuration Y. It does not by itself prove human or machine authorship, provider identity without trusted key provenance, factual accuracy, ownership, or intent.

The repository began as a research design document; it is preserved in full at [ docs/RESEARCH.md](/gtesei/llm-watermark/blob/main/docs/RESEARCH.md), covering the watermarking technique taxonomy (KGW, unigram, unbiased reweighting, ITS/EXP/Gumbel, SynthID-Text, semantic and multi-bit schemes, cryptographic marks), provider status as of 2026-08-14 (Anthropic, OpenAI, Google, Meta), attack and robustness analysis, benchmark methodology, and the P0–P6 implementation roadmap with citations.

Apache-2.0.