Show HN: WMTrace – See how LLM text watermarks work (Anthropic just adopted one)

Forensic detection, benchmarking, and explainable analysis of LLM text watermarks, provenance signals, and hidden-payload channels.

WMTrace is a scheme-aware forensic workbench, not a monolithic "AI detector". It answers narrow, honest questions — does this text carry calibrated evidence for a declared watermark scheme, key, and configuration? — and it states, with every result, exactly what that result does not prove.

Important

A negative watermark result does not mean text was human-written. A recovered payload does not prove who embedded it. No result here supports provider attribution without trusted key provenance.

pip install -e ".[dev]"     # from a clone; Python 3.10+
wmtrace serve               # polished web UI at http://127.0.0.1:8177

The Analyze tab on a generated watermarked sample: summary pills per detector, calibrated statistics (z = 16.1, Holm-adjusted p = 3.5e-48), and the keyed green-token heatmap. Reproduce it locally with wmtrace serve → http://127.0.0.1:8177/?demo=1.

Evidence, not verdicts. Results are versioned evidence bundles with categorical grades (E0–E5) and mandatory scope statements — never an opaque "97% AI" score.Four problems, four namespaces. Watermark detection, generic AI-text detection, model attribution, and provenance verification stay separate in code, API, and UI, and are never averaged together.Abstention is an answer. Detectors declareunsupported

(E0) orinsufficient

(E1) before scoring rather than guessing on short, low-entropy, or incompatible input.No guessed provider secrets. There is noanthropic

oropenai

detector here and never will be until an official verification API or reproducible public specification exists. All keys in this build are published research keys.Executable documentation. Every worked example in theresearch specis reproduced exactly by golden tests — the docs cannot drift from the code.

Model-based detectors (SynthID-Text reference, Fast-DetectGPT, Binoculars), C2PA verification, and provider adapters are roadmap items — see the research spec for the full plan and the reasoning.

wmtrace serve

opens a responsive, dark/light, mobile-friendly interface:

Analyze— paste text, get evidence cards per detector: grade badge, decision, exact statistics (N, green count, z, exact binomial p, Holm-adjusted p), a per-token green heatmap, Unicode findings, and a "what this result means" scope statement.Embed— generate green-biased sample text under the demo key, or hide zero-width / lexical payloads in your own text, then round-trip it into the analyzer with one click. Invisible characters can be revealed inline.About— evidence-grade table, detector manifests, and responsible-interpretation guidance.

Query parameters: ?demo=1

auto-loads a watermarked sample and analyzes it; ?theme=light|dark

forces a theme (used for the screenshots above).

wmtrace scan document.txt
wmtrace scan document.txt --json          # full evidence bundles

wmtrace detect --scheme kgw-toy document.txt

wmtrace inspect document.txt

wmtrace embed --scheme kgw-toy --length 150 --seed 11 > marked.txt
wmtrace detect --scheme kgw-toy marked.txt

echo "Meet at the usual place." | wmtrace embed --scheme zero-width --payload Hi --ascii
php
flowchart LR
    I[Input text] --> H[Hash + explicit representations<br/>raw / NFC / NFKC]
    H --> C[Capability check<br/>E0 unsupported · E1 insufficient]
    C --> D[Detector plugins<br/>score → calibrate → explain]
    D --> S[Statistics<br/>exact binomial · z · Holm]
    S --> E[Evidence bundles<br/>grade + scope statement]
    E --> O1[JSON] & O2[Terminal] & O3[Web UI]

Each detector implements the five-method plugin contract (manifest / capabilities / score / calibrate / explain

); the shared statistics layer owns significance testing, so no detector rolls its own. Normalization is never silent — every bundle names the representation it analyzed.

A grade is an evidence category, not a posterior probability of authorship.

git clone https://github.com/gtesei/llm-watermark && cd llm-watermark
python -m venv .venv && . .venv/bin/activate   # or: uv venv
pip install -e ".[dev]"                        # or: uv pip install -e ".[dev]"

pytest                    # 41 tests: golden examples, calibration, round-trips, API
pytest tests/test_golden_examples.py -v   # the research spec's examples, executable

wmtrace serve             # web UI

Layout: src/wmtrace/{core,statistics,detectors,web}

• cli.py

; tests in tests/

; the full research spec (taxonomy, provider status, benchmark methodology, roadmap) in docs/RESEARCH.md; release and design history in

.

CHANGELOG.md

• Never use a detector score as sole evidence for punishment or accusation.
• Watermark evidence indicates compatibility with a declared scheme — not authorship, intent, factual accuracy, or provider identity.
• The demo keys in this repository are published and illustrative; results under them bind nothing to any provider.
• Prefer abstention over a forced binary answer; report minimum length and effective sample size.

Recommended report scope statement:

This result provides evidence compatible with watermark scheme X under configuration Y. It does not by itself prove human or machine authorship, provider identity without trusted key provenance, factual accuracy, ownership, or intent.

The repository began as a research design document; it is preserved in full at docs/RESEARCH.md, covering the watermarking technique taxonomy (KGW, unigram, unbiased reweighting, ITS/EXP/Gumbel, SynthID-Text, semantic and multi-bit schemes, cryptographic marks), provider status as of 2026-08-14 (Anthropic, OpenAI, Google, Meta), attack and robustness analysis, benchmark methodology, and the P0–P6 implementation roadmap with citations.

Apache-2.0.