AI Watermark removers can't verify themselves I built the tool that can

Developer Neeeophytee released a seven-skill agent pack, 'ai-watermarks-reality-check,' that audits AI provenance evidence such as C2PA metadata and text watermarks without removing them, reporting statuses like PRESENT, VALID, and ABSENT. The pack, compatible with Claude Code, Codex, Gemini CLI, and MCP clients, addresses gaps in Anthropic's newly launched Claude watermarking, which lacks a public detector and detailed specification, and notes that common publishing actions can strip metadata. It requires Node.js 22.20+ for installation and Python 3.9+ for the skills.

Seven portable agent skills for checking what AI provenance evidence exists, whether it verifies, what a publishing pipeline destroys, and what must still be disclosed. The pack deliberately does not remove watermarks or Content Credentials. It makes provenance observable without pretending that one signal can prove authorship. Anthropic says newly launched Claude models are gaining imperceptible text watermarks and supported image files can carry signed C2PA metadata. Its public detector and detailed text-watermark specification are not yet available. Meanwhile, common publishing actions such as resizing, screenshots, conversion, and social uploads can remove file metadata. This pack gives teams a reproducible workflow: inspect → verify → transform-test → privacy-audit → disclosure-check The signed original reports PRESENT , VALID , and PRESERVED VALID . The downloaded and post-tool copies report ABSENT , NOT VERIFIED , and LOST OR UNAVAILABLE . This comparison covers the C2PA provenance layer. It does not test proprietary pixel watermarks or keyed text watermarks. Watch the 34-second landscape demo /Neeeophytee/ai-watermarks-reality-check/blob/main/media/watermark-two-sides-34s-1080p.mp4 or open the vertical social cut /Neeeophytee/ai-watermarks-reality-check/blob/main/media/watermark-two-sides-vertical.mp4 . The skills work with Claude Code https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview , Codex https://learn.chatgpt.com/docs/build-skills , Gemini CLI https://codelabs.developers.google.com/gemini-cli/how-to-create-agent-skills-for-gemini-cli , and other Agent Skills-compatible tools. Use them to audit supported evidence in files and text from Claude, ChatGPT, Gemini, open models, image generators, or any other source. Coverage follows the evidence type rather than a vendor name. C2PA, metadata, disclosure records, pipeline survival, and hidden Unicode channels can be inspected wherever they appear. There is no universal detector for every proprietary or secret-key watermark. When an official detector, configuration, or key is unavailable, the result remains UNVERIFIABLE , UNSUPPORTED , or UNKNOWN . | Skill | Use it to | |---|---| audit-provenance | Start here. Answer the five questions for one asset in one pass | inspect-content-provenance | Locate C2PA structure and hidden text channels without overclaiming | verify-content-credentials | Validate a C2PA manifest and report integrity separately from signer trust | map-provenance-survival | Batch edited copies and create a shareable C2PA survival report | audit-metadata-privacy | Inventory GPS, author, device, software, and date metadata without changing the source | check-ai-transparency | Check whether evidence and human-readable disclosure are ready for review | detect-text-watermark | Run every available text-provenance detector and report what cannot be checked | npx skills add Neeeophytee/ai-watermarks-reality-check The current skills installer requires Node.js 22.20 or newer. The installed skills themselves require only Python 3.9 or newer. Or copy one directory from skills/ into your agent's skill directory. Each skill folder is self-contained: the shared parsing core is vendored into every scripts/ directory, so a single folder works on its own. For MCP-compatible clients, all seven analyzers can also be exposed as local stdio tools: { "mcpServers": { "provenance": { "command": "python3", "args": "/absolute/path/to/mcp/server.py" } } } No hosted service is required. The MCP client starts python3 mcp/server.py on the user's machine and exchanges JSON-RPC messages with it through standard input and output. The server dispatches calls to the same read-only analyzers, reads local files, and returns structured results. Cryptographic verification uses the user's local c2patool installation. Remote-manifest access stays disabled unless the caller explicitly enables it. The server supports two protocol eras. It implements the current stateless revision 2026-07-28 and still answers the legacy initialize handshake used by 2025-11-25 and earlier clients. | Behaviour | Support | |---|---| server/discover mandatory in 2026-07-28 | Implemented | Per-request meta "io.modelcontextprotocol/protocolVersion" | Honoured | UnsupportedProtocolVersionError -32022 with supported / requested | Returned | Legacy initialize negotiation | Retained | Every script uses the Python standard library and requires Python 3.9 or newer. Cryptographic C2PA verification additionally requires the official c2patool 0.20.0 or newer . Start with the front door, which answers all five questions in one pass: python3 skills/audit-provenance/scripts/audit provenance.py image.png \ --c2patool /path/to/c2patool \ --trust-anchors /path/to/policy.pem The individual analyzers remain directly available: python3 skills/inspect-content-provenance/scripts/inspect file.py image.png python3 skills/verify-content-credentials/scripts/verify c2pa.py image.png python3 skills/map-provenance-survival/scripts/map survival.py \ --original original.png \ --derivatives-dir transformed-copies/ \ --derivative social-download:platform-roundtrip=downloaded.jpg \ --c2patool /path/to/c2patool \ --report survival-report.html python3 skills/audit-metadata-privacy/scripts/audit metadata.py image.png python3 skills/check-ai-transparency/scripts/check transparency.py \ examples/transparency-record.json python3 skills/detect-text-watermark/scripts/detect text watermark.py draft.md Scripts emit JSON to stdout and diagnostics to stderr. Remote-manifest fetching is disabled by default and requires --allow-network . A missing verifier, an unsupported verifier version, an inaccessible remote manifest, or an unpublished detector produces an explicit unknown state rather than a pass. Survival reports refuse to overwrite an existing file. They show only filenames by default, while the machine-readable JSON retains the full reproducibility record. Use --include-paths only when the report will stay private. A generated report never claims that a proprietary watermark detector ran. See the generated example /Neeeophytee/ai-watermarks-reality-check/blob/main/examples/provenance-survival-example.md and the community benchmark template /Neeeophytee/ai-watermarks-reality-check/blob/main/examples/platform-benchmark-template.md before publishing a result. The provenance survival skill can now audit a directory of edited or downloaded copies in one run and create a self-contained Markdown or HTML report. This is useful for testing a CMS, editor, CDN, social network, or watermark-removal workflow without changing any of the files being measured. --derivatives-dir recursively adds visible, non-symlink files with stable labels. --report survival-report.html creates a portable, script-free report.- Local paths and the original command are redacted from reports by default. - The JSON result remains on stdout and continues to validate against the published schema. - Every report states its boundary: this workflow measures C2PA survival, not proprietary pixel or keyed text watermarks. | Dimension | States | Meaning | |---|---|---| | Manifest presence | PRESENT , POSSIBLE , ABSENT , UNKNOWN | Whether a C2PA manifest is observed | | Integrity | VALID , INVALID , NOT VERIFIED , UNKNOWN | Whether a conforming verifier validated the claim | | Signer trust | TRUSTED , UNTRUSTED , NOT CHECKED , UNKNOWN | Whether the signing chain reaches the selected trust list | | Text watermark | UNVERIFIABLE | Anthropic has not published its detector as of 2026-08-13 | POSSIBLE means a structural carrier, sidecar, or format-appropriate malformed hint was located; inspect the marker confidence before acting. Only STRUCTURAL means the location and carrier form match the specification. PRESENT is only ever returned by a conforming verifier. ABSENT is emitted only for a completed bounded scan of a supported container or an explicit live verifier "no claim" result. VALID never automatically means TRUSTED . Absence of a mark never proves that content was human-made. A literal mention of "C2PA" in readable text is recorded in c2pa mentions and is never evidence. Detection is structural, at the locations the C2PA 2.4 specification defines. | Container | Carrier | ABSENT possible? | |---|---|---| | PNG | caBX chunk | yes | | JPEG | APP11 JUMBF segment | yes | | WebP | RIFF C2PA chunk | yes | | TIFF/DNG | private tag 0xCD41 , type 7, in the last main IFD | yes | | GIF | C2PA GIF application extension | yes | | HTML | head <script type="application/c2pa" containing Base64; <link rel="c2pa-manifest" | yes | | Text | A.8 variation-selector wrapper; A.9 block in host comments or front matter | yes | Associated File /AFRelationship /C2PA Manifest | no | | | BMFF MP4/HEIC/AVIF | uuid box with the C2PA UUID, or jumb | no | | OOXML / ODF / ZIP | packaged manifest entry | no | | Any | detached .c2pa sidecar | not format-specific | Fail closed. The right-hand column is the important one. For formats marked no , this build inspects the carriers listed but cannot walk the container exhaustively because of compressed PDF object streams and fragmented BMFF. Finding nothing therefore yields UNKNOWN with a stated reason, never ABSENT . | Confidence | Meaning | |---|---| STRUCTURAL | Found at a spec-defined location | MODERATE | A format-appropriate key or XML namespace reference | SIDECAR | A detached .c2pa manifest beside the asset | A bounded read can never produce a conclusive clean result. detect-text-watermark streams the whole file where practical and always reports: | Field | Meaning | |---|---| file sha256 | SHA-256 over every byte on disk, always | scanned sha256 | SHA-256 over the region actually analysed | file bytes / scanned bytes | sizes of each | scan complete | whether the two coincide | If scan complete is false, the status is INCONCLUSIVE with a reason and exit 2 , even when nothing suspicious was seen in the part that was read. A prefix hash is never presented as the file hash. Every entrypoint uses the same convention, so an orchestrating agent can apply one rule: | Code | Meaning | |---|---| 0 | Conclusive and good: verified valid, no required gaps, no risk found | 1 | Conclusive and bad: invalid, required gaps, HIGH privacy risk, provenance lost, covert channel found | 2 | Inconclusive: unknown state, missing or unsupported verifier, unsupported container | Machine-readable JSON Schemas for all seven tools live in schemas/ /Neeeophytee/ai-watermarks-reality-check/blob/main/schemas . Every branch of every tool returns the same top-level field set, and CI validates real outputs against these schemas. The schemas also encode the evidence invariants: a document claiming integrity: VALID with manifest presence: ABSENT fails validation. detect-text-watermark runs a detector registry and reports honestly per adapter: | Adapter | Runs? | State when it cannot run | |---|---|---| anthropic-official | never | UNVERIFIABLE : no public detector or specification exists | synthid-text | never in this build | NOT CONFIGURED without keys, UNSUPPORTED with them | kgw-research | never in this build | NOT CONFIGURED without a key, UNSUPPORTED with one | unicode-covert-channel | yes | not applicable | c2pa-text-manifest | yes | not applicable | The state vocabulary distinguishes five situations, so a caller can never mistake "did not look" for "looked and found nothing": | State | Meaning | |---|---| DETECTED | The detector ran and found the signal | NOT DETECTED | The detector ran and did not find it | NOT CONFIGURED | Required keys or config were not supplied; nothing was analysed | UNSUPPORTED | Configured, but this build ships no scorer; nothing was analysed | UNVERIFIABLE | The scheme cannot be checked by a third party at all | FAILED | The detector errored | Only DETECTED and NOT DETECTED mean a detector ran , and only those contribute to the top-level status. SynthID and KGW accept configuration but this build performs no scoring for either; they are unavailable integrations, not silent passes. Keyed model-level watermarks bias token sampling with a secret key; detection requires that key. A third party cannot check them, and this pack says so rather than guessing. A standards-compliant C2PA text manifest 2.4 §A.8 is itself a run of variation selectors after a ZWNBSP. It is recognised as provenance and excluded from covert-channel findings rather than reported as suspicious. The covert-channel scan finds Unicode tag characters ASCII smuggling , bidi overrides, variation selectors, zero-width characters, exotic spaces, and mixed-script homoglyphs. These are text-integrity and prompt-injection signals , not watermark evidence: they identify no model or vendor. Deliberate non-goal: statistical or stylometric AI-text classifiers are not implemented and will not be added. Their false-positive rates make them unsafe for accusing a person of AI authorship. - No watermark removal, evasion, or provenance stripping. - No source files are modified. - No authorship verdicts. - No legal-compliance claims. - No statistical AI-text detection. - Reports avoid printing sensitive metadata values by default. See SOURCES.md /Neeeophytee/ai-watermarks-reality-check/blob/main/SOURCES.md . Product behavior is volatile; the source review date is recorded there. GNU Affero General Public License v3.0 /Neeeophytee/ai-watermarks-reality-check/blob/main/LICENSE AGPL-3.0-only