People paste screenshots into an LLM and ask "does this page follow the rules?". I wanted a number for how far that works, with the answer key coming from somewhere the model can't see: the DOM.
Ten rules, frozen from my own site's style rules (the thresholds are mine, not a standard):
I built the site at the commit before the fixes (7bec67f), opened 12 pages at 1280 and 375 in headless Chrome via CDP, and measured every rule with getComputedStyle / getBoundingClientRect. That gives 12 × 11 = 132 judgments (R2 counts twice), of which 36 are "element not present on this page". 96 pairs remain.
From the same render, the script clips 1:1 crops of the elements in question — 164 PNGs, SHA-256 of each in a manifest. That matters: my first attempt measured positions in an iframe and shot with headless Chrome, and at 375px the two disagreed by 300–700px, so six footer crops were blank white.
// R1 = smallest font-size among text nodes inside <main>, excluding chrome
const EXCL = 'nav, .crumbs, form, button, .works-nav, .toc-block, footer, .site-header, .site-footer';
const textEls = [...document.querySelectorAll('main *')].filter((e) =>
!['SCRIPT','STYLE','SVG','PATH','NOSCRIPT'].includes(e.tagName) && !e.closest(EXCL) && vis(e) &&
[...e.childNodes].some((n) => n.nodeType === 3 && n.textContent.trim()));
Each judge got, per page, a frozen instruction ("for each rule answer YES / NO / unknown / not-applicable with one line of reasoning; a rule with no image is not-applicable; do not guess; at most three free remarks") plus that page's crops.
Two different delivery paths, so this is not a model-vs-model comparison. Scoring is a regex over the four verbatim values against the gold — no human in the loop.
My first summary said "GPT 94% on 96 pairs, and it got all 35 of the 44px pairs right, telling 40px from 45px". An external reviewer looked at the gold and pointed out that in the before version, every breadcrumb is 40px and every footer link is 45px: within each 44px rule there are only violations or only non-violations. A judge that answers "small links violate, the footer doesn't" scores 35/35 without measuring anything. The same holds for R5.
So the main metric is now the 60 pairs whose rule contains both labels (R1–R4). The 44px rules were scored (GPT 36/36) but that score cannot test threshold discrimination, so it is reported and not counted.
Accuracy = correct YES/NO including true negatives, divided by all pairs (unknowns count against). GPT answered 60/60, so the two bases coincide; Gemini answered 57/60, so I also give the answered-only rate.
| Judge | Group | Pairs | Answered | Correct | FP rate | FN rate |
|---|---|---|---|---|---|---|
| GPT | contrastive R1–R4 | 60 | 100% | 54/60 = 90% | 7% | 21% |
| GPT | visual (R2, R4) | 36 | 100% | 36/36 | 0% | 0% |
| GPT | numeric (R1, R3) | 24 | 100% | 18/24 | 16% | 60% |
| Gemini | contrastive R1–R4 | 60 | 95% | 41/60 = 68% (41/57 = 72% of answered) | 26% | 36% |
| Gemini | visual | 36 | 97% | 28/36 | 12% | 44% |
| Gemini | numeric | 24 | 92% | 13/24 | 47% | 20% |
What I'd hand to image review today: mixed heading sizes, heading line counts, line length with a multi-em gap. What stays in the DOM: anything within a couple of pixels of its threshold. What I can't say yet: whether image review discriminates the 44px tap-target threshold — the fixed version of the site needs to be shot to create contrast.
Part 3 of the series "Measure the Outcome, Not the Proxy" (part 1: transfer size ≠ speed; part 2: shorter ≠ requirements preserved — both on this account).
Verification record (environment, verdict, evidence: gold JSON, both score sheets, the 164 crops as a ZIP, all 24 verbatim answers) and the canonical Japanese write-up. The DOM measurer that fixed the pages. Reproduction kit with the images, the gold, the judge script for Gemini and the scorer (code MIT, data CC BY 4.0). I verify AI-generated and outsourced work against pre-registered pass/fail criteria — text-only, no calls: Sumitsuke / verification.