# Can an LLM measure UI thresholds from screenshots? 164 crops vs DOM gold labels

> Source: <https://dev.to/tauridev/can-an-llm-measure-ui-thresholds-from-screenshots-164-crops-vs-dom-gold-labels-4gbb>
> Published: 2026-09-21 08:31:50+00:00

People paste screenshots into an LLM and ask "does this page follow the rules?". I wanted a number for how far that works, with the answer key coming from somewhere the model can't see: the DOM.

Ten rules, frozen from my own site's style rules (the thresholds are mine, not a standard):

I built the site at the commit *before* the fixes (`7bec67f`), opened 12 pages at 1280 and 375 in headless Chrome via CDP, and measured every rule with `getComputedStyle` / `getBoundingClientRect`. That gives 12 × 11 = **132 judgments** (R2 counts twice), of which 36 are "element not present on this page". 96 pairs remain.

From the **same render**, the script clips 1:1 crops of the elements in question — 164 PNGs, SHA-256 of each in a manifest. That matters: my first attempt measured positions in an iframe and shot with headless Chrome, and at 375px the two disagreed by 300–700px, so six footer crops were blank white.

```
// R1 = smallest font-size among text nodes inside <main>, excluding chrome
const EXCL = 'nav, .crumbs, form, button, .works-nav, .toc-block, footer, .site-header, .site-footer';
const textEls = [...document.querySelectorAll('main *')].filter((e) =>
  !['SCRIPT','STYLE','SVG','PATH','NOSCRIPT'].includes(e.tagName) && !e.closest(EXCL) && vis(e) &&
  [...e.childNodes].some((n) => n.nodeType === 3 && n.textContent.trim()));
```

Each judge got, per page, a frozen instruction ("for each rule answer YES / NO / unknown / not-applicable with one line of reasoning; a rule with no image is not-applicable; do not guess; at most three free remarks") plus that page's crops.

Two different delivery paths, so this is **not** a model-vs-model comparison. Scoring is a regex over the four verbatim values against the gold — no human in the loop.

My first summary said "GPT 94% on 96 pairs, and it got all 35 of the 44px pairs right, telling 40px from 45px". An external reviewer looked at the gold and pointed out that in the *before* version, every breadcrumb is 40px and every footer link is 45px: within each 44px rule there are **only violations or only non-violations**. A judge that answers "small links violate, the footer doesn't" scores 35/35 without measuring anything. The same holds for R5.

So the main metric is now the **60 pairs whose rule contains both labels** (R1–R4). The 44px rules were scored (GPT 36/36) but that score cannot test threshold discrimination, so it is reported and not counted.

Accuracy = correct YES/NO including true negatives, divided by **all** pairs (unknowns count against). GPT answered 60/60, so the two bases coincide; Gemini answered 57/60, so I also give the answered-only rate.

| Judge | Group | Pairs | Answered | Correct | FP rate | FN rate | 
|---|---|---|---|---|---|---|
| GPT | contrastive R1–R4 | 60 | 100% | **54/60 = 90%** | 7% | 21% | 
| GPT | visual (R2, R4) | 36 | 100% | **36/36** | 0% | 0% | 
| GPT | numeric (R1, R3) | 24 | 100% | 18/24 | 16% | 60% | 
| Gemini | contrastive R1–R4 | 60 | 95% | **41/60 = 68%** (41/57 = 72% of answered) | 26% | 36% | 
| Gemini | visual | 36 | 97% | 28/36 | 12% | 44% | 
| Gemini | numeric | 24 | 92% | 13/24 | 47% | 20% | 

What I'd hand to image review today: mixed heading sizes, heading line counts, line length with a multi-em gap. What stays in the DOM: anything within a couple of pixels of its threshold. What I can't say yet: whether image review discriminates the 44px tap-target threshold — the fixed version of the site needs to be shot to create contrast.

*Part 3 of the series "Measure the Outcome, Not the Proxy" (part 1: transfer size ≠ speed; part 2: shorter ≠ requirements preserved — both on this account).*

[Verification record](https://sumitsuke.jp/via/devto/lab/screenshot-ai-vs-dom-ten-rules/) (environment, verdict, evidence: gold JSON, both score sheets, the 164 crops as a ZIP, all 24 verbatim answers) and the canonical Japanese write-up. [The DOM measurer that fixed the pages](https://sumitsuke.jp/via/devto/lab/readability-dom-eight-items/). [Reproduction kit](https://github.com/sumitsuke/screenshot-ai-vs-dom-ten-rules) with the images, the gold, the judge script for Gemini and the scorer (code MIT, data CC BY 4.0). I verify AI-generated and outsourced work against pre-registered pass/fail criteria — text-only, no calls: [Sumitsuke / verification](https://sumitsuke.jp/works/verification/).