How We Built an AI Visual CRO Auditor That Draws Bounding Boxes Over UX Friction Plyxo Community Edition has released an open-source Visual CRO engine that uses multimodal vision models to audit webpages for UX friction, drawing bounding boxes over problem areas and generating Tailwind CSS fixes. The tool captures high-DPI screenshots via headless Chrome, feeds them to a vision model with a structured prompt, and returns normalized coordinates for an interactive overlay. When traditional automated audit tools like Google Lighthouse scan a webpage, they test for DOM metrics and performance : Largest Contentful Paint LCP , missing ARIA labels, image dimensions, or meta tags. What they CANNOT tell you: To solve this, we built an open-source Visual CRO Conversion Rate Optimization engine for Plyxo Community Edition https://github.com/pixelfogg/Plyxo-CRO-SEO-AIO-AEO-GEO . It captures high-DPI full-page screenshots, feeds them into multimodal vision models, returns normalized coordinate bounding boxes ymin, xmin, ymax, xmax over friction zones , and generates copy-paste Tailwind CSS fixes. Here is how the architecture works under the hood. Target URL ↓ Puppeteer / Headless Chrome ↓ High-DPI Screenshot + DOM Heuristics Multimodal Vision Model ↓ Normalized 0-1000 Coordinates JSON Interactive Canvas Overlay + Tailwind Code Remediation Standard screenshots often miss sticky headers, modals, or hydration popups. We use a headless Chrome pipeline that enforces high-DPI rendering and waits for network idle: python // packages/core/src/scanners/screenshot.ts import puppeteer from 'puppeteer'; export async function captureViewport url: string { const browser = await puppeteer.launch { headless: 'new', args: '--no-sandbox', '--disable-setuid-sandbox' } ; const page = await browser.newPage ; await page.setViewport { width: 1440, height: 900, deviceScaleFactor: 2 // High DPI for crisp font & badge recognition } ; await page.goto url, { waitUntil: 'networkidle2', timeout: 30000 } ; // Clean scroll to trigger lazy-loaded sections await page.evaluate = window.scrollTo 0, document.body.scrollHeight / 2 ; await new Promise r = setTimeout r, 600 ; await page.evaluate = window.scrollTo 0, 0 ; const screenshotBuffer = await page.screenshot { fullPage: false, // Hero/above-the-fold is where 80% of CRO friction happens encoding: 'base64' } ; await browser.close ; return screenshotBuffer; } Standard LLMs return chatty explanations. For an interactive UI overlay, we need structured JSON with normalized visual coordinates 0-1000 scale . Here is the system prompt and structured schema we pass to the vision model: js const SYSTEM PROMPT = You are an expert Conversion Rate Optimization CRO and UX Design Auditor. Analyze the provided desktop screenshot of a landing page. Identify top UX/CRO friction points: 1. Contrast/Visibility issues unclear CTAs 2. Visual clutter / Cognitive overload 3. Lack of immediate value proposition / hierarchy 4. Trust signal deficiencies For each issue, you MUST provide: - 'title': Short descriptive title - 'severity': 'critical' | 'warning' | 'info' - 'box 2d': Normalized coordinates ymin, xmin, ymax, xmax between 0 and 1000 - 'frictionReason': Why this hurts conversion - 'proposedCodeFix': Concrete Tailwind CSS / HTML remediation code ; Once the backend returns the normalized coordinate array, we render dynamic highlight boxes that scale responsively with any container: python // components/VisualCroOverlay.tsx import React, { useState } from 'react'; interface FrictionBox { id: string; title: string; severity: 'critical' | 'warning' | 'info'; box 2d: number, number, number, number ; // ymin, xmin, ymax, xmax proposedCodeFix: string; } export function VisualCroOverlay { screenshotUrl, issues }: { screenshotUrl: string; issues: FrictionBox } { const selectedIssue, setSelectedIssue = useState