{"slug": "we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out", "title": "We stopped writing Playwright selectors and let AI figure it out", "summary": "A developer built Confidence Gate, an AI-powered test execution engine that replaces brittle Playwright selectors with plain English test steps. The system resolves elements through the accessibility tree, executes actions in a real Playwright browser, and verifies outcomes visually using a vision model. When selectors break between deploys, the engine automatically repairs them by re-querying the accessibility tree and scoring candidate elements against the original target description.", "body_md": "If you've maintained a Playwright or Cypress test suite for more than a few months, you know the drill. A designer renames a class, a developer restructures a form, and suddenly 30 tests are broken — not because the feature broke, but because .submit-btn became [data-action=\"submit\"].\n\nYou end up in a loop: fix selectors, ship, selectors break, fix selectors. The tests stop being useful because nobody trusts them.\n\nWe built Confidence Gate — an AI-powered test execution engine where you describe test steps in plain English and the system figures out the rest.\n\nInstead of:\n\nawait page.locator('[data-testid=\"email-input\"]').fill('[user@example.com](mailto:user@example.com)');\n\nawait page.locator('button[type=\"submit\"]').click();\n\nawait expect(page).toHaveURL('/dashboard');\n\nYou write:\n\n{ \"action\": \"enter the email from the test data in the email field\",\n\n\"expected\": \"the email field contains the entered address\" }\n\n\"expected\": \"the dashboard is displayed and the login form is gone\" }\n\nThe engine translates each step into a typed intent, resolves the target element from the accessibility tree, executes it in a real Playwright browser, takes a screenshot, and verifies the outcome visually.\n\nEach step goes through four stages:\n\n**1. Intent generation** — The natural language action is converted to a structured JSON ({ action: \"click\", target: { label: \"Sign In\", role: \"button\" }, value: null }). This separates intent from implementation.\n\n**2. Element resolution** — A multi-tier resolver finds the element: accessibility tree first (fast, reliable), CSS heuristics second, AI-assisted fallback third.\n\n**3. Execution + behavior detection** — Playwright executes the action. A mutation observer watches for DOM changes, URL changes, and value changes to confirm something actually happened.\n\n**4. Verification** — A vision model looks at the post-action screenshot and checks it against the expected result. If behavior was detected but verification fails, the engine assumes it hit the wrong element and\n\nretries with a blacklisted selector.\n\nWhen a selector stops working between deploys, the repair loop kicks in. It re-queries the accessibility tree, scores candidate elements against the original target description, and picks the best match. The new selector is cached so the next run is fast.\n\nAfter a run, every step result feeds into a score (0–100) built from:\n\nThe score maps to a gate decision: ship, caution, or block. You can call the API from CI and fail a deployment if the score drops below your threshold.\n\ngit clone [https://github.com/OaktreeInnovations/confidence-gate.git](https://github.com/OaktreeInnovations/confidence-gate.git)\n\ncd confidence-gate\n\ncp .env.example .env\n\nmake up\n\nOpen [http://localhost:3001](http://localhost:3001) and you're running.\n\nWe're working on four things in order:\n\nThe repo is MIT licensed and open to contributions. If any of this is interesting to you — especially the browser recording or the AI execution engine — come say hi on GitHub.", "url": "https://wpnews.pro/news/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out", "canonical_source": "https://dev.to/anjo_zulaybar_0b0a0e967eb/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out-1hh8", "published_at": "2026-05-28 08:02:46+00:00", "updated_at": "2026-05-28 08:23:05.105897+00:00", "lang": "en", "topics": ["ai-tools", "ai-products", "ai-startups", "natural-language-processing", "artificial-intelligence"], "entities": ["Playwright", "Cypress", "Confidence Gate"], "alternates": {"html": "https://wpnews.pro/news/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out", "markdown": "https://wpnews.pro/news/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out.md", "text": "https://wpnews.pro/news/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out.txt", "jsonld": "https://wpnews.pro/news/we-stopped-writing-playwright-selectors-and-let-ai-figure-it-out.jsonld"}}