Introducing Vedha: New Testing framework for Agentic World. Test your web app with an AI agent in 5 minutes (and catch the bugs it would miss)

A developer released Vedha, an open-source MCP server built on Playwright that lets AI agents test web apps in a real browser while verifying server-side effects rather than trusting on-screen text. In a demonstration, an agent reported a checkout as failed after the backend returned a 500 even though the page displayed "Order placed," and a verified flow was promoted into a plain @playwright/test spec for CI. Vedha works with Claude Code, Cursor, VS Code and other MCP clients and restricts browsing to domains the user explicitly allows.

AI agents are good at clicking through a web app. They're bad at knowing whether it actually worked. An agent sees "Order placed" on the screen and reports success, even when the API behind it returned 500 and saved nothing. This tutorial sets up an AI agent that tests your app in a real browser and checks what the server did . You'll go from zero to a test you can keep in CI in about five minutes. We'll use Vedha , an open-source MCP server built on Playwright. It works with Claude Code, Cursor, VS Code and any other MCP client. In Claude Code: claude mcp add vedha -- npx -y @vedhajs/mcp --allow-domain localhost --out ./vedha-out For Cursor, VS Code or Claude Desktop, add this to your MCP config: { "mcpServers": { "vedha": { "command": "npx", "args": "-y", "@vedhajs/mcp", "--allow-domain", "localhost", "--out", "./vedha-out" } } } Two things to note: --allow-domain localhost is required: the agent can only browse the domains you list. Point it at your dev server, not the open web. npx playwright install chromium once. Optional: install the agent skill, which teaches the agent how to test with Vedha declare checks, mark writes, never retry an unknown outcome : npx skills add HMAKT99/VEDHA Start your app, then ask: Test the checkout on localhost:3000: place an order for Ada and confirm it worked. The agent opens the page and works through the flow with vedha act . Each call is one browser action, and it can say what success looks like. Here is the call an agent sends to fill the form and submit it: { "op": "fill", "intent": "enter name", "target": { "label": "Name" }, "value": "Ada", "then": { "op": "click", "intent": "place the order", "target": { "role": "button", "name": "Place order" }, "mutating": true, "expect": { "text contains": "Order placed" } } } mutating: true tells Vedha the click changes data, so it should check what the server did, not just the page. On a healthy app, this is what the agent gets back real output, trimmed : step 1 step-2 : verified result: verified effect: server confirmed POST /api/orders 201; read-back /api/orders confirms POST /api/orders changed on the page 3 added, 3 removed : + status: Order placed Vedha saw the 201 , then read the order list back to confirm the order is really there. Now the same test against a broken backend: the API returns 500 , but the frontend still shows "Order placed". This is what the agent gets back real output, trimmed : step 1 step-2 : verified result: failed health check failed : declared checks passed, but the page reported: POST /api/orders returned 500 hint: treat this action as failed: the server or page reported an error even if the page looks fine requests: POST /api/orders 500 + status: Order placed open failure: "place the order" - declared checks passed, but the page reported: POST /api/orders returned 500. Do not report success unless a later step fixed it. "Order placed" is on the page, and the agent's own check passed. Vedha still reports the step as failed , says why, and reminds the agent not to claim success. A browser tool that only checks the screen would have reported a pass. Two more things Vedha does here without being asked: outcome unknown and vedha probe for example GET /api/orders . Once a flow is verified, ask the agent: Save that as a Playwright test. It calls vedha promote , which writes a plain @playwright/test spec from the verified steps. This is the real output for the order flow above: // Generated by Vedha from an explored session. Review before committing. import { test, expect } from "@playwright/test"; test "places an order", async { page } = { await test.step "open the shop", async = { await page.goto "/" ; } ; await test.step "enter name", async = { await page.getByLabel "Name", { exact: true } .fill "Ada" ; } ; await test.step "place the order", async = { await page.getByRole "button", { name: "Place order", exact: true } .click ; await expect page.locator "body" .toContainText "Order placed" ; await expect.poll async = { const res = await page.request.get "/api/orders" ; const body = await res.json ; const items = Array.isArray body ? body : body ; return items.some it = Object.entries { name: "Ada", qty: 1, product: "Widget" } .every k, val = it k === val , ; } .toBe true ; } ; } ; Notice the last assertion: because Vedha confirmed the order on the server during the run, the generated test checks the server too, not just the text. It also lists its assumptions at the top, so review it before committing. Commit the spec and run it with your normal Playwright setup. One more line makes the rest of your suite safer too: npm install -D @vedhajs/playwright-test js import { test, expect } from "@vedhajs/playwright-test"; // instead of "@playwright/test" With that import, any test whose own checks pass while the app answers with a 5xx, or a write request fails, fails instead of going green. That covers tests written by your agent and by people. More on guard mode here https://dev.to/arun kumart 22e9a9959b10/your-ai-agent-writes-playwright-tests-that-pass-while-your-api-returns-500-1p0d . How much difference does it make? In a small pilot one model, held-out tasks, 3 trials each , the agent missed 9 seeded bugs with Vedha against 26 with Playwright MCP, using about 30% fewer tokens. Details and limits here https://dev.to/arun kumart 22e9a9959b10/is-playwright-alone-enough-for-agentic-testing-433p . git clone https://github.com/HMAKT99/VEDHA && cd VEDHA pnpm install && pnpm build npx tsx examples/demo/run.ts Repo, docs and benchmark data: https://github.com/HMAKT99/VEDHA https://github.com/HMAKT99/VEDHA Apache-2.0 If you try it on your app, I'd like to hear what it caught, and anything it got wrong.