{"slug": "introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with", "title": "Introducing Vedha: New Testing framework for Agentic World. Test your web app with an AI agent in 5 minutes (and catch the bugs it would miss)", "summary": "A developer released Vedha, an open-source MCP server built on Playwright that lets AI agents test web apps in a real browser while verifying server-side effects rather than trusting on-screen text. In a demonstration, an agent reported a checkout as failed after the backend returned a 500 even though the page displayed \"Order placed,\" and a verified flow was promoted into a plain @playwright/test spec for CI. Vedha works with Claude Code, Cursor, VS Code and other MCP clients and restricts browsing to domains the user explicitly allows.", "body_md": "AI agents are good at clicking through a web app. They're bad at knowing whether it actually worked. An agent sees \"Order placed\" on the screen and reports success, even when the API behind it returned 500 and saved nothing.\n\nThis tutorial sets up an AI agent that tests your app in a real browser **and checks what the server did**. You'll go from zero to a test you can keep in CI in about five minutes.\n\nWe'll use **Vedha**, an open-source MCP server built on Playwright. It works with Claude Code, Cursor, VS Code and any other MCP client.\n\nIn Claude Code:\n\n```\nclaude mcp add vedha -- npx -y @vedhajs/mcp --allow-domain localhost --out ./vedha-out\n```\n\nFor Cursor, VS Code or Claude Desktop, add this to your MCP config:\n\n```\n{\n  \"mcpServers\": {\n    \"vedha\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@vedhajs/mcp\", \"--allow-domain\", \"localhost\", \"--out\", \"./vedha-out\"]\n    }\n  }\n}\n```\n\nTwo things to note:\n\n`--allow-domain localhost` is required: the agent can only browse the domains you list. Point it at your dev server, not the open web.`npx playwright install chromium` once.\n**Optional:** install the agent skill, which teaches the agent how to test with Vedha (declare checks, mark writes, never retry an unknown outcome):\n\n```\nnpx skills add HMAKT99/VEDHA\n```\n\nStart your app, then ask:\n\nTest the checkout on localhost:3000: place an order for Ada and confirm it worked.\n\nThe agent opens the page and works through the flow with `vedha_act`. Each call is one browser action, and it can say what success looks like. Here is the call an agent sends to fill the form and submit it:\n\n```\n{\n  \"op\": \"fill\", \"intent\": \"enter name\", \"target\": { \"label\": \"Name\" }, \"value\": \"Ada\",\n  \"then\": [{\n    \"op\": \"click\", \"intent\": \"place the order\",\n    \"target\": { \"role\": \"button\", \"name\": \"Place order\" },\n    \"mutating\": true,\n    \"expect\": { \"text_contains\": \"Order placed\" }\n  }]\n}\n```\n\n`mutating: true` tells Vedha the click changes data, so it should check what the server did, not just the page.\n\nOn a healthy app, this is what the agent gets back (real output, trimmed):\n\n```\nstep 1 (step-2): verified\nresult: verified\neffect: server confirmed (POST /api/orders 201; read-back /api/orders confirms POST /api/orders)\nchanged on the page (3 added, 3 removed):\n  + status: Order placed\n```\n\nVedha saw the `201`, then read the order list back to confirm the order is really there.\n\nNow the same test against a broken backend: the API returns **500**, but the frontend still shows \"Order placed\". This is what the agent gets back (real output, trimmed):\n\n```\nstep 1 (step-2): verified\nresult: failed (health_check_failed): declared checks passed, but the page reported: POST /api/orders returned 500\nhint: treat this action as failed: the server or page reported an error even if the page looks fine\nrequests: POST /api/orders 500\n  + status: Order placed\nopen failure: \"place the order\" - declared checks passed, but the page reported:\nPOST /api/orders returned 500. Do not report success unless a later step fixed it.\n```\n\n\"Order placed\" **is** on the page, and the agent's own check passed. Vedha still reports the step as **failed**, says why, and reminds the agent not to claim success. A browser tool that only checks the screen would have reported a pass.\n\nTwo more things Vedha does here without being asked:\n\n`outcome_unknown` and `vedha_probe` (for example `GET /api/orders`).\nOnce a flow is verified, ask the agent:\n\nSave that as a Playwright test.\n\nIt calls `vedha_promote`, which writes a plain `@playwright/test` spec from the verified steps. This is the real output for the order flow above:\n\n```\n// Generated by Vedha from an explored session. Review before committing.\nimport { test, expect } from \"@playwright/test\";\n\ntest(\"places an order\", async ({ page }) => {\n  await test.step(\"open the shop\", async () => {\n    await page.goto(\"/\");\n  });\n  await test.step(\"enter name\", async () => {\n    await page.getByLabel(\"Name\", { exact: true }).fill(\"Ada\");\n  });\n  await test.step(\"place the order\", async () => {\n    await page.getByRole(\"button\", { name: \"Place order\", exact: true }).click();\n    await expect(page.locator(\"body\")).toContainText(\"Order placed\");\n    await expect.poll(async () => {\n      const res = await page.request.get(\"/api/orders\");\n      const body = await res.json();\n      const items = Array.isArray(body) ? body : [body];\n      return items.some((it) =>\n        Object.entries({ name: \"Ada\", qty: 1, product: \"Widget\" }).every(([k, val]) => it[k] === val),\n      );\n    }).toBe(true);\n  });\n});\n```\n\nNotice the last assertion: because Vedha confirmed the order on the server during the run, the generated test checks the server too, not just the text. It also lists its assumptions at the top, so review it before committing.\n\nCommit the spec and run it with your normal Playwright setup. One more line makes the rest of your suite safer too:\n\n```\nnpm install -D @vedhajs/playwright-test\njs\nimport { test, expect } from \"@vedhajs/playwright-test\"; // instead of \"@playwright/test\"\n```\n\nWith that import, any test whose own checks pass while the app answers with a 5xx, or a write request fails, **fails** instead of going green. That covers tests written by your agent and by people. ([More on guard mode here](https://dev.to/arun_kumart_22e9a9959b10/your-ai-agent-writes-playwright-tests-that-pass-while-your-api-returns-500-1p0d).)\n\nHow much difference does it make? In a small pilot (one model, held-out tasks, 3 trials each), the agent missed **9** seeded bugs with Vedha against **26** with Playwright MCP, using about 30% fewer tokens. [Details and limits here](https://dev.to/arun_kumart_22e9a9959b10/is-playwright-alone-enough-for-agentic-testing-433p).\n\n```\ngit clone https://github.com/HMAKT99/VEDHA && cd VEDHA\npnpm install && pnpm build\nnpx tsx examples/demo/run.ts\n```\n\nRepo, docs and benchmark data: **[https://github.com/HMAKT99/VEDHA](https://github.com/HMAKT99/VEDHA)** (Apache-2.0)\n\nIf you try it on your app, I'd like to hear what it caught, and anything it got wrong.", "url": "https://wpnews.pro/news/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with", "canonical_source": "https://dev.to/arun_kt_bb670b3a571f5efd8/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with-an-ai-agent-in-5-1jd3", "published_at": "2026-11-01 03:37:29+00:00", "updated_at": "2026-11-01 03:51:58.046237+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "agent-protocols", "ai-products"], "entities": ["Vedha", "Playwright", "Claude Code", "Cursor", "VS Code", "HMAKT99"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with", "markdown": "https://wpnews.pro/news/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with.md", "text": "https://wpnews.pro/news/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with.txt", "jsonld": "https://wpnews.pro/news/introducing-vedha-new-testing-framework-for-agentic-world-test-your-web-app-with.jsonld"}}