{"slug": "building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser", "title": "Building an AI QA Agent That Tests Your Jira Tickets in a Real Browser", "summary": "A developer built an open-source AI QA agent that takes a Jira ticket ID and a test environment URL, reads the ticket, writes test steps, and executes them in a real headful Chrome window. The agent combines Agent Factory, a studio for designing and chatting with agents, with BaaS (Browser as a Service), which drives the browser by opening pages, clicking, waiting, and taking screenshots. Setup instructions cover running both components locally via Docker, configuring an OpenAI-backed LLM client, and wiring Jira API token authentication into the Test Orchestrator agent.", "body_md": "Give it a Jira ticket ID and a test environment URL, and this agent reads the ticket, writes test steps, and runs them in a real Chrome window while you watch.\n\nIt's built from two open-source pieces. **Agent Factory** is the studio where agents are designed, edited, and chatted with. **BaaS** (Browser as a Service) is the part that actually drives the browser: it opens pages, clicks, waits, and takes screenshots on the agent's behalf.\n\nThis post walks through setting both up locally and running the bug check agent against your own app.\n\nRequired:\n\nRecommended:\n\nClone the repo: [Ursa-Minor-Beta/agent-factory-docker-api-ui](https://github.com/Ursa-Minor-Beta/agent-factory-docker-api-ui).\n\nThen create your env file and bring everything up with the bundled database:\n\n```\ncp .env.example .env\n# edit .env – at minimum set the admin email and password\ndocker-compose --profile with-db up --build\n```\n\nThe admin email and password in `.env` are what you'll use to log in to the factory later, so pick something you'll remember.\n\nI recommend running BaaS locally while you debug and in Docker while you test, because locally you can watch exactly what the agent does in the browser. If the local setup fights you (OS permissions and the like), fall back to Docker. It's stable.\n\nStart MongoDB and create the env file:\n\n```\ncd baas\ndocker compose up -d mongodb\ncp .env.example .env\nvim .env\n```\n\nSet these values in `.env`:\n\n```\nAPI_KEY=<pick-a-key>\n\nBROWSER_EXECUTABLE='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'\nBROWSER_HEADFUL=true\n\nLLM_CLIENT=openai\nOPENAI_TOKEN=\"<your token here>\"\nOPENAI_ORGANIZATION=\"<your organization here>\"\n```\n\n`BROWSER_HEADFUL=true` is what makes the Chrome window visible. The path above is for macOS; point it at your own Chrome binary on other systems. Remember your `API_KEY`, because the studio needs it in Step 3.\n\nNext, map `host.docker.internal` to localhost so the Dockerized factory and the local BaaS can find each other:\n\n```\nsudo vim /etc/hosts\n# add this line:\n127.0.0.1       host.docker.internal\n```\n\nFinally, load the env and start BaaS:\n\n```\nset -a && . ./.env && set +a\ngo run ./cmd/baas\n```\n\nYou'll see warnings about missing Pandoc, PDF-to-image, and wmctrl. Ignore them: you don't need those tools, and this is what a successful start looks like.\n\nOpen the studio at [http://localhost:8080](http://localhost:8080) and log in with the admin credentials from Step 1. Then:\n\n`BAAS_API_KEY`, set to the same value as `API_KEY` in the BaaS `.env`.\nThat's the whole setup. To check the browser actually works, go to **Agents → Browser Screenshot** and enter `https://google.com`.\n\nA new Chrome window will open. It may ask for permissions, so allow them. If everything is wired up, the window loads Google and the chat returns a browser session ID plus a screenshot of the page.\n\nLog in to [Atlassian API tokens](https://id.atlassian.com/manage-profile/security/api-tokens) and choose **Create API token with scopes**. Then encode your email and token together:\n\n```\necho -n \"you@example.com:your-api-token\" | base64\n```\n\nAdd the output to the studio's **Secrets** under the name `Jira_auth`.\n\nIn the factory, go to **Agents**, find **Test Orchestrator**, and click **Edit**. Two things to change:\n\n`input-1` node, replace the default URL (`https://google.com`) with your test environment URL, and set your `jiraSubdomain`.` http-1` node.\nFor login, I recommend hardcoding it in `http-1`. Login flows rarely change, so scripting them once saves tokens on every run.\n\nIf you'd rather skip a separate login step, set the body of `http-1` to the following. It starts a browser session, opens the URL, and takes a screenshot:\n\n```\n\"body\": \"{\\\"program\\\":\\\"navigate('{{node:input-1.url}}'); sleep('3s'); waitReady(body,'timeout:20s'); takeScreenshot('screenshot', 'timeout:10s');\\\", \\\"sessionID\\\": \\\"{{node:http-0.response.result.sessionID}}\\\", \\\"stopSession\\\": false}\",\n```\n\nOpen **Test Orchestrator** and click **Chat**. Send it your Jira ticket ID (something like `SCRUM-165`) and your test environment URL. The agent takes it from there and runs the first test.\n\nThe prompt in `llm-generate-steps` is where test steps get written. Replace the example details there with what you know about your application. Pasting in your user documentation, if you have any, works well.\n\nThe orchestrator uses **Test Step Executor** as a skill, and its prompts are worth adapting to your app and its quirks. Each LLM node has one job:\n\n`llm-0`` llm-1``llm-2`\nI hope this agent saves you some manual testing. If you run into problems or have ideas, get in touch.\n\nOne honest ask: this is a side project, and I'd like to know whether to keep building it. If it's useful to you, [star the repo on GitHub](https://github.com/Ursa-Minor-Beta/agent-factory-docker-api-ui), send me a DM, or leave a comment below. Any of those tells me the time is well spent.\n\nKeep humans on the loop and don’t ship on Fridays!", "url": "https://wpnews.pro/news/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser", "canonical_source": "https://dev.to/quality_minder/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser-3pc7", "published_at": "2026-09-25 17:05:32+00:00", "updated_at": "2026-09-25 17:30:59.979339+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models"], "entities": ["Agent Factory", "BaaS", "Jira", "Atlassian", "Chrome", "MongoDB", "Docker", "OpenAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser", "markdown": "https://wpnews.pro/news/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser.md", "text": "https://wpnews.pro/news/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser.txt", "jsonld": "https://wpnews.pro/news/building-an-ai-qa-agent-that-tests-your-jira-tickets-in-a-real-browser.jsonld"}}