How to run an AI bug bash before a release

A new guide details how teams can run a pre-release AI bug bash by giving an AI QA agent called Mo a written brief and a reachable staging or preview URL, without writing test files beforehand. The brief should name the release change, supply test credentials, restrict the run to affected flows, and list prohibited actions such as submitting real payments or changing account permissions; sessions are launched with the command `qa start "$(cat brief.md)"`, which returns session JSON with a `sessionId` and `webUrl`. Confirmed findings should be reviewed for reproduction steps and evidence before filing, then converted into regression tests, while business-risk decisions, ambiguous UX, subjective design review, and physical-device checks stay with a targeted human pass.

TL;DR - An AI QA agent like Mo can bug bash a web app from a written brief against a staging or preview URL https://momentic.ai/blog/test-coding-agent-pull-requests-preview-deploys . You do not need to write test files before the run. - Give Mo https://momentic.ai/mo a reachable target URL or an uploaded mobile build, plus test credentials. Keep the brief specific about scope, test data, and actions it must avoid. - Review each finding’s reproduction steps and evidence before filing a bug. Add regression tests https://momentic.ai/blog/types-of-regression-testing for confirmed bugs so your test suite catches them after the fix. - Use a targeted human pass for business-risk decisions, ambiguous UX, subjective design review, and physical devices https://momentic.ai/blog/real-devices-vs-emulators-simulators . Why bug bashes need to change as release cadence increases A fixed pre-release bug bash covers less of the changed application when commits and deploys arrive more often. Engineers still have limited time to click through staging, and a coding agent can add or change flows faster than a manual pass can inspect them. A release check needs enough throughput to examine those changes without asking the same people to spend longer on every deploy. A maintained Playwright https://momentic.ai/blog/playwright-ci-cd-guide or YAML regression suite checks behavior you already know to test. When a change creates an unexpected interaction between flows, someone has to find it before they can add a regression test. A manual bash can do that exploratory work, but its coverage depends on which paths engineers choose to try. Mo offers a brief-driven exploratory pass against a running app before release. You give it a target and boundaries, then review the behavior it finds rather than treating every finding as a bug. Confirmed bugs can become tests in the repo. The exploratory pass looks for missed behavior, while the regression suite checks that known failures stay fixed. Step 1: Write the bug bash brief A useful bug bash brief names the release change and the environment where you want it checked. Give the agent a reachable staging or preview URL, or identify the mobile build. Then limit the run to the flows affected by the release. “Test everything” gives the agent little basis for deciding which paths matter and can produce findings outside the change you need to verify. For an order-management release, you could write the brief like this. - Check the order-management area on the release candidate’s preview deployment. - Create an order, edit its shipping details, cancel it, and confirm each change appears in the order history. - Sign in with the dedicated test account provided for this run. - Use only orders created for this bash. Leave existing and shared test records unchanged. - Do not submit real payments, send customer notifications, or change account permissions. Give the agent any setup details it needs to reach those flows, including credentials, required account permissions, and known test-data constraints. State the expected behavior when it is specific to this release. For example, if cancellation should now restore inventory, put that expectation in the brief rather than relying on the agent to infer it. Treat prohibited actions as part of test correctness. A run that charges a card or modifies shared records has crossed the test boundary, even if it also finds a bug. If a flow requires an irreversible action and you cannot provide safe test data, exclude that action and reserve it for a controlled human check. Step 2: Run the session against staging, preview, or a mobile build Run Mo against a reachable staging or preview URL after you have scoped the brief. The agent can explore the running app without test files or code changes up front. Give it test credentials in the brief, and use a tunnel if the target is local or private. For a web run, pass the brief to the CLI. qa start "$ cat brief.md " The command returns session JSON with a sessionId and webUrl . Keep the session ID for triage, and open the web URL to inspect the run. To check progress and see the full session status, run: qa status --full For a PR preview, use qa start --pr and keep the brief scoped to the changes under review. For a mobile run, provide a build and the same kind of brief, including test accounts and prohibited actions. Mobile sessions run on Android emulators or iOS simulators, not physical devices. If release risk depends on a specific handset or hardware behavior, reserve that check for a human pass. Step 3: Triage findings and read the evidence A completed bug bash produces candidates for investigation, not a filing queue. Start with the session report, then inspect each finding before you create an issue. qa report <session-id qa bugs describe For each finding, compare the expected and actual behavior against the brief. Read the reproduction steps, and inspect any available recording or screenshot. Check that the agent used the intended account and test data, especially when permissions or existing records affect the result. Reproduce likely regressions against the same build. A failed request caused by an unavailable staging dependency, a test-data collision, or behavior outside the brief may need a different fix or no product issue at all. Record the environment and steps for findings you can confirm. If the evidence leaves the cause unclear, keep the finding open for investigation rather than filing it as a regression. Triage protects the signal-to-noise ratio of the release check. A short list of reproducible failures gives engineers something they can fix and recheck. A longer list of unverified observations makes that work harder. Step 4: Turn confirmed bugs into regression tests A confirmed finding should become a regression test after you fix it. The exploratory session records the failure, but a test in the repo can check the same behavior on later commits. Export the session report for the coding agent working on the fix. qa report <session-id --output <export-path Give the agent the exported bugs.json and verdicts.json , along with the relevant reproduction steps. Review its code change against the reported behavior rather than treating a proposed fix as proof that the bug is gone. Once the fix reaches the target environment, use qa send to ask Mo to recheck the finding in the same session. The mo-qa skill packages this fix-and-recheck loop. Add a YAML regression test that reproduces the original failure and asserts the expected behavior. Where practical, confirm the test fails against the buggy build and passes against the fix before committing it to the repo. The bash found a behavior worth checking. The maintained suite keeps checking it in CI/CD. When a human bug bash still earns its place A human pass can check behavior the agent cannot reach. Mo can run against mobile builds in emulators and simulators, but those runs do not test a physical device. If a release changes camera use or touch interactions, check the affected flows on real hardware before shipping. Humans also need to make release decisions that a bug report cannot settle. A checkout flow might complete successfully while its confirmation screen leaves users unsure whether they placed an order. Someone who understands the product and its business risk needs to decide whether that ambiguity blocks the deploy. Run the AI pass first to surface reproducible failures. Then focus the human bash on physical-device behavior, ambiguous UX, and subjective design review rather than asking engineers to repeat the same exploratory pass.