How to test pull requests from coding agents on preview deploys before you review them

A Momentic engineering guide recommends that reviewers test coding-agent pull requests against each PR's deployed Vercel or Netlify preview URL before reviewing the diff, because code review alone cannot show whether changed flows work in a browser. The guide provides a GitHub Actions workflow that triggers on `pull_request`, polls `listDeployments` up to 30 times at 10-second intervals to resolve the preview URL for the PR's head commit, and passes it to the test suite as `PREVIEW_URL` so end-to-end tests run against the deployed build rather than localhost. It also advises giving the coding agent a way to run the test suite before committing or opening the PR, and running an exploratory pass with Mo against the preview when a PR changes behavior no test covers.

TL;DR - For agent-authored PRs, test the deployed preview URL before reviewing the diff. A code review cannot show whether the changed flow works in a browser. - Run end-to-end tests in GitHub Actions https://momentic.ai/docs/running-tests/ci/github-actions against each PR’s Vercel or Netlify preview URL. The test target should be the deployed build, not localhost. - Give the coding agent a way to run the test suite before it commits or opens the PR. The agent can inspect failures and revise its changes before CI runs. - When a PR changes behavior that no test covers, run an exploratory pass with Mo https://momentic.ai/mo against the preview and review its findings alongside the PR. Why code review alone doesn't catch what agent PRs break Code review alone is not enough for agent-generated code. A reviewer can inspect a diff for intent, correctness, and maintainability, but the diff does not show how the deployed application behaves with real browser state or environment settings. An agent might change a shared sign-in handler and update the tests beside it. The handler may look correct while a checkout flow loses the user’s session after a redirect. A reviewer who focuses on the changed files may never open checkout, and a test confined to the sign-in page will not exercise that path. As coding agents produce more changes, reviewers have more behavior to verify. Generating a plausible patch does not establish that unaffected pages still work. Diff review remains necessary, but approval of an agent-written PR also needs evidence from the running application, especially for flows that cross pages or depend on deployed configuration. Run end-to-end tests against the preview URL in CI An agent-written PR needs a browser check against the code reviewers will actually open. A local test run can pass while the Vercel or Netlify preview fails because of deploy configuration, missing environment variables, or a route that behaves differently after deployment. Run the critical end-to-end flows against the PR’s preview URL after the deploy succeeds. A GitHub Actions job can trigger on pull request , look up a successful preview deployment for the PR’s head commit, and pass its URL to the test suite. For example, if your host records previews as GitHub deployments, a github-script step can resolve the URL before Momentic runs. on: pull request permissions: contents: read deployments: read jobs: preview-e2e: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - id: preview uses: actions/github-script@v7 with: script: | const { owner, repo } = context.repo; const sha = context.payload.pull request.head.sha; for let attempt = 0; attempt < 30; attempt++ { const { data: deployments } = await github.rest.repos.listDeployments { owner, repo, sha, per page: 100 } ; for const deployment of deployments { if deployment.environment == 'Preview' continue; const { data: statuses } = await github.rest.repos.listDeploymentStatuses { owner, repo, deployment id: deployment.id, per page: 1 } ; const status = statuses 0 ; if status?.state === 'success' && status.environment url { core.setOutput 'url', status.environment url ; return; } } await new Promise resolve = setTimeout resolve, 10000 ; } core.setFailed 'No successful preview deployment for the PR commit' ; - run: npx momentic run --labels critical --reporter steps --upload-results env: PREVIEW URL: ${{ steps.preview.outputs.url }} Configure the test suite to use PREVIEW URL as its application URL. The environment variable in the workflow does not, by itself, change where a test navigates. Match the Preview deployment filter to the environment name your host uses, and keep the commit check so CI cannot accidentally test an older preview. If Vercel or Netlify posts the URL in a PR comment instead of a GitHub deployment status, replace the lookup step with a resolver for that comment. The critical label keeps the PR gate focused on flows that must pass before review or merge. The steps reporter makes failures readable in CI, while uploaded results give the reviewer evidence to inspect. Claude Code https://momentic.ai/blog/claude-code-for-end-to-end-testing , Codex, and Devin PRs use the same gate as human-written PRs. Agent authorship changes the need for independent verification, not the mechanics of targeting a preview. Make the agent run tests before it opens the PR Codex https://momentic.ai/blog/codex-ui-testing-guide can catch a broken flow before it commits if it can run the application and its end-to-end tests in its workspace. Give the agent a test-running path, then instruct it to run the tests after changing code, inspect failures, fix the cause, and rerun the affected tests before committing. Install the Momentic skill in the workspace where Codex works. npx momentic skills --yes The Momentic MCP server provides another way to give a coding agent access to testing tools. Whichever integration you use, give the agent access to the application and the existing YAML tests. The agent should run those tests against its local instance and report what passed or failed with the PR. A passing run should refer to the final code, not an earlier revision the agent subsequently changed. Keep this pre-commit run separate from the GitHub Actions check. The agent tests the code it intends to submit and can iterate while it still has the local environment available. CI runs the committed revision against the Vercel or Netlify preview deploy, where deployment configuration or runtime behavior may differ. If the agent cannot start the app or run the tests in its workspace, record that limitation rather than treating an unrun test suite as a pass. Exploratory testing on the preview when there's no test yet An agent-written PR can change a flow that no existing end-to-end test covers. For example, a new checkout step may render on the preview deploy while blocking users who return with an existing cart. A reviewer skimming the diff may miss the runtime behavior, and a fixed test suite cannot check a path nobody has added to it. Use Mo for an exploratory pass against the live PR preview. Install the GitHub App, then start the documented PR flow. qa start --pr The flow captures the PR revision and its preview URL, so the findings refer to a specific version of the code rather than whichever deploy happens to be live later. Give Mo a brief that names the changed behavior and the paths worth trying. Specify how to sign in, which account or test data it may use, and what actions are out of bounds, such as sending real emails or placing orders. Read Mo’s findings alongside the preview and the diff. Evidence from an attempted flow can expose behavior the agent did not test and the reviewer did not think to inspect. Exploratory findings still need engineering judgment. Confirm a reported failure, decide whether it belongs to this PR, and add a repeatable test when the behavior should remain covered on later commits. Putting it together: a verification loop for agent PRs An agent-authored PR needs three checks because each catches a different failure mode. Before committing, the agent runs the existing tests locally and uses failures to revise its code. That catches known regressions before the reviewer sees the PR, but it cannot verify the deployed artifact. Once the PR has a preview URL, CI runs the critical end-to-end tests against that Vercel or Netlify deploy. A required check https://momentic.ai/blog/ci-e2e-test-gating can block merge if the tests fail, while --upload-results gives the reviewer evidence to inspect. The preview run can catch differences between the agent’s local environment and the code that actually deployed. Changes without tests need a separate exploratory pass. Give Mo the PR preview and a brief describing the changed behavior and relevant boundaries. If Mo finds a failure, its report gives the agent or reviewer repro steps to investigate. You can then add a test for confirmed behavior before a later PR depends on it. Keep the failure path connected to the author of the change. The agent fixes local failures before committing, CI returns deployed failures to the PR, and exploratory findings return to the agent or reviewer for confirmation. The reviewer still decides whether the evidence covers the behavior the PR was meant to change. Engineering takeaway As coding agents write more of a PR, the preview deploy becomes the thing worth testing. A diff lets you inspect the code, but the preview lets you check whether the committed changes work in the deployed app. Human review still matters for intent and maintainability. To get started with Momentic, run npx @momentic/wizard@latest .