{"slug": "accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act", "title": "Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 2", "summary": "Amazon Web Services announced QA Studio, a reference solution for agentic QA automation using Amazon Nova Act, that enables batch regression testing and CI/CD pipeline integration. The tool groups use cases into test suites with parallel execution on AWS Fargate and provides a command-line interface for automated pipelines. This extends the previous post's foundation to support organized regression testing and deployment gating.", "body_md": "[Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/)\n\n# Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 2\n\nProduction quality assurance (QA) workflows require more than individual test execution. You must organize tests into regression suites that run as a batch, and integrate them into continuous integration and continuous delivery (CI/CD) pipelines so that test results gate deployments automatically.\n\nIn a [previous post](https://aws.amazon.com/blogs/machine-learning/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act/), we introduced QA Studio, a reference solution for agentic QA automation built with Amazon Nova Act. We showed how to define individual use cases in natural language, run them on demand through AI-powered visual navigation, and inspect execution artifacts with full trajectory visibility.\n\nIn this post, we extend that foundation to demonstrate how QA Studio addresses batch regression testing and pipeline integration through test suites that organize and parallelize execution, and a command-line interface that brings agentic testing into automated CI/CD pipelines.\n\n# Test suites for organized regression testing\n\nWith QA Studio, you can group individual use cases, each validating a specific user journey, into collections called test suites that run together. These test suites support structured regression testing across functional areas.\n\nSuites execute as a batch with parallel processing: when a suite runs, each use case executes independently on its own Amazon Elastic Container Service (Amazon ECS) on AWS Fargate worker task. Because each use case runs on its own Fargate worker task, a suite of 20 tests can execute concurrently rather than sequentially. This reduces total suite duration relative to serial execution.\n\nYou can organize suites by functional area, release stage, or testing purpose. Examples include smoke tests that validate critical paths on every deployment, regression suites that run across the full application, and integration tests that verify cross-feature workflows before release.\n\n## Creating and managing test suites\n\nYou create test suites in the QA Studio web interface by providing a name, description, and optional tags, then add existing use cases to the suite. Each use case retains its own configuration, including starting URL, variables, secrets, and headers. When the suite executes, these configurations apply independently to each use case.\n\n*Figure 1 — Test suite detail page showing use cases and execution history*\n\n## Suite execution and results\n\nWhen a suite executes, QA Studio creates individual execution records for each use case and dispatches them to the worker queue. The suite execution page provides an aggregate view: how many use cases succeeded, failed, or are still running. You can drill into individual execution results to review trajectory logs, screenshots, and session recordings for any failed test.\n\nEach suite maintains its own execution history, giving you a longitudinal view of regression stability. Consistent passes build confidence in the tested functionality, while intermittent failures highlight areas that need attention.\n\n*Figure 2 — Suite execution results showing aggregate status and individual use case outcomes*\n\n# CI/CD integration with the QA Studio CLI\n\nThe QA Studio web interface works well for interactive test creation and on-demand execution. CI/CD pipelines require a different interface: command-line execution with structured output, non-interactive authentication, and exit codes that integrate with pipeline orchestrators.\n\nThe QA Studio CLI (`qa-studio`\n\n) provides this interface. It connects to the same API backend as the web application. But instead of dispatching tests to Fargate workers, it runs them with Amazon Nova Act on the machine where the CLI executes, such as a CI/CD runner. Results are reported back to the QA Studio deployment.\n\n## Installing and authenticating\n\nThe QA Studio CLI is part of the project’s GitHub repository. Clone the repository, then install the CLI as a Python package with its optional runner dependencies:\n\nFor CI/CD environments, the CLI supports OAuth 2.0 client credentials authentication. You create an OAuth client in the QA Studio web interface with the required scopes (`api/suite.read`\n\n, `api/suite.write`\n\n, `api/executions.read`\n\n, `api/executions.write`\n\n, `api/usecases.read`\n\n, `api/usecases.execute`\n\n), then configure the credentials as pipeline environment variables:\n\nThe CLI automatically requests and caches access tokens, refreshing them when they expire. No interactive browser login is required.\n\n## Running tests and suites\n\nThe `qa-studio run`\n\ncommand executes individual use cases or entire test suites:\n\n## Environment and variable overrides\n\nCI/CD pipelines often need to run the same tests against different environments. The CLI supports several override mechanisms that modify test behavior without changing the test definitions stored in QA Studio.\n\nThe `--base-url`\n\nflag replaces the domain of the starting URL while preserving the path and query parameters. A single test can then target development, staging, or production environments:\n\nThe `--var`\n\nflag overrides template variables defined in the use case. Variables referenced in test steps using `{{VariableName}}`\n\nsyntax are substituted at runtime. This supports environment-specific configuration without duplicating test definitions:\n\nThe `--region`\n\nflag controls which AWS Region the browser runs in, and `--model-id`\n\nselects the Amazon Nova Act model version:\n\n## Headers and secrets\n\nUse cases can define custom HTTP headers that are sent with every request during test execution. This is useful for authentication tokens, feature flags, or custom identifiers that the application under test requires. Headers are configured in the use case settings and applied automatically during both web interface and CLI execution.\n\nSecrets provide secure storage for sensitive values like passwords, API keys, or tokens. Secrets are stored in AWS Secrets Manager, encrypted at rest. QA Studio is designed so that secret values aren’t written to execution logs or history records. Test steps reference secrets by name, and the actual values are retrieved at runtime. This separation means CI/CD pipelines can execute tests that require credentials without exposing those credentials in pipeline configuration or logs.\n\n## Exit codes and pipeline integration\n\nThe CLI uses exit codes that map directly to pipeline success and failure states:\n\nExit code |\nMeaning |\nPipeline behavior |\n| 0 | All tests passed | Pipeline continues |\n| 1 | One or more tests failed | Pipeline fails (test failure) |\n| 2 | CLI error (auth, configuration, API) | Pipeline fails (infrastructure error) |\n\nThis three-state model allows pipelines to distinguish between test failures (exit code 1) and infrastructure problems (exit code 2). You can configure different notification or retry strategies for each case.\n\nThe `--format`\n\nflag controls output formatting. The default `json`\n\nformat provides structured output for programmatic consumption. The `human`\n\nformat provides a readable summary for pipeline logs:\n\nDuring execution, the CLI creates execution records in QA Studio, updates step statuses in real time, and uploads artifacts including trajectory logs and session recordings. You can monitor CLI-triggered executions from the web interface alongside manually triggered runs, maintaining a unified execution history regardless of how tests were initiated.\n\n*Figure 3 — CLI execution output showing test results*\n\n# CI/CD platform examples\n\nThe following examples demonstrate QA Studio integration with common CI/CD tools. Each example assumes OAuth client credentials and AWS credentials are stored as pipeline secrets.\n\n## GitHub Actions\n\nThe `if: always()`\n\ncondition on the artifact upload step verifies that test recordings and logs are preserved even when tests fail, providing the debugging context that you need to investigate failures.\n\n## GitLab CI\n\nGitLab CI variables (`OAUTH_CLIENT_ID`\n\n, `OAUTH_CLIENT_SECRET`\n\n, `OAUTH_TOKEN_ENDPOINT`\n\n, `AWS_ACCESS_KEY_ID`\n\n, `AWS_SECRET_ACCESS_KEY`\n\n) should be configured as protected and masked CI/CD variables in the project settings. The `regression-tests`\n\njob uses the GitLab schedule trigger, running only when triggered by a pipeline schedule rather than a code push.\n\n## Jenkins\n\nJenkins uses the `withCredentials`\n\nblock to inject secrets into the build environment without exposing them in console output. The `post.always`\n\nblock archives test artifacts regardless of build outcome.\n\n# Conclusion\n\nTest suites and CI/CD integration extend QA Studio from an interactive test creation tool into a platform for continuous quality assurance. Test suites organize regression coverage into manageable collections with parallel execution. The CLI brings agentic test execution into automated pipelines with environment overrides, secure credential handling, and exit codes that map to pipeline success and failure states.\n\nThese capabilities build on the foundation described in our previous post: natural language test definitions, AI-powered visual navigation, and end-to-end trajectory visibility. Together, they demonstrate how agentic QA automation with Amazon Nova Act can integrate into existing software delivery workflows, providing automated quality feedback without requiring you to maintain framework-specific test code.\n\nIn a future post, we plan to explore how agentic test automation can extend to mobile applications.\n\nThe QA Studio reference solution, including test suites and CLI integration, is available on [GitHub](https://github.com/aws-samples/sample-qa-studio). For deployment instructions and detailed documentation, see the project README.", "url": "https://wpnews.pro/news/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act", "canonical_source": "https://aws.amazon.com/blogs/machine-learning/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act-part-2/", "published_at": "2026-07-14 16:47:32+00:00", "updated_at": "2026-07-14 16:54:22.733331+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "artificial-intelligence", "developer-tools"], "entities": ["Amazon Web Services", "Amazon Nova Act", "QA Studio", "AWS Fargate", "Amazon Elastic Container Service"], "alternates": {"html": "https://wpnews.pro/news/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act", "markdown": "https://wpnews.pro/news/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act.md", "text": "https://wpnews.pro/news/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act.txt", "jsonld": "https://wpnews.pro/news/accelerating-software-delivery-with-agentic-qa-automation-using-amazon-nova-act.jsonld"}}