When developing an application or service, one of the common questions we face is how many concurrent requests it can handle and how the services it depends on will respond. Even if we verify that all functionalities work correctly, this does not guarantee that the system will maintain its performance as demand increases.
This is why software testing goes beyond verifying functional requirements. It also allows us to evaluate aspects such as performance, stability, and responsiveness. In particular, load testing is not simply about pushing a system until it fails, but rather about understanding how it behaves under different conditions and identifying when it stops meeting the expected requirements.
In the previous article, we built a service to identify duplicate products using Amazon Bedrock, S3 Vectors, and DynamoDB, including an AWS Lambda function to perform real-time searches. During our initial tests, the service responded within a few seconds, but one question remained unanswered: how would it behave if many users accessed it simultaneously?
To answer this question, in this second part, we will expose the service through Amazon API Gateway and evaluate its behavior under load. Before running the tests, however, we need to define what we want to measure and which criteria we will use to evaluate the results. To do this, we will introduce some fundamental concepts of Site Reliability Engineering (SRE). We will then automate load testing with Grafana k6 on AWS and, finally, generate a report in which generative AI will help us interpret the collected metrics.
Before evaluating how our API behaves under load, we need to establish what acceptable performance and reliability actually mean. A service may continue responding to requests, but that does not necessarily mean it is meeting the expected performance requirements.
This is where Site Reliability Engineering (SRE) comes into play. Originally developed at Google, SRE is a discipline that applies software engineering principles to operating and maintaining reliable systems. It focuses on defining measurable objectives, selecting indicators that reflect service behavior, and using automation to reduce repetitive operational tasks.
For our use case, these concepts provide a foundation for moving beyond simply checking whether the API responds. Instead, we can evaluate specific indicators such as response times, error rates, and performance under different load conditions.
SRE is built on a set of principles that guide how reliable services are designed, operated, and maintained. These range from defining reliability objectives and managing risk to automation, monitoring, and keeping systems as simple as possible.
The following infographic summarizes seven of these principles, based on Google's Site Reliability Engineering book.
Illustration by the author. Based on Google's Site Reliability Engineering book.
Service reliability depends not only on availability but also on the ability to handle demand while maintaining acceptable performance. In this context, SRE includes several practices aimed at anticipating problems, optimizing resources, and reducing the impact of changes in production.
Among these practices, we can highlight:
Evaluating a service's capacity and performance requires more than simply running tests and collecting metrics. We also need to define what we want to measure and which service levels we consider acceptable.
In this context, SRE introduces three related concepts: SLI (Service Level Indicator), SLO (Service Level Objective), and SLA (Service Level Agreement). Together, they connect technical measurements with service objectives and commitments.
The following infographic summarizes what each concept represents, the question it helps answer, and a practical example of how it can be applied.
Illustration by the author. Based on Google's SRE framework.
Grafana k6 is an open-source tool developed in Go and maintained by Grafana Labs, designed to perform load and performance testing on applications, APIs, and microservices. Test scenarios are defined using JavaScript, making it easier to automate tests and integrate them into development workflows.
Although we often associate these tools with stress testing, k6 allows us to evaluate how a system behaves under different load conditions, without necessarily pushing it to the point of failure.
Illustration by the author. Based on Grafana k6 documentation.
Depending on what we want to evaluate, we can define different test scenarios:
To execute these tests, k6 provides several components that allow us to configure load, validate responses, collect metrics, and integrate results with other tools.
We can group them into four main components:
To run load tests on AWS, we start with the game-catalog-resolve-product Lambda function developed in the previous article, which uses Amazon Bedrock, S3 Vectors, and DynamoDB to identify duplicate products.
In this second part, we expose the function through Amazon API Gateway (HTTP API) with IAM authorization, requiring k6 requests to be signed using AWS credentials.
To automate the testing process, we use AWS Step Functions to orchestrate the execution of k6 on Amazon ECS Fargate and the subsequent report generation through an AWS Lambda function. Test scripts and results are stored in Amazon S3, while Amazon ECR stores the container images.
The entire infrastructure is defined using Terraform, and the code is available on GitHub, allowing the solution to be reproduced.
Architecture diagram by the author.
In this use case, we will run different load testing scenarios against our API. To do this, we use Grafana k6 running inside a Docker container to send requests and collect performance metrics. Once the tests are complete, a Python-based process calculates the results and generates a PDF report that includes an interpretation provided by Amazon Bedrock.
The project is available on GitHub: game-catalog-k6.
Serverless pipeline that runs Grafana k6 load tests against an API on Amazon ECS Fargate orchestrated by AWS Step Functions, and generates a PDF report where Amazon Bedrock interprets the metrics. All the infrastructure is defined with Terraform.
📝 Article: 🔗 Part 1 (the API under test): Identical or Similar? AI-Powered Product Matching with Amazon Bedrock and S3 Vectors · repository
The thresholds…
game-catalog-k6/
├── Dockerfile, entrypoint.sh
├── search.js, sigv4.js
├── scenarios/
├── queries.json
├── report/
│ └── config/
├── terraform/
└── build-and-push.sh
The goal is to make this solution reusable for testing other APIs without modifying the underlying infrastructure or the report generation process. To do this, we only need to configure three elements:
api_id
In our case, we use Amazon API Gateway (HTTP API) with IAM authorization. The service URL follows this structure:
https://<api-id>.execute-api.us-east-1.amazonaws.com
The API identifier can be found in the AWS Management Console under API Gateway → Your API → API details.
To configure the project, we specify it in terraform/terraform.tfvars:
api_id = "<api-id>"
Terraform uses this identifier to construct the API URL and configure the necessary permissions for the k6 container to invoke the API using IAM-signed requests.
Once we have identified the API, we need to define which requests we want to send.
Our API accepts text searches through POST /search. To define the requests executed during testing, we use the queries.json file, which contains different video game searches in English and Spanish.
[
{"query": "fifa 14 ps3 digital code", "lang": "en"},
{"query": "fifa 14 ps3 codigo digital", "lang": "es"},
{"query": "gta vice city ps2", "lang": "en"},
{"query": "el mario kart de la wii", "lang": "es"}
]
If the API expects a different request structure, we will also need to adapt the request body in search.js.
scenarios
Once we have defined what to request, we need to establish how the load will be applied. For this, we use JSON files in the scenarios/ directory, each with its own configuration.
Each JSON file defines a different load testing scenario:
Each file allows us to configure virtual users, test duration, load patterns, and performance thresholds.
For example, smoke.json contains:
{
"name": "smoke",
"description": "1 user for 1 minute: checks that the API works before adding load.",
"queries_file": "queries.json",
"endpoint": "/search",
"think_time_seconds": 1,
"executor": {
"executor": "constant-vus",
"vus": 1,
"duration": "1m"
},
"thresholds": {
"http_req_failed": ["rate<0.01"],
"http_req_duration": ["p(95)<5000"]
}
}
The main parameters are:
Once the configuration files are ready, we can deploy the solution using AWS CloudShell, with the tools needed to run Terraform and build Docker images.
AWS CloudShell does not include Terraform by default. We can download it to our home directory, which persists across sessions:
curl -sLo /tmp/terraform.zip https://releases.hashicorp.com/terraform/1.9.8/terraform_1.9.8_linux_amd64.zip
mkdir -p ~/bin && unzip -o /tmp/terraform.zip -d ~/bin
echo 'export PATH="HOME/bin:$PATH"' >> ~/.bashrc && source ~/.bashrc
terraform version
git clone https://github.com/RominaElenaMendezEscobar/game-catalog-k6.git
cd game-catalog-k6/terraform
echo 'api_id = "<api-id>"' > terraform.tfvars
terraform init
terraform apply \
-target=aws_ecr_repository.k6 \
-target=aws_ecr_repository.report
../build-and-push.sh
terraform apply
The deployment is performed in three stages: first, we create the Amazon ECR repositories; next, we build and push the container images; and finally, we deploy the remaining resources.
Terraform provisions the required infrastructure, including Amazon ECS Fargate, AWS Step Functions, AWS Lambda, Amazon S3, and IAM roles and permissions. It also uploads the test scenarios and queries that k6 will use to Amazon S3.
With the infrastructure deployed, we can start a test execution using AWS Step Functions, specifying the scenario we want to run.
For example, to execute load.json:
aws stepfunctions start-execution \
--state-machine-arn "$(terraform output -raw state_machine_arn)" \
--input '{"scenario": "load", "label": "baseline"}'
Screenshot by the author. AWS Management Console.
The state machine orchestrates three steps:
run_id). This identifier is used in the following two steps to ensure that the results and report are associated with the same test execution.runs/<scenario>/<date>/, along with the configuration used for the test.
Screenshot by the author. Amazon S3 bucket showing the files and results generated by the automated load testing pipeline.
Once the test is complete, the pipeline processes the k6 results and generates a PDF report that combines the calculated metrics with an interpretation provided by Amazon Bedrock.
For this stage, we use an AWS Lambda function deployed as a container image, as we need several Python libraries: pandas for data processing, matplotlib for generating charts, and fpdf2 for creating the PDF document.
Unlike traditional ZIP-based deployments, which have a 250 MB uncompressed size limit, AWS Lambda supports container images of up to 10 GB. This makes it easier to package the code and its dependencies into a reproducible environment while maintaining a serverless execution model.
The following infographic summarizes the main benefits of this approach.
Illustration by the author. Based on AWS Lambda documentation.
The container image is built using a Dockerfile and pushed to Amazon ECR. Terraform then configures the AWS Lambda function to use this image, as illustrated in the following workflow.
Illustration by the author. Based on AWS documentation.
We can view an example of the generated PDF report at the following link:
Report generated by the author using Python, Grafana k6, and Amazon Bedrock.
The report consists of several pages organized into the following sections:
It is important to emphasize that all metrics are calculated using Python code based on the k6 results. Amazon Bedrock is used only to interpret these values and generate the narrative analysis, with an additional validation step to verify the numerical values mentioned in the generated text.
In this article, we started with a question: how does an API behave as concurrency increases? To answer it, we implemented an automated load testing pipeline that allows us to evaluate API performance under different load scenarios and analyze the results.
This exercise allowed us to put some Site Reliability Engineering (SRE) concepts into practice: defining which aspects of the service we want to measure (SLIs), establishing what service levels we consider acceptable (SLOs), and using Grafana k6 to translate those objectives into measurable thresholds during testing.
Beyond running load tests, we also need to interpret a considerable amount of performance data. To address this, we incorporated generative AI to facilitate the analysis and generate a report that summarizes the key findings without requiring an in-depth understanding of every metric. It is important to emphasize that the metrics are calculated by code, while AI is used only to interpret them, maintaining a clear separation between the collected data and the generated conclusions.
Another important aspect is the solution's reusability. By modifying the API, queries, and load testing scenarios, we can use the same pipeline to evaluate other services. In addition, preserving the configuration and results of each execution provides traceability and facilitates future comparisons.
However, this MVP has some limitations and opportunities for improvement:
Ultimately, the goal is not simply to determine how many requests an API can handle, but to establish a reproducible process that allows us to measure its behavior, identify potential performance degradation, and evaluate how its performance evolves as we introduce changes.