{"slug": "automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with", "title": "Automated API Load Testing on AWS: Grafana k6 and AI-Powered Reporting with Amazon Bedrock", "summary": "A developer built an automated load-testing pipeline that exposes an Amazon Bedrock and S3 Vectors duplicate-product service through Amazon API Gateway, drives traffic with Grafana k6 on AWS, and uses generative AI on Amazon Bedrock to interpret the collected metrics. The approach grounds test criteria in SRE concepts—SLIs, SLOs, and SLAs—so results are judged against defined response-time and error-rate objectives rather than simple pass/fail checks.", "body_md": "When developing an application or service, one of the common questions we face is how many concurrent requests it can handle and how the services it depends on will respond. Even if we verify that all functionalities work correctly, this does not guarantee that the system will maintain its performance as demand increases.\n\nThis is why software testing goes beyond verifying **functional requirements**. It also allows us to evaluate aspects such as performance, stability, and responsiveness. In particular, **load testing** is not simply about pushing a system until it fails, but rather about understanding how it behaves under different conditions and identifying when it stops meeting the expected requirements.\n\nIn the previous article, we built a service to identify duplicate products using Amazon Bedrock, S3 Vectors, and DynamoDB, including an AWS Lambda function to perform real-time searches. During our initial tests, the service responded within a few seconds, but one question remained unanswered: **how would it behave if many users accessed it simultaneously?**\n\nTo answer this question, in this second part, we will expose the service through Amazon API Gateway and evaluate its behavior under load. Before running the tests, however, we need to define what we want to measure and which criteria we will use to evaluate the results. To do this, we will introduce some fundamental concepts of Site Reliability Engineering (SRE). We will then automate load testing with Grafana k6 on AWS and, finally, generate a report in which generative AI will help us interpret the collected metrics.\n\nBefore evaluating how our API behaves under load, we need to establish what acceptable performance and reliability actually mean. A service may continue responding to requests, but that does not necessarily mean it is meeting the expected performance requirements.\n\nThis is where **Site Reliability Engineering (SRE)** comes into play. Originally developed at Google, SRE is a discipline that applies software engineering principles to operating and maintaining reliable systems. It focuses on defining measurable objectives, selecting indicators that reflect service behavior, and using automation to reduce repetitive operational tasks.\n\nFor our use case, these concepts provide a foundation for moving beyond simply checking whether the API responds. Instead, we can evaluate specific indicators such as **response times, error rates, and performance under different load conditions.**\n\nSRE is built on a set of principles that guide how reliable services are designed, operated, and maintained. These range from defining reliability objectives and managing risk to automation, monitoring, and keeping systems as simple as possible.\n\nThe following infographic summarizes seven of these principles, based on Google's Site Reliability Engineering book.\n\n`Illustration by the author. Based on Google's Site Reliability Engineering book.`\n\nService reliability depends not only on availability but also on the ability to handle demand while maintaining acceptable performance. In this context, SRE includes several practices aimed at anticipating problems, optimizing resources, and reducing the impact of changes in production.\n\nAmong these practices, we can highlight:\n\nEvaluating a service's capacity and performance requires more than simply running tests and collecting metrics. We also need to define what we want to measure and which service levels we consider acceptable.\n\nIn this context, SRE introduces three related concepts: **SLI (Service Level Indicator), SLO (Service Level Objective), and SLA (Service Level Agreement)**. Together, they connect technical measurements with service objectives and commitments.\n\nThe following infographic summarizes what each concept represents, the question it helps answer, and a practical example of how it can be applied.\n\n`Illustration by the author. Based on Google's SRE framework.`\n\n**Grafana k6** is an open-source tool developed in Go and maintained by Grafana Labs, designed to perform load and performance testing on applications, APIs, and microservices. Test scenarios are defined using JavaScript, making it easier to automate tests and integrate them into development workflows.\n\nAlthough we often associate these tools with stress testing, k6 allows us to evaluate how a system behaves under different load conditions, without necessarily pushing it to the point of failure.\n\n`Illustration by the author. Based on Grafana k6 documentation.`\n\nDepending on what we want to evaluate, we can define different test scenarios:\n\nTo execute these tests, k6 provides several components that allow us to configure load, validate responses, collect metrics, and integrate results with other tools.\n\nWe can group them into four main components:\n\nTo run load tests on AWS, we start with the `game-catalog-resolve-product` Lambda function developed in the previous article, which uses Amazon Bedrock, S3 Vectors, and DynamoDB to identify duplicate products.\n\nIn this second part, we expose the function through **Amazon API Gateway (HTTP API)** with IAM authorization, requiring k6 requests to be signed using AWS credentials.\n\nTo automate the testing process, we use **AWS Step Functions** to orchestrate the execution of k6 on Amazon ECS Fargate and the subsequent report generation through an AWS Lambda function. Test scripts and results are stored in Amazon S3, while Amazon ECR stores the container images.\n\nThe entire infrastructure is defined using Terraform, and the code is available on GitHub, allowing the solution to be reproduced.\n\n`Architecture diagram by the author.`\n\nIn this use case, we will run different load testing scenarios against our API. To do this, we use Grafana k6 running inside a Docker container to send requests and collect performance metrics. Once the tests are complete, a Python-based process calculates the results and generates a PDF report that includes an interpretation provided by Amazon Bedrock.\n\nThe project is available on GitHub: [game-catalog-k6](https://github.com/RominaElenaMendezEscobar/game-catalog-k6).\n\nServerless pipeline that runs **Grafana k6** load tests against an API on **Amazon ECS Fargate**\norchestrated by **AWS Step Functions**, and generates a **PDF report** where **Amazon Bedrock**\ninterprets the metrics. All the infrastructure is defined with **Terraform**.\n\n📝 **Article:**\n🔗 **Part 1 (the API under test):** [Identical or Similar? AI-Powered Product Matching with Amazon Bedrock and S3 Vectors](https://dev.to/aws-builders/identical-or-similar-ai-powered-product-matching-with-amazon-bedrock-and-s3-vectors-3pm) · [repository](https://github.com/RominaElenaMendezEscobar/aws-games-products)\n\nThe thresholds…\n\n```\ngame-catalog-k6/\n├── Dockerfile, entrypoint.sh\n├── search.js, sigv4.js\n├── scenarios/\n├── queries.json\n├── report/\n│   └── config/\n├── terraform/\n└── build-and-push.sh\n```\n\nThe goal is to make this solution reusable for testing other APIs without modifying the underlying infrastructure or the report generation process. To do this, we only need to configure three elements:\n\n`api_id`\nIn our case, we use Amazon API Gateway (HTTP API) with IAM authorization. The service URL follows this structure:\n\n```\nhttps://<api-id>.execute-api.us-east-1.amazonaws.com\n```\n\nThe API identifier can be found in the AWS Management Console under API Gateway → Your API → API details.\n\nTo configure the project, we specify it in `terraform/terraform.tfvars`:\n\n```\napi_id = \"<api-id>\"\n```\n\nTerraform uses this identifier to construct the API URL and configure the necessary permissions for the k6 container to invoke the API using IAM-signed requests.\n\nOnce we have identified the API, we need to define which requests we want to send.\n\nOur API accepts text searches through `POST /search`. To define the requests executed during testing, we use the `queries.json` file, which contains different video game searches in English and Spanish.\n\n```\n[\n  {\"query\": \"fifa 14 ps3 digital code\", \"lang\": \"en\"},\n  {\"query\": \"fifa 14 ps3 codigo digital\", \"lang\": \"es\"},\n  {\"query\": \"gta vice city ps2\", \"lang\": \"en\"},\n  {\"query\": \"el mario kart de la wii\", \"lang\": \"es\"}\n]\n```\n\nIf the API expects a different request structure, we will also need to adapt the request body in search.js.\n\n`scenarios`\nOnce we have defined what to request, we need to establish how the load will be applied. For this, we use JSON files in the scenarios/ directory, each with its own configuration.\n\nEach JSON file defines a different load testing scenario:\n\nEach file allows us to configure virtual users, test duration, load patterns, and performance thresholds.\n\nFor example, `smoke.json` contains:\n\n```\n{\n  \"name\": \"smoke\",\n  \"description\": \"1 user for 1 minute: checks that the API works before adding load.\",\n  \"queries_file\": \"queries.json\",\n  \"endpoint\": \"/search\",\n  \"think_time_seconds\": 1,\n  \"executor\": {\n    \"executor\": \"constant-vus\",\n    \"vus\": 1,\n    \"duration\": \"1m\"\n  },\n  \"thresholds\": {\n    \"http_req_failed\": [\"rate<0.01\"],\n    \"http_req_duration\": [\"p(95)<5000\"]\n  }\n}\n```\n\nThe main parameters are:\n\nOnce the configuration files are ready, we can deploy the solution using AWS CloudShell, with the tools needed to run Terraform and build Docker images.\n\nAWS CloudShell does not include Terraform by default. We can download it to our home directory, which persists across sessions:\n\n```\ncurl -sLo /tmp/terraform.zip https://releases.hashicorp.com/terraform/1.9.8/terraform_1.9.8_linux_amd64.zip\nmkdir -p ~/bin && unzip -o /tmp/terraform.zip -d ~/bin\necho 'export PATH=\"HOME/bin:$PATH\"' >> ~/.bashrc && source ~/.bashrc\nterraform version\ngit clone https://github.com/RominaElenaMendezEscobar/game-catalog-k6.git\ncd game-catalog-k6/terraform\necho 'api_id = \"<api-id>\"' > terraform.tfvars\nterraform init\nterraform apply \\\n  -target=aws_ecr_repository.k6 \\\n  -target=aws_ecr_repository.report\n../build-and-push.sh\n\nterraform apply\n```\n\nThe deployment is performed in three stages: first, we create the Amazon ECR repositories; next, we build and push the container images; and finally, we deploy the remaining resources.\n\nTerraform provisions the required infrastructure, including Amazon ECS Fargate, AWS Step Functions, AWS Lambda, Amazon S3, and IAM roles and permissions. It also uploads the test scenarios and queries that k6 will use to Amazon S3.\n\nWith the infrastructure deployed, we can start a test execution using AWS Step Functions, specifying the scenario we want to run.\n\nFor example, to execute `load.json`:\n\n```\naws stepfunctions start-execution \\\n  --state-machine-arn \"$(terraform output -raw state_machine_arn)\" \\\n  --input '{\"scenario\": \"load\", \"label\": \"baseline\"}'\n```\n\n`Screenshot by the author. AWS Management Console.`\n\nThe state machine orchestrates three steps:\n\n`run_id`). This identifier is used in the following two steps to ensure that the results and report are associated with the same test execution.`runs/<scenario>/<date>/`, along with the configuration used for the test.\n\n`Screenshot by the author. Amazon S3 bucket showing the files and results generated by the automated load testing pipeline.`\n\nOnce the test is complete, the pipeline processes the k6 results and generates a PDF report that combines the calculated metrics with an interpretation provided by Amazon Bedrock.\n\nFor this stage, we use an **AWS Lambda function deployed as a container image**, as we need several Python libraries: `pandas` for data processing, `matplotlib` for generating charts, and `fpdf2` for creating the PDF document.\n\nUnlike traditional ZIP-based deployments, which have a 250 MB uncompressed size limit, AWS Lambda supports container images of up to 10 GB. This makes it easier to package the code and its dependencies into a reproducible environment while maintaining a serverless execution model.\n\nThe following infographic summarizes the main benefits of this approach.\n\n`Illustration by the author. Based on AWS Lambda documentation.`\n\nThe container image is built using a `Dockerfile` and pushed to Amazon ECR. Terraform then configures the AWS Lambda function to use this image, as illustrated in the following workflow.\n\n`Illustration by the author. Based on AWS documentation.`\n\nWe can view an example of the generated PDF report at the following link:\n\n[github.com report.pdf](https://github.com/RominaElenaMendezEscobar/game-catalog-k6/blob/main/output/s3/game-catalog-loadtest/runs/smoke/2026-10-23--22-44-20/report.pdf)\n\n`Report generated by the author using Python, Grafana k6, and Amazon Bedrock.`\n\nThe report consists of several pages organized into the following sections:\n\nIt is important to emphasize that **all metrics are calculated using Python code** based on the k6 results. Amazon Bedrock is used only to interpret these values and generate the narrative analysis, with an additional validation step to verify the numerical values mentioned in the generated text.\n\nIn this article, we started with a question: how does an API behave as concurrency increases? To answer it, we implemented an automated load testing pipeline that allows us to evaluate API performance under different load scenarios and analyze the results.\n\nThis exercise allowed us to put some **Site Reliability Engineering (SRE)** concepts into practice: defining which aspects of the service we want to measure (SLIs), establishing what service levels we consider acceptable (SLOs), and using Grafana k6 to translate those objectives into measurable thresholds during testing.\n\nBeyond running load tests, we also need to interpret a considerable amount of performance data. To address this, we incorporated **generative AI** to facilitate the analysis and generate a report that summarizes the key findings without requiring an in-depth understanding of every metric. It is important to emphasize that the metrics are calculated by code, while AI is used only to interpret them, maintaining a clear separation between the collected data and the generated conclusions.\n\nAnother important aspect is the solution's reusability. By modifying the API, queries, and load testing scenarios, we can use the same pipeline to evaluate other services. In addition, preserving the configuration and results of each execution provides traceability and facilitates future comparisons.\n\nHowever, this MVP has some limitations and opportunities for improvement:\n\nUltimately, the goal is not simply to determine how many requests an API can handle, but to establish a reproducible process that allows us to **measure its behavior, identify potential performance degradation, and evaluate how its performance evolves as we introduce changes.**", "url": "https://wpnews.pro/news/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with", "canonical_source": "https://dev.to/aws-builders/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with-amazon-bedrock-39g", "published_at": "2026-10-26 09:36:06+00:00", "updated_at": "2026-10-26 09:43:44.536028+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "mlops", "generative-ai"], "entities": ["Amazon Bedrock", "Amazon API Gateway", "Grafana k6", "AWS Lambda", "Amazon S3 Vectors", "Amazon DynamoDB", "Google"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with", "markdown": "https://wpnews.pro/news/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with.md", "text": "https://wpnews.pro/news/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with.txt", "jsonld": "https://wpnews.pro/news/automated-api-load-testing-on-aws-grafana-k6-and-ai-powered-reporting-with.jsonld"}}