Testing and Verification in AI SDLC, Technical Debt Management and Guardrails | Weekly Reflections #1

A developer built a lightweight end-to-end test suite using Pytest and Playwright that runs from a local machine, allowing AI coding agents to execute the tests and validate their own changes without manual QA. The developer also added an API key authenticator enabled only in local and QA environments so tests could reach a service that previously accepted authentication only from inside the cluster, and paired the work with a technical debt management process that treats missing harnesses and guardrails as a fast-growing source of debt.

This week I worked on two things that ended up being really connected to each other: an end-to-end test setup for one of the projects I'm working on, and a technical debt management process for my squad. Before a risky release on this project, we depended on manual testing. I wanted a way out of that, so I built a lightweight end-to-end e2e suite, meaning tests that run the application from outside its infrastructure and rely on inspecting its behavior a black box test . I used Pytest and Playwright, a browser automation library. This kind of setup fits contained services well, like backend APIs and admin panels such as Django Admin. I made sure the tests run from my local machine. That was the most important choice, because it means AI agents can run the suite on their own and validate their own changes. The same suite can be pointed to a local running application or pointed at the QA environment, which is very similar to the production environment. We no longer need manual tests for this project, not at least for the most important feature of it. For the next risky release, we run the suite, and if the change needs more coverage, we add test cases first. With AI, writing new cases is very fast now. There was one thing in the way. The service only accepted authentication from inside the cluster, so the tests couldn't reach it from my local machine. I added an API key authenticator that is only enabled in the local and QA environments, so production stays as secure as before. The new authentication plus the base Pytest/Playwright structure took me less than a day. It was really worth it. The other thing I worked on this week was a technical debt management process. Most of the ideas came from this awesome TDM guide by @Mari-Wie: https://github.com/Mari-Wie/TDMGuide https://github.com/Mari-Wie/TDMGuide . It's a practitioner's guide for setting up technical debt management inside a team. Working on both at the same time, I started to see the lack of harness and guardrails as one of the newest and fastest-growing sources of technical debt. The engineering principles for preventing and fixing debt still work with AI. What changed is that we delegate the implementation to agents instead of writing it ourselves. Everything moves faster, but the time is not zero, so we still need to prioritize. I believe having a healthy technical debt management process is easier now than ever . With proper skills and agents integrated with the ticket system most of them have mcp tools or CLI available , we can record debt as we find it while working on the projects. The speed from coding agents allow us to keep working on that backlog and with the proper process we can always fix the most valuable item first. With guardrails for software architecture and design, security and performance, we avoid or at least mitigate new debt. But we only get the most of the development speed provided by the coding agents if they produce good quality software and if we reduce the time spent on other parts of the SDLC like review, testing and deployment. Those are the new bottlenecks, and guardrails are how we reduce them. Unit, integration and even e2e tests are not enough for that. To automate or semi-automate reviews, deploys and validation, we also need to test and monitor performance, race conditions and other kinds of problems. A definition I keep using: Technical debt is the invisible result of past decisions that makes the software harder to change Kruchten 2012 . Customers see features and defects. Only developers see the debt, and it costs nothing until the system needs to change. Measuring Interest . For each Technical Debt Item, we estimate the hours lost every time it bites us, on a fixed scale e.g 15 min, 1h, 4h, 1 day, 2+ days . Interest times Frequency gives the Interest Burden per time e.g weeks . Repayment Effort how long is gonna take to fix it divided by the Interest Burden gives the ROI in weeks. For example, 3h × 4 times per week = 12h per week, so a 40h fix pays for itself in just over 3 weeks. A Tech Debt Budget for every development cycle/week/sprint. Every sprint has a fixed debt budget. 0% can happen, but not as a permanent state. Contagious Technical Debt. Some debt gets more expensive to fix over time. An API with debt starts with one consumer, and every new consumer is one more thing to change later. Data changes are still hard. Even with agentic coding, data changes and migrations are still hard to change. How do we build performance and race condition checks that an agent can run locally, the same way it runs the e2e suite? How do we make data changes and migrations safer when agents are making more of the changes? Do you know how much your debt is costing you, or is it just a feeling? Coding agents made writing code faster, so review, testing and deployment are now where the time goes. Less than a day of work on tests and authentication removed manual QA from one project, and the debt indicators do something similar for prioritization. Both depend on good guardrails, and that's where I want to spend more time.