The next primitive in AI coding is the agent driving the product, watching what happens, and validating its own work
Most of the conversation about AI coding stops at code generation. The agent reads a ticket, writes a diff, opens a PR. The interesting frontier is the loop that closes after the writing. The question is whether the agent can run the thing it just built, look at the result, and decide whether it actually worked.
For the last few weeks, that’s exactly what we’ve been doing internally. We gave Kilo the ability to test our own VS Code extension and CLI end-to-end, so it drives the product and observes what actually happens rather than describing what should. The gap between “wrote code” and “verified it works”
For a browser app, this problem is mostly solved. Point an agent at a dev server, hand it Playwright, teach it how to sign in, and it can click through a real user flow, take a screenshot, and confirm the feature does what you asked. Our cloud repo has worked this way for a while. When an engineer builds a feature there, the agent usually just spins up a Chrome window and exercises it. The models have been trained heavily on this pattern, so it comes naturally. The IDE extension is a much harder target. There are nested iframes within iframes, a UI that doesn’t behave like a normal web page, and a CLI that’s driven through a terminal rather than a DOM. Historically, models struggled to drive it at all.
That’s changed. Computer-use and tool-use ability jumped enough in recent models that an agent can now operate the extension directly - with the newest models, close to reliably.
What “self-testing” actually looks like
The setup is a repository you ask your agent to install. Once it’s in place, Kilo can:
Open VS Code windows and drive the extension the way a user would
Compile the CLI and run it through the terminal, then assert on the output
Run a full end-to-end pass on the feature it just built
Profile and benchmark itself, comparing before and after a change
None of this happens invisibly. The windows it opens are right there on screen, fully observable. You can watch the agent drive the extension. It’s genuinely fun to watch — and, more importantly, it means a human can catch it going sideways.
The benchmarking piece was the most useful part, and the least expected. Because the agent can run its own profiling, capture a baseline, make a change, and re-measure, you can hand it a loop like: baseline this feature, optimize it, re-benchmark, repeat until it’s under 50 milliseconds. One of us spent two weeks doing basically nothing but pointing the agent at performance issues and letting it grind them down. That’s not a workflow that existed a year ago.
Why this matters more than another test suite
The obvious reaction is “so it’s just automated testing.” It isn’t, and that difference matters for how you use it.
A traditional test suite encodes what a human decided to check, once, in the past. Self-testing lets the agent decide what to check now, for the change it’s making, and then actually go check it. As a reviewer, instead of manually clicking through four community PRs, you read the code, decide which edge cases are worth exercising, and let the agent verify them live. The human stays in the loop for judgment; the agent handles the mechanical verification.
It also reframes what the model is for. We tend to think of the harness — the tools, the context, the ability to see and act — as scaffolding around the model. But the harness is often what separates a demo from something you’d actually ship. A model that can see its own output produces dramatically higher-quality work than one flying blind and hoping the diff was right.
Where this goes
If computer-use ability keeps improving at the current rate, agents driving their own applications stops being a clever trick and becomes the default for every app, not just the browser-friendly ones. The extension was the hard case, and it’s now mostly working. The near-term extensions are straightforward: let the agent verify it can run cloud agents, or deploy a client on Docker, without requiring a pile of secrets — the kind of check that catches “works on my machine” before a contributor hits it.
The longer-term shift is more interesting. When the agent can build, run, observe, profile, and re-run on its own, the human job moves up a level — from writing and verifying, to deciding what’s worth building and what “good” looks like. That’s the version of Kilo Speed we’re actually chasing: a tighter loop between intent and a working, verified result. It’s less about typing speed than about closing that gap.