Building a self-healing scraper agent that won't ship a fix it can't prove

A developer built patchwright, an agent that repairs broken web scrapers, for Google's All Things Agentic Hackathon. The agent diagnoses changes in website layouts, rewrites scrapers, proves fixes in a sandbox, and requires human approval. The project uses Gemini 3.7 Flash for diagnosis and rewriting, and Gemma 4 for review, running on Cloud Run with Firestore state.

I built patchwright for Google's All Things Agentic Hackathon, and this post is my write-up of it, written for the purpose of entering. The short version: it's an agent that repairs broken web scrapers. When a site changes its layout and a scraper stops working, the agent figures out what changed, rewrites the scraper, proves the new version in a sandbox, has a second model review it, and then waits for a human to approve. The longer version is what I actually learned, which is mostly that the interesting problems were not the ones I expected. I run Aroppo https://aroppo.com , a site that aggregates opportunities for creators. It depends on about nineteen scrapers pointed at other people's websites. In one two-week stretch those scrapers made 744 fetches across 380 ingestion runs, and the sites they read redesign whenever they feel like it. When one does, a selector breaks, the data quietly goes to zero or, worse, goes wrong, and I spend an afternoon reading HTML to find out why. So the friction was real before the hackathon gave me a reason to fix it. I want to be straight about prior art: self-healing scrapers exist commercially. I'm not claiming the idea. What I wanted to get right was the part those tools tend to gloss over, which is trust. An agent that rewrites your code and deploys it is only useful if you can believe the fix. The demo runs against a fake listings site I built called Fauxpost. It holds twelve records and renders them through a template. When you "break" it, the code mutates the template only. It never touches the twelve records. That sounds like a small implementation detail. It is the whole thing. Because the data is fixed, "did the patch work?" is always the same question: does the scraper's output match the twelve records the page was built from? There is no fixture to maintain, nothing to drift, and no way for verification to be fooled. A patch that returns twelve nicely-formatted but wrong records fails, because wrong is measurable against something that cannot move. Every other guarantee in the project leans on that one. Gemini 3.7 Flash does the diagnosis and the rewrite, driven through Google's Agent Development Kit for JavaScript. Gemma 4 writes the reviewer's brief. It runs on Cloud Run, keeps state in Firestore, and has a spending cap on it so a runaway loop can't run up a bill. Three dependencies total. The server is plain Node with no framework. Almost none of my time went where I planned. The framework and the models behaved slightly differently from the docs in ways that read as bugs in my own code right up until they didn't. A couple of examples that made me laugh once I understood them. Gemini 3.7 spends tokens thinking before it answers, and I had set the output budget too low, so answers came back chopped off mid-word and looked exactly like a serialization bug. And I named one of my agents s3 diagnose . It read its own name, decided it was working with Amazon S3, and confidently diagnosed a missing bucket permission on a problem that had nothing to do with storage. Names are prompts. Lesson learned. This was the part I was most careful about and still got wrong twice. The agent writes new scraper code, and that code has to run somewhere before a human sees it. My first sandbox was a node:vm context, which felt like enough. It was not. Creating a bare vm context does not give you an empty room; it quietly includes a few globals I did not want, one of which would have corrupted the channel the sandbox uses to report results the first time a patch tried to log something. And a patch that tried to import a Node module threw the error in my process rather than inside the sandbox, which would have taken the whole service down instead of failing one repair. So the sandbox became two layers: a separate process with nothing inherited from the parent, and inside it a vm context stripped down to a fixed allowlist. Neither layer is sufficient alone, and I only know that because I sat down and tried to break out of it myself. The threat here isn't a hacker. It's a language model that writes an infinite loop or reaches for the network to "check" something, and both of those have to fail safely, every time. I had well over a hundred tests passing before I deployed, and deploying still found bugs, because the tests stub the models and never touch the real cloud services. Cloud Logging reserves the field name severity and uses it to set a log entry's level. I had a field called severity describing how badly a scraper was broken, so every successful repair got filed as a CRITICAL error. Separately, my events were being written concurrently and racing on a sequence number, which silently dropped about half of them until I made the numbering a proper transaction. And when I added the Gemma reviewer, the framework refused to accept the model by name, and the smaller of the two Gemma models fell apart into repetition under a strict output format while the larger one answered cleanly. All of those are the kind of thing you cannot mock your way into finding. After a patch passes the sandbox, a separate agent running Gemma writes a short brief for whoever is about to approve it: what changed, what got more forgiving, what got riskier. It is a different model family from the one that wrote the patch, which is the point. The author does not get to review its own work. It is advisory and it stays that way by construction, not by good intentions. The approval endpoint does not read it. It can be slow, rate-limited, or absent, and the approve button still works. It runs on the free tier, so it adds nothing to the bill. A second opinion at the exact moment a person says yes felt worth the effort, and it is the piece I would least want to remove. Two things. First, switching models mattered more than any prompt tuning I could have done. A head-to-head on identical inputs between Gemini 3.5 and 3.7 cut the time roughly in half and made a category of failures disappear, so I stopped tuning prompts entirely. Tuning against zero failures just teaches the model your noise. Second, the failures were never where I guessed. I assumed the hard cases would stump the diagnosis. They never did; the diagnosis was right every time I looked. What slipped was the code generation, small syntax mistakes in an otherwise correct fix. Knowing that changes what I build next, which is a repair step aimed squarely at syntax rather than a smarter diagnostician I do not need.