{"slug": "our-ai-builder-said-done-when-the-output-matched-a-regex", "title": "Our AI builder said \"done\" when the output matched a regex", "summary": "Zugo, a tool that turns a sentence into a running app, has improved its build verification by replacing a regex check for a closing HTML tag with an in-frame error listener. The new system injects a script into the generated document to report errors via postMessage, distinguishes console warnings from failures, and ensures that once a build is marked bad, it cannot be marked good again. This addresses the issue where builds were falsely marked successful even if the page failed to render.", "body_md": "We build [Zugo](https://zugo.dev), which turns one written sentence into a running game, site or app. For a while, the thing that decided whether a build had succeeded was this:\n\n```\nfunction htmlComplete(text) {\n return /<\\/html>/i.test(text);\n}\n```\n\nA closing tag. That was the gate. If the model emitted ``\n\n, the turn was a success, the green tick appeared, and the credits were charged. Whether the page actually opened was not part of the decision, because at the moment we charged, nothing had mounted the document anywhere.\n\nIf you are building anything that generates code, this is the trap. Writing the file is easy to observe. Whether the file works is not, so it quietly stops being what \"done\" means.\n\nOur second-line check counted opening and closing tags. Its own header admitted what it was: a balance check, not \"it renders.\" A document can be perfectly balanced, end in ``\n\n, and throw on the first line of its first script. That build billed as a complete success.\n\nWorse, the failure was invisible in exactly the way that costs the most trust: the tick went green the instant the stream finished, and the real answer arrived seconds later. Measured on our own paths, that gap is 2.6s for a site, 4.2s for a game and 6.2s for a React app, plus one silent re-mount if the first verdict is bad. So a broken build showed a green tick over a blank white frame for up to about twelve seconds. Long enough to read it as the product lying to you and close the tab.\n\nThe obvious fix is to wrap the preview in an error boundary and listen. That does not work here, and the reason is worth knowing if you sandbox generated code.\n\nThe preview frame is `sandbox=\"allow-scripts\"`\n\nwith **no** `allow-same-origin`\n\n. That gives it an opaque origin, which is the whole point: a generated document must not be able to reach the host app, its storage or its session. But the same wall means the host cannot see inside. A React error boundary around the preview catches nothing from the document. If a user ever sees our error boundary, the **editor** crashed, not their build.\n\nSo the listener has to live inside the frame, which means we inject it.\n\nWe prepend a small script to the generated document that reports `error`\n\n, `unhandledrejection`\n\nand `console.error`\n\nback to the host via `postMessage`\n\n. Two details decide whether it is worth anything.\n\n**It has to be in , ahead of everything the build wrote.** Our first version appended the hook before\n\n``\n\n. A script that threw during load ran before any listener existed, so the loudest failures were exactly the ones nothing reported. The ordering is now pinned by a test that asserts the hook's index is lower than both ``\n\nand any script the build placed in the body.**A console.error is not a failure.** React logs ordinary development warnings through\n\n`console.error`\n\n, and \"each child in a list should have a unique key\" is not a broken app. So the harness reports it as a separate field:\n\n``` js\nvar ce = console.error;\nconsole.error = function () {\n rep({ logErr: [].map.call(arguments, s).join(\" \").slice(0, 400) });\n return ce.apply(console, arguments);\n};\n```\n\nEverything downstream that decides pass or fail checks `if (d.logErr) return;`\n\nbefore it counts anything. The warning is still recorded, it just never fails a healthy build. Collapsing these two into one channel is how you get a verifier that cries wolf until people stop reading it.\n\nThe last rule is the one I would keep if I had to throw the rest away. Verdicts only travel in one direction:\n\n```\nif (tn.files !== vFiles || (tn.rendered && (!tn.rendered.ok || v.ok))) return tn;\n```\n\nOnce a build has been marked bad, no later signal is allowed to mark it good. A second render, a retry, a re-mount, an error that happens not to repeat: none of them can promote a failure back to a success. Combined with a 9 second window after mount during which errors still count, this closes the obvious escape hatch where a flaky build eventually reports clean and the user is told everything is fine.\n\nAnd the green tick now waits. Between \"the stream finished\" and \"the frame answered\" the UI says it is checking whether the build opens. Neutral, not a claim. We would rather show uncertainty for three seconds than certainty that turns out to be wrong.\n\nThe bug was not the regex. The regex was fine at what it did. The bug was letting the cheap observable stand in for the expensive one, because the expensive one arrives late and the cheap one is available right now.\n\nThree questions worth asking about your own pipeline:\n\nIf you want to watch the version with the harness in it, [Zugo](https://zugo.dev) is free to try, and the [templates](https://zugo.dev/templates) are all real builds you can open and edit.", "url": "https://wpnews.pro/news/our-ai-builder-said-done-when-the-output-matched-a-regex", "canonical_source": "https://dev.to/zugodev/our-ai-builder-said-done-when-the-output-matched-a-regex-agi", "published_at": "2026-08-02 07:04:40+00:00", "updated_at": "2026-08-02 07:41:23.286899+00:00", "lang": "en", "topics": ["developer-tools", "generative-ai"], "entities": ["Zugo"], "alternates": {"html": "https://wpnews.pro/news/our-ai-builder-said-done-when-the-output-matched-a-regex", "markdown": "https://wpnews.pro/news/our-ai-builder-said-done-when-the-output-matched-a-regex.md", "text": "https://wpnews.pro/news/our-ai-builder-said-done-when-the-output-matched-a-regex.txt", "jsonld": "https://wpnews.pro/news/our-ai-builder-said-done-when-the-output-matched-a-regex.jsonld"}}