AIArticle Boris Cherny's runaway Swift-rewrite experiment shows agent quality is now bounded by verifier quality, not model quality.
Priya Nair Boris Cherny, who created Claude Code at Anthropic, went on stage at Y Combinator's Startup School and described the most instructive agent experiment anyone has run in public this year. He asked Claude — running in Slack, wired to a macOS runner on GitHub and an empty Swift repo — to rewrite Anthropic's Electron-based Claude desktop app in Swift. The whole prompt: run the Electron app in a Mac VM, screenshot it, compare it pixel by pixel to the Swift version, and don't stop until you're done.
At the time of the interview, the task had been running for roughly two weeks. Asked how many agents it had spawned, Cherny guessed thousands, maybe tens of thousands — he wasn't sure. Claude had, unprompted, created an internal Slack channel and started live-blogging its own progress with screenshots every few minutes.
That detail is charming. It is also doing a lot of work in the retelling, because the honest headline is different: fourteen days, thousands of agents, and no shipped app. This experiment is being passed around as evidence that coding agents have arrived. Read carefully, it's evidence of something narrower and more useful — and of one big mistake you shouldn't copy.
Persistence is the new capability, and it's real #
The genuinely new thing here isn't code generation. It's that the loop didn't die. Cherny's claim that Opus 5 can run for "days, weeks, months at a time" without scaffolding matches what the experiment shows: an agent that provisions its own CI runner, sets up its own feedback channel, and keeps grinding through a task nobody expected it to finish quickly. Two years ago the failure mode of every autonomous coding agent was drift — it would lose the plot within an hour and start hallucinating progress. An agent that stays coherent and on-task for two weeks is a different class of tool, whatever it ultimately produces.
Cherny's broader thesis is that this shifts where the skill lives. Prompt engineering, he argued, mostly doesn't matter anymore — his prompt here was three sentences. The skill is picking a task that "seems a little bit too hard" and then building the agent a way to verify its own work. Anthropic eats this cooking internally: Claude Code's team deleted over 80% of its system prompt for Opus 5, on the logic that most of it was correcting behaviors the model now just has. Their process is worth stealing — delete first, use the tool, add instructions back only when a real failure demands one, because you can't predict which instructions the model actually needs.
That's the part practitioners should take seriously. The verification loop is now the product. Your prompt is three sentences; your test harness is the thing you'll spend the week on.
The verifier he chose is the wrong lesson #
Which makes it awkward that the verifier in this experiment is bad, and the Hacker News thread landed on exactly why. Pixel-by-pixel screenshot comparison is a shallow oracle. It checks the one property of a UI that matters least — static appearance — and is blind to everything that makes an app an app: state transitions, error handling, streaming behavior, keyboard shortcuts, accessibility, memory footprint, what happens when the network drops mid-response. An agent optimizing against a pixel diff will happily produce a Swift-shaped screenshot of an Electron app.
John Gruber's critique at Daring Fireball cuts from the other side: pixel-perfect parity isn't just a weak target, it's the wrong target. The Claude desktop app's problem was never only that it's Electron — it's that it isn't designed like a Mac app. A Swift rewrite whose success criterion is "indistinguishable from the Electron version" faithfully reproduces every non-native idiom. You've spent two weeks of GPU time laundering a web app into AppKit. Gruber, who timed the current app at 30 seconds to launch, was not gentle about this.
Both critiques reduce to the same principle, and it's the one that should outlive this story: an autonomous agent's output quality is bounded by the fidelity of its verifier. Cherny says verification is "the single most important thing that people do not get right" — and then demonstrates the failure mode himself. The lesson isn't "give your agent a way to check its work." It's that the checking mechanism is the spec, and a lazy one gets you a lazy product no matter how smart the model is.
What to actually do with this #
If you want to run Cherny's playbook on your own gnarly migration — and you should, it's the right playbook — fix the verifier before you scale the agents:
Make verification executable, not visual. For a Swift port, that'sxcodebuild test
in the loop, XCUITest flows for the critical paths, snapshot tests where pixels genuinely matter, and behavioral parity tests against the original app's observable outputs. Screenshots are a debugging aid, not an oracle.Verify properties, not appearance. Enumerate what the old appdoes— the state machine, not the chrome — and encode each behavior as a check. This is tedious. It's also the actual engineering work; the agent handles the rest.Budget like it's infrastructure. Two weeks of thousands of agents is real money — HN's back-of-envelope put it in the tens of thousands of dollars, and nothing Cherny said contradicts that. A long-running agent is a compute commitment you should cost like a CI fleet, not a chat session.Keep a human sign-off gate. The strongest HN comment noted that even perfect agent self-verification doesn't answer the question that matters to your users: did a human confirm this works? Agents verify to iterate; you verify to ship.
The experiment isn't done, and that's fine — Cherny was explicit that he was stress-testing a task slightly beyond the frontier, which is exactly how you find the frontier. But be precise about what it proved. Two weeks of autonomous operation: proven. Self-organizing infrastructure and progress reporting: proven, and genuinely new. Shipping a production Mac app from a three-sentence prompt: not proven, and the pixel-diff oracle suggests that even "done" won't mean what it should. The models stopped being the bottleneck. Your test suite is the bottleneck now, and no one is post-training their way around that for you.
Sources & further reading #
[Boris Cherny on Trying to Get Claude Code to Rewrite the Claude App](https://daringfireball.net/linked/2026/08/02/cherny-claude-swift)— daringfireball.net -
[Boris Cherny: Building Claude Code](https://www.ycrootaccess.com/p/boris-cherny-building-claude-code)— ycrootaccess.com -
[Boris Cherny: Building Claude Code (Startup School interview)](https://youtu.be/qyPCVqFUyDo)— youtu.be -
Boris Cherny on Trying to Get Claude Code to Rewrite the Claude App— news.ycombinator.com - Head of Anthropic's Claude Code Says Prompt Engineering Not That Important— searchenginejournal.com
Priya Nair· AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.
Discussion 0 #
No comments yet
Be the first to weigh in.