The top story: when agents build everything, judgment is the queue #
Spawn, Claude Code and Factory each ran out of humans to review the work.
Spawn
Jacob woke up one morning to 700 questions from his agents. Do we build this feature? Add a setting? Remove this thing? He’d already answered most of them before, some more than once.
Spawn is ten people. It spent $3.1 million on tokens last month and shipped 77,000 substantial changes. Code stopped being the bottleneck. Jacob became it.
His fix: two years of recorded calls, turned into principles for each person’s taste, then AI judges that apply them. In the first eight days the judges returned more than 11,000 verdicts. 7,296 decided. 2,213 sent back because the question itself was wrong. 1,863 held for a human. Jacob gave his own answer 27 times. These are his numbers, and he says the setup only works on one model, Anthropic’s Fable.
Claude Code
The Claude Code team’s write-up on how they do code review. Claude writes about 90% of the code for Claude Code. Between June and September, merged pull requests more than doubled, and the share that got reverted fell by more than 40%. Anthropic’s monorepo went from about 2,500 changes a day in May to over 30,000 in September.
Every change passes through layers: verification before the PR, an AI review on every PR, a security review that blocks, CI, human approval scaled by risk, then a merge queue, feature flags and staged rollouts. Three details:
- Agent approvals don’t count as human approvals. If an agent writes the PR, the person running the agent counts as one of the required reviewers.
- An agent author and an agent reviewer can loop forever. One PR cycled more than fifty times overnight. When they ran git blame on what the reviewer kept flagging, about 59% was code that had been there since the first pass. Now the reviewer asks the author to rethink the approach after three cycles.
- Incidents are the eval. Every PR that caused an incident goes into a collection, and the reviewer gets scored against all of it daily.
Software Factory
The software factory fight. Lauren called software factories marketing: “all you need is good software engineering and investing your tokens in the right places.” Eno Reyes at Factory, where I work, replied that a factory standardizes the process, not the output. Every piece of work starts as a signal. Every change goes through the same checks no matter who wrote it, including “a risk judge that can’t see who wrote it.” When production breaks, they trace it back to the PR.
His bar for automerge comes from self-driving cars: “Once AI-written changes break things less often than human ones, and you can actually measure that, you let them merge.” The people who used to stand on the line end up writing its rules. He calls them “the regulators.”
My take, from Day 23: design never had a factory because it had no diff. Now I design in code through the Figma MCP, so a design change is a commit, with an author, a timestamp, a diff and a rollback. What’s still missing is the baseline. Spawn reverted 1,055 of 123,548 changes over the past year, about 0.9%. Most design teams can’t say how many shipped decisions got redesigned within six months.
What I learned: Spawn, Claude Code and Factory landed on the same rule. A human doesn’t read every change. A human writes down what every change gets checked against. Spawn wrote principles. Claude Code wrote REVIEW.md. Eno calls the people who write those rules regulators. Most designers have never written their taste down.
Models and releases #
Inkling 2 Small (Thinking Machines). Scores 31 on the AA Intelligence Index, a big jump over its predecessor. Available for fine-tuning and inference on Tinker today, with “full weight release planned soon.”
ChatGPT Images 2.6 (OpenAI). Animated GIFs from a prompt or an image, with frame-by-frame edits. Saved reference images of yourself, pets or characters you can reuse without up again. Two new API models: Flare makes images in as little as six seconds, Sunburst brings higher-quality generation and more consistency across edits. It also adds image markup and SVG export.
Gemini Omni 1.2 Flash (Google). Video generation and editing up to 15 seconds, fewer jump cuts, closer to your reference assets. API price down 30% from the last generation, to $0.07 per second at 720p. 15-second clips at no extra cost in Google Vids.
DeepSeek V4.1 Flash, now in Cursor. Every effort level sits on the CursorBench cost frontier, $0.60 to $0.98 per task.
Arete-27B (Nous Research). Qwen3.8-27B, post-trained on roughly 53 million tokens. On Nous’s 803-prompt eval, the rate at which it adopts China’s party line fell from 22% to 1%, with no significant drop on Hermes Bench. Their line: “The values in an open model are one fine-tune away.”
What I learned: For designers, the line in the Images launch is SVG export. You can edit a vector, version it and hand it to code. A raster you just regenerate. On Arete: if 53 million tokens can take one set of values out of a model, the same budget can put another in.
Harnesses #
Droid 1.0 (Factory). I covered this on Day 18, so the short version. Same model, Claude Opus 5.5, same Terminal-Bench 4 tasks: Droid passed 65.2%, Claude Code 57.6%, and Droid cost up to 24% less than the other harnesses we tested. The biggest change is Script mode. The model writes a small program that makes the tool calls, instead of calling tools one at a time and reading every result. Moving 151 Linear issues took an estimated 36 requests and about 3.1 million input tokens the old way. As one script: 2 requests, about 61,000 tokens. These are our benchmark runs; the methodology is in the post.
Garry Tan deleted a skill. He removed /investigate because Opus 5.5 does it better without it. His point: run eval loops on your agent skills, because every new model makes some of them unnecessary.
What I learned: Three of this week’s harness wins were deletions. Droid went from sending 11 tool definitions up front to 2. Garry cut a whole skill. Claude Code’s reviewer asks the author to rethink the approach after three cycles. Most of us add skills and rules and never test whether the new model still needs them.
Should you still check the AI’s work? #
Two posts this week disagree.
Addy Osmani, “Critical thinking when agents do the work.” Everything now comes out formatted and confident, so “polish is no longer a useful signal.” He maps a slide from off (the agent does the task, you still own the result) to deferral, to what Steven Shaw and Gideon Nave at Wharton call cognitive surrender, where you feel no need to check at all. It gets harder as models improve. Fewer obvious errors means fewer reasons to look twice.
Ethan Mollick, five days later. He thinks there is “way too much concern expressed about verification of AI output in everyday work.” His case: data on recent models suggests they beat human experts, including PhD specialists, at finding facts and checking claims.
They’re arguing about different things. Mollick means factual errors. Addy means whether you formed your own view before you accepted the agent’s.
What I learned: My split, which is close to Addy’s own: hand the model the fact-checking. Keep the decisions about what to build and whether it’s good enough to ship.
Science #
Anthropic Science Blog, guest post by Jakob Macke, Daniel Gedon and Stefan Wahl. Scientists still usually write their models by hand, then weigh several competing versions that each explain the data a different way. AI can now write, run, fit and revise those models as code, try many structures, and find out which explanations the data support.
What I learned: Last week it was math proofs. Generating candidate explanations got cheap. Deciding which one to believe is still the scientist’s job.
Work and the job market #
Alex Imas, “What will be scarce?” (my Day 16). Peter Thiel says the scarce skills are relational. Imas explains why people will pay for them. In one study, a human-made print was worth 44% more as a one-of-one than as one of many. An AI print got only 21% more. Buyers didn’t believe the scarcity. He also calls prompt engineering and monitoring AI systems transitional jobs.
Proton’s engineering manifesto, via 404 Media (my Day 19). “Programming is dead.” The useful part is the diagnosis: Proton is stuck in phase 1, using LLMs to speed up tasks inside the old workflow. Remove one bottleneck and another takes its place. The memo’s list of candidates: CI speed, design, approvals, specs, code review.
Miles Deutscher, “We’ve Entered the Singularity.” A five-year forecast that steelmans its own bear case, including Anthropic’s June essay: “We are not there yet, and recursive self-improvement is not inevitable.” The numbers I’d keep: US unemployment around 4.3%, but entry-level professional jobs down about 29% since January 2024, and unemployment for young college grads at 5.3%. His prediction: “Companies won’t fire their way into the AI era. They’ll just stop hiring juniors.”
What I learned: Imas calls prompting transitional. Proton puts design on the list of next bottlenecks. Put together: the job that lasts is being the person the tool waits on, like Jacob with his 700 questions. Junior roles are where people used to learn that judgment, and Miles’s numbers say those are the roles going first.
What I wrote this week #
I post one insight a day on X in my series, Day N/100 of becoming an AI-native designer. Here are this week’s.
Day 16: Why relational work gets paid. On Alex Imas’s “What will be scarce?” Thiel gave a list of skills. Imas gave the reason anyone pays for them: we want what other people want, and more when not everyone can have it. Starbucks automated coffee, then reversed course with handwritten notes on cups, ceramic mugs and more baristas per store. Farming went from about 40% of American workers in 1900 to under 2%, and nobody eats less.
Day 17: Taste is the last queue. On Spawn. When agents build around the clock, everything waits on judgment calls. Spawn’s judges only work because two years of decisions were on the record. Recorded calls are training data for your judgment, and a principles file is a design system for decisions instead of components. Start here: write down the last 10 times you said no to a design, and why. That’s the first draft of your judge.
Day 18: The model is a user who re-reads the whole screen every turn. On Droid 1.0. Every instruction, tool and old result gets re-sent each turn, so clutter is a tax you pay on every turn after it. A five-number summary out of 5,384 CI jobs is how a good dashboard works. 2 tools instead of 11 is progressive disclosure. CI now fails any change that pushes first-keystroke p95 above 250 ms, which is a design requirement written as a test. The context window is a screen, not a storage bin.
Day 19: Design might be the next bottleneck. On Proton’s manifesto. Once code stops being the bottleneck, the memo lists design among the candidates for the next one.
Day 21: Polish stopped being a signal. On Addy Osmani’s essay. Nobody chooses to slide from off into surrender. You read one answer carefully, skim the next, scan the skeleton of the one after. His three fixes: write down what you think the answer is before you ask. When a check goes green, ask what it proved, then what it didn’t. Ask the agent to list the decisions and assumptions it made.
What I learned writing them: Four of these are about the same job from different angles. Written-down taste, a clean context window, design as the next bottleneck, your own answer before the agent’s. In each one, the agent does the work and the human sets the check.
3 things I’m taking into next week #
- Write the taste down. Spawn’s judges, Claude Code’s REVIEW.md, Eno’s regulators. None of them works on judgment that lives only in someone’s head.
- Measure the baseline before handing over autonomy. Spawn knows its revert rate is about 0.9%. Claude Code uses reverts and incidents as the guardrail. Eno’s bar is that AI changes merge on their own once they break things less often than human ones, measured.
- Delete before you add. Droid’s 2 tools instead of 11. Garry’s retired skill. A review loop that asks for a rethink after three cycles. Every new model is a reason to re-test what you built for the last one.
This is my reading list. Not my research. If one of these changed how you work this week, reply and tell me.
📌 Subscribe to The Artifacts: