Let It Break, Part 3
Last night an agent session I was running submitted three pull requests to pnp/sp-dev-fx-webparts — an MCP client web part, an Azure AI Agent chat, and an M365 Copilot Agent chat. All three passed automated validation. I didn't change a line of code.
On paper: a great session. Then I exported the transcript and counted my own messages.
710 messages total. Thirty were mine. Twelve of those thirty were me telling the agent it was doing the wrong thing.
That's a 40% steering rate. For a session that shipped everything.
The pipeline was simple. Claude writes the implementation plan. My orchestrator — DeepSeek V4-Flash under a general-purpose harness — reviews it. Codex implements. The orchestrator reviews the result and opens the PR.
The orchestrator's only job was glue. Make things click. Resolve issues when something breaks. Never write the code itself.
Starting conditions: no code. One planning doc from a previous session. That was it.
Once the pipeline held, the build was absurd. Fork to three submitted PRs in 40 minutes.
The Azure AI Agent took two minutes — @azure/ai-projects runs browser-native, so Codex just had to wire up the SDK and a Fluent UI chat. The Copilot Agent took six. The MCP client took eleven, the longest of the three, because it needed a local Node.js bridge relaying WebSocket to stdio alongside the web part. About 3,500 lines of shipped code across the three.
The validation bot flagged warnings — a missing .nvmrc, missing screenshots, a README badge that has to be an <img> tag instead of Markdown. Diagnosing and fixing all of it took 24 minutes. Total active time on deliverables: about 75 minutes. The session's wall clock says 21.6 hours, but that's because it opened the previous evening on unrelated CI work and sat idle overnight.
Seventy-five minutes for three PR-ready SPFx 1.23.2 samples. So why did I need to correct the agent twelve times?
I classified all thirty of my messages. Eight were instructions — the initial asks. Five were confirmations: yes, proceed, next. Five were information — URLs, references, the SDK docs it should use.
And twelve were corrections. Some of them, verbatim:
"That's not what I asked. Learn how to drive Claude"
"Stop implementing"
"Claude plans, you review, codex implements, then you review"
"Claude plans"
That last one — two words — was the fifth time I explained the same pipeline. It finally stuck.
The corrections weren't spread evenly. They clustered in exactly two places: setting up the pipeline, and fixing the warnings. Both are moments where the agent tried to do everything itself instead of delegating. The third sample, built after the pipeline finally landed, needed zero corrections. The capability was there the whole time. The process wasn't.
Five of the twelve were workflow violations — the agent doing the work itself instead of orchestrating it. I half expected those. Asking a generalist agent to not implement fights its default behaviour hard.
The three that bother me were wrong tech choices. It scaffolded with Yeoman on SPFx 1.22 when I'd told it to use the new CLI on 1.23.2. It picked a community MCP library when I'd pointed it at the official SDK. The information was in its context. I had provided it. It didn't retrieve it at decision time.
These aren't reasoning failures. They're context retrieval failures — and that's a different problem with a different fix. A smarter model doesn't help if the right constraint doesn't surface at the moment the decision is made. What helps is making constraints impossible to miss: pinned instructions, skills, a checklist that runs before scaffolding instead of after.
Three days earlier I ran a similar session. 764 messages, 36 of them mine, roughly half of those corrections — and the PRs were getting denied by maintainers, because the agent kept pushing unreviewed AI-generated code.
This session dropped from roughly 50% steering to 40%, and output quality went from "denied" to "passed validation untouched." One change made the difference: I stopped assuming the agent would infer its role from the workflow, and stated it outright. You are an orchestrator. You make things click. You resolve issues. You do not implement.
It still took five corrections to make it stick. But it stuck.
The PRs are #6357, #6358 and #6359. The skill is at github.com/vystartasv/skills.
Next session I'm counting again. The number I'm chasing isn't more PRs per hour. It's fewer messages from me.