Excitement continues around coding assisstants and we’re really now at the point where fully agentic loops are possible - pull a ticket, understand it, change the code, open the PR, respond to review comments. Not toy tickets either. Real ones, off the real board.
It works. That’s the part I want to say plainly before I say anything else, because there’s a lot of writing at the moment that gets the capability question wrong in one direction or the other. The technology isn’t the barrier.
My own experience is that the code that comes out is usually better than what a mid-level developer would write unsupervised and the designs it proposes for bigger systems are credible. Not always right, but credible - they need refining, sometimes because the spec was partial and it filled the gap with a reasonable but wrong assumption, sometimes because it assumed something about the codebase that used to be true. But credible is a much higher bar than people who stopped paying attention in 2024 think it is. As is the code quality.
Where it falls down is duller than the discourse suggests. Single files that grow until they’re four hundred lines of things that should have been three files. Dead scaffolding left behind after a refactor. A helper written that duplicates one three directories over. And taste - knowing that a thing is technically correct and still the wrong shape and that we’ll pay for it later or in a non-obvious way. Occasionally it will do “crazy” things like ignore your spec and write tests for whatever the heck it wrote. Its not perfect and I’m not claiming that, but its gotten very very good.
You can throw AGENTS.md and CLAUDE.md at those problems. It helps at the margin. It does not hold across a long unattended run, and I’ll come back to why.
Static analysis holds far better, because a failing build isn’t something a model can talk itself out of. Most of what I listed above can be turned into a rule, and I’ve turned a lot of it into rules - more on the specifics later, because it’s the highest-leverage thing I’ve done on any of this. I continue to maintain, and see plenty of evidence, that neurosymbolic systems are far more powerful than purely generative systems. But it’s all halving. Each rule takes out some fraction of what’s left, and the next one is narrower and more effort and catches less, and the number gets satisfyingly small without ever once touching zero. What’s sat at the bottom isn’t a smaller version of the same problem either. It’s the category that was never expressible as a rule to begin with: correct but the wrong shape. That call was defensible and it isn’t the one I’d have made. There’s no analyzer for taste, and I don’t think one is coming. It might not matter.
But none of this is what actually troubles me, I’ve made my peace. The thing that troubles me is at what scale can I still stand behind the output, and what exactly am I standing behind it with?
Lets start with the vendors, because their terms are clear.
Several of the big providers will now indemnify you against intellectual property claims on generated code. If someone alleges the model reproduced their work, they’ll defend it. But not one of them will indemnify correctness. Nobody is offering to cover the cost of the change that took your production system down at three in the morning or the change that had your agents spend a day going round in circles burning your money.
That asymmetry isn’t an oversight. IP exposure is boundable and you can model the claim rate, cap the payout, price it, its traditional cost of doing business stuff. Defect liability, and incorrectness more generally, is unbounded and correlated with your customer’s business, not with yours. They’ll take the risk they can put a number on and leave the other one on your desk.
So the liability sits with you and at some point the rubber hits the road. That much I think is uncontroversial, but it doesn’t tell you anything you can act on.
A useful place to start is being clear about what your customers are actually asking, because the fear underneath this is real: you end up in a room with a customer’s security team and the honest answer to “who reviewed this” is “nothing with a pulse, but we ran a lot of validation loops.” That’s an uncomfortable position and probably a losing one. That might change. Slowly, if at all.
But it isn’t the question the frameworks ask.
Go and read what they actually require. SOC 2’s CC8.1 is written as a process outcome — changes authorised, designed, documented, tested, approved, tracked, deployed — and the 2022 revision to the points of focus sharpened one thing in particular: segregation of duties in change and development, so that nobody approves or tests their own work. ISO 27001:2022 is the same shape. A.8.32 change management. A.8.25 secure development lifecycle. A.8.29 security testing in development and acceptance. A.5.3 segregation of duties. What an auditor samples is change tickets, approval evidence, test evidence, and the access configuration proving that whoever wrote it didn’t promote it.
Authorship isn’t a control object anywhere in that.
Which isn’t to say nobody asks who. They do, just not in the sense people reach for in this argument. Personnel screening is a control (CC1.4, A.6.1). Outsourced development is a control (A.8.30). Federal, ITAR-adjacent and some financial services buyers will ask where your developers sit and whether they’re employees. And the questionnaire currently in your sales team’s inbox may well have grown an AI section, because most of them have. Read what it’s asking, though: subprocessors, what model you use, data flows, whether your source code leaves your boundary and what happens to it when it does. That’s supplier risk (A.5.19–A.5.22, A.8.4, CC9.2). It’s a real question. It just isn’t a question about who typed the characters.
The one that genuinely starts to bite is attribution. Traceability means a named identity on the change and a different named identity on the approval and gents committing under a shared service account and landing changes with nobody answerable attached fails that and not because a model wrote the code, but because there’s no accountable person on it. That’s an exception in any auditor’s sample as the control doesn’t care where the diff came from. It cares that someone answerable stood behind it before it shipped. Hold onto that word, stood, it’s doing a lot of work and I’ll come back to whether it survives contact with volume.
And that has to be the shape of it, because it’s already how everything works. Your application ships tens of thousands of lines you have never read, in transitive dependencies you didn’t choose, compiled by a toolchain making optimisation decisions nobody signs off individually. We don’t read the IL generated by managed runtimes, and we don’t review the register allocation.
I want to be careful with that analogy, because it only carries one of the two things people use it for. It does establish that the assurance regime we built was never about authorship - it was about accountable approval, and we conflate the two because for forty years they happened to be the same person. It does not establish that unread agent output is the same kind of object as compiler output. The compiler earns its trust through determinism and through millions of people hitting the same code paths, with accountable publishers the whole way up. A bespoke diff has neither. Nobody else will ever run it. So the analogy tells you authorship was never the control - it doesn’t tell you the approval can be skipped.
However, separate them and the picture reorganises.
Look at what’s actually shipping. Copilot’s agent, Devin, Claude Code, Seer, Bits Code. Every one of them will plan, code, test, open a PR and iterate on review comments and red builds. None of them lands the merge.
I’d been reading that as immaturity — vendors being careful while the tech catches up. It may have started there. But look at how the gate is built and that reading falls apart. GitHub’s coding agent can’t mark its own PR ready for review, can’t approve, can’t merge, and the developer who dispatched it is explicitly barred from approving it either, so required-approvals rules and branch protection survive intact. That isn’t a vendor hedging on capability. That is segregation of duties, implemented in a product, exactly as CC8.1 describes it.
Worth being precise about where the stop actually lives, though, because it isn’t inside the agents. Claude Code is a shell with your credentials; gh pr merge is a command like any other. Devin will happily surface merge and auto-merge toggles inside its own review UI. What holds the line is the control plane around them — branch protection, required reviews, code owners, token scope. And that line is under load: as of August 2026 you currently cannot grant an agent push access on GitHub without also granting merge access, because the same permission covers both. There’s a standing request to split them. The gate holds because someone configured it to, not because the tooling is coy.
Notice too that the perimeter has already moved once. Since around March 2026 you can flip a setting so Actions workflows run on agent PRs without a human clicking approve. The workflow gate loosened but the merge gate didn’t. That’s the shape of a boundary that’s been reasoned about, not one that’s waiting on better models.
And we’ve already agreed to move it in one case. Dependabot and Renovate auto-merge is settled practice: a workflow approves the PR, enables auto-merge, and the change lands when checks pass. Machine-authored, machine-approved, CI as the only gate, no human on the button. Narrow change class, cheap and complete automated assurance — which tells you what it takes to move the line rather than that the line is soft.
Though I think there’s a sharper version of that. What really made those merges movable is that the human approval on them was already nominal. Nobody was reading those diffs before auto-merge either - the click was theatre and everyone knew it. Which suggests the question to ask isn’t “what can we safely let go of”, its “where are we already pretending”, and those are the places to either automate or start actually reading.
Which means liability isn’t a ceiling on how much agentic work you can do. It’s a constraint on where the human sits.
That’s a much better problem to have, because it converts a legal wall into an engineering one. If the human sits at merge, the cost of reviewing a change is what determines your throughput. Not model capability. Not context windows. Review economics.
And review economics you can attack.
Before we move on there’s a couple of exceptions I think are worth talking about.
Firstly Regimes built on design authority rather than approval authority: DO-178C in avionics, IEC 62304 for medical devices, ISO 26262 in automotive. These want traceability from requirement to code to test with named responsible engineers at each hop. That’s a genuinely harder fit. Not impossible - but the trace has to be generated as a first-class artefact of the process, not reconstructed afterwards by someone with a spreadsheet and a bad week. Most of us aren’t in those regimes, I’m not so forgive me if I’m a little off here. But if it holds and you are, discount everything below accordingly.
Secondly its conceivable that where the human sits shifts fundamentally, or is removed - that the accountability moves from the change to the process itself. I’ve put that in the predictions at the end rather than here, because its a direction of travel rather than an exception to what follows.
Seven things worth considering for your review process, none of these are free but neither is pure speed. Apply selectively.
Put the rules where the model can’t ignore them. This is the answer to the AGENTS.md problem. Prose instructions are competing for attention against two hundred thousand tokens of actual code, and over a long run the code wins. Every complaint I listed at the top is a complaint that should have been a failing build. File and method length: an analyzer rule with warnings as errors. Dead scaffolding: dead-code detection in CI plus a coverage delta on new files. “Don’t invent another abstraction layer”: architecture tests asserting the layer graph. If you can’t express a standard as something that goes red, it will drift, and you will keep writing it into a markdown file and keep being disappointed. So yes, static analysis basically.
Break the circularity. If the agent wrote the code and the agent wrote the tests, “the tests pass” validates nothing at all. I’ve seen extreme versions of this where an agent has decided to ignore parts of a spec specifically in place to steer the agent awawy from failrue modes. Its reasoning: “patterns in my training data”, it wrote the tests for the same adjustments. Mutation testing is the check on the check - it tells you whether those tests would actually catch a regression, or whether they’re assertions shaped like tests. I’d run it as a gate on agent PRs specifically even if you don’t run it everywhere. Tools like Stryker.NET and StrykerJS can help with this. Worth pricing it before you turn it on everywhere. Mutation runs are slow and on a high volume agent workflow that’s real CI spend and real cycle time, which is part of why I’d gate it to agent PRs and to the risk bands that warrant it rather than running it across the board.
Hide the grader. If the acceptance criteria are visible as executable checks, you get optimisation toward the checks rather than toward the problem. Pin the spec before the run, verify after, don’t hand the agent the marking scheme. Which is counterintuitive but, in my experience, can stop the agent from going too narrow in its scope. It does cut against the point above, though, and the sequencing matters. If you withhold the criteria and the agent writes the tests anyway, you’ve just recreated the circularity one paragraph after closing it - now its testing against its own guess at the spec. It also increases the odds of the failure I opened with, the partial spec filled in with a reasonable but wrong assumption. So: pin the spec, let it write the tests it wants, then verify against the pinned spec afterwards. The withheld thing is the marking scheme, not the requirement.
Cap the blast radius. Reject anything over a file or line threshold and make it decompose. What counts as a small PR will be different for you than me, but small things stay genuinely reviewable at volume. Large ones get rubber-stamped, and rubber-stamping is the real failure mode: not bad agent output, but good-looking agent output arriving faster than anyone can actually read it. This happens with both human and machine generated code.
Make it show its work as a failing test. A PR that opens with a red test that goes green is dramatically cheaper to review than one that asserts a fix. It also happens to be the honest artefact: it demonstrates the bug existed and demonstrates it doesn’t now.
Route by risk before anything runs. Not every ticket should be eligible for full automation and not every ticket should be eligible for marginal review. This is the lever that actually scales, because it changes how much scrutiny each change needs rather than trying to make scrutiny cheaper - everything else on this list is an attempt to reduce the cost of a review, and this one reduces the number of reviews that have to be expensive. Its classic risk driven development. Its also the one place I’d spend senior human attention without hesitation, because classifying a ticket up front is a judgement call on a structured question, which as I’ll get to is the thing humans stay good at. The classification is a five minute conversation. The review it saves you is not.
Consider whether the review needs to happen before the merge at all. The honest challenge to everything above is that I’ve assumed pre-merge review is the place you buy assurance. It isn’t the only place. Progressive rollout, flags, a fast and rehearsed rollback, and observability sharp enough to catch the defect in minutes are all ways of making being wrong cheap rather than making wrongness rare, and they scale in a way that human attention demonstrably doesn’t. Where a change class is genuinely reversible I think that’s the better trade and it should feed straight back into how you route by risk. Where it isn’t: migrations, anything that writes to customers, anything with a compliance surface. It buys you nothing there, because the cost of being wrong is realised before you can pull it back. That would suggest you should sort your change classes by reversibility before you sort them by anything else.
Then measure the thing that matters, which is not your confidence in any individual PR. Sample-audit what you merge and track defects reaching production per agent PR against per human PR and across the review techniques applied. That turns an unanswerable judgement call into a number you can set your autonomy dial against. With the caveat that it will take longer to say anything than you want it to. Defect escape is a rare event, attribution back to a specific change is unreliable, and at most people’s volumes you’ll be waiting a good while before the difference between two arms is distinguishable from noise. Which is its own irony: the same low base rate that defeats your reviewer defeats your measurement of your reviewer. Start collecting it now anyway, because the alternative is having the argument on vibes in two years.
All these gates are per change, and that’s their limit. The failure I’d actually bet on isn’t a bad change getting through, it’s the residue I started this post with, showing up again at a scale nothing here touches. Correct but the wrong shape was the thing no rule could catch in a single diff. Aggregate it over thousands of individually defensible changes and you get a codebase quietly losing its shape underneath you, with nobody’s job being to hold the whole thing. Same category, longer timescale, and this time you can’t even point at the change that did it. Its all been tickets based around changes.
The best I’ve got is twofold:
Everything above assumes the human at the merge gate is actually reading. I want to spend a while on that assumption, because it isn’t a detail, it’s the core of the whole argument, and I think it’s the weakest thing in this post.
I argued that authorship was never the control object and accountable approval always was. Fine. But if the approval degrades into a signature that follows the diff without anyone reading it, then the control I’ve just told you to lean on is a checkbox, the audit still passes, and the assurance is gone. Worse: it raises the possibility that the coupling I called incidental wasn’t. That approval worked for forty years partly because the approver had written enough of the surrounding code to know when something smelled wrong, and separating the two removes the thing that made the signature mean anything.
I don’t think that’s fatal but it deserves significant attention.
Ask a person to review two hundred machine generated PRs a week and you have built a treadmill. There’s no design work in it, no chase, no moment where the thing finally clicks. It’s the self-driving car problem: the system does the driving, the human is nominally supervising, and after forty minutes on a straight road the human is not supervising anything. And it can be largely self-un selecting - those who could review at that scale don’t want to.
But the obvious self driving car analogy got me thinking, we talk about AI as if it were unprecedented and in what it can do some of it genuinely is. But this problem isn’t new at all. Automating most of a task and leaving a person to watch the rest is one of the most thoroughly studied failure modes in engineering, and the field that studied it has been publishing results since before I could type and so I went and did some reading.
One excellent paper on this is Lisanne Bainbridge’s Ironies of Automation, Automatica 19(6), 1983. Five pages about process control plants. It describes what’s happening in our industry right now with an accuracy that’s genuinely uncomfortable to read (note these are not her actual ironies which are focused more on the designer):
Automation takes over the parts humans do well and leaves them the residual, monitoring, which is the thing humans do worst. She cites the vigilance literature for the claim that a person cannot hold effective attention on a source where very little happens for much more than half an hour, however motivated they are. Mackworth had established that watching radar in 1948 and the finding has never stopped replicating.
The operator’s skill decays through disuse, so they are least capable of intervening at precisely the moment the automation hands them something it couldn’t handle. Endsley later formalised this as the out of the loop performance problem: the supervisor loses situation awareness and takes over worse than if they’d been doing the job all along.
Barry Strauch then revisited all this for IEEE Transactions on Human-Machine Systems in 2018 under the title Ironies of Automation: Still Unresolved After All These Years, which tells you roughly how much progress forty years bought.
And there’s a sting specific to our case. Reliability makes it worse. If one PR in twenty is wrong, reviewers stay sharp, because they keep finding things and finding things is reinforcing. At one in five hundred, four hundred and ninety nine clean diffs have trained the reviewer to expect the five hundredth to be clean too. Improving the model degrades the review. Is there a point where this is acceptable? That its ok that the human reviewer isn’t effective when the four hundred and ninety nine clean changes are good? I don’t know.
There’s also a pressure here that has nothing to do with attention. Signing off a change you didn’t write and can’t fully verify is personal risk with no upside attached, the approver carries the blame if it breaks and gets nothing if it doesn’t. Approval regimes tend to respond to that the same way: the signature migrates to whoever has the least standing to refuse it. That’ll erode your gate on its own, without any help from the vigilance decrement, and probably faster.
And I don’t have a solution. Five things that might help, offered with the caveat that Bainbridge proposed remedies too and Strauch’s paper thirty five years later is a catalogue of them not taking. So treat these as mitigations with a poor family history rather than as answers.
Stop asking people to be detectors. “Find what’s wrong with this diff” is signal detection on rare events, which humans are measurably bad at and which is exactly the part that feels like a treadmill. “Is this the right thing to do at all? Does the failing test actually test the bug that was reported? Does this belong in this part of the system?” is judgement on a structured question. Different task, much harder to do on autopilot, and considerably less soul-destroying.
Cap the volume, and never make it a role. The decrement is driven by time on task. The moment somebody’s job title is effectively reviewer, you’ve built the treadmill by construction and no amount of tooling will unbuild it.
Reviewers have to still be building. In the same subsystem, on the same week. This is Bainbridge’s second irony directly - review capability is downstream of hands-on familiarity and it decays without it. Which incidentally caps how far you can push the ratio, because the reviewers have to be doing enough real work to stay worth having. There are other benefits to this too: it keeps people sharp for when the tools don’t work, or can’t help.
Salting the flow. Aviation and airport screening inject known defects at a controlled rate to keep detection live and, more usefully, measurable. It works, but it’s also adversarial toward your own engineers, and done clumsily it will cost you more in trust than it buys you in vigilance. I’d think very hard before doing it, but I didn’t feel able to leave it off this list.
Change the crew, not the operator. Aviation never did fix vigilance. What it did instead was put two people in the cockpit with defined cross-check callouts, an explicit protocol for challenging the other one, and CRM training to make the challenge socially survivable. Every suggestion above this is an attempt to make one person’s attention hold up. This one gives up on that and changes the structure around them, a two person rule on the top risk band only, with named challenge questions rather than open ended “review this”. Its expensive though, which is why it belongs at the top band and nowhere else.