Autonomy Is Earned per Action, Not per Agent

METR's 2025 randomized trial found 16 experienced open-source developers took 19% longer on 246 real tasks when AI was allowed, with time shifting from writing into prompting, waiting and reviewing, according to the article's account of the study. Klarna said in February 2024 its AI assistant handled 2.3 million conversations in a month, the workload of 700 full-time agents, and estimated a $40 million profit improvement, but by May 2025 CEO Sebastian Siemiatkowski told Bloomberg the company was hiring human agents again after cost-dominated evaluation and falling quality. The piece argues autonomy should be granted per action class rather than per agent, separating reliability, trust, autonomy, authorization and accountability, and treating human review as temporary support retired as evidence accumulates.

Agents can now produce actions faster than people can responsibly verify them. The closest measurement is not an agent study but one of AI-assisted coding: in METR's 2025 randomized trial, 16 experienced open-source developers took 19% longer on 246 real tasks when AI was allowed.<sup 1 user-content-fn-1 </sup The slowdown is not the point; the redistribution is. Time moved from writing into prompting, waiting and reviewing, and in that one setting the time those added exceeded the time saved on writing. METR's follow-up says newer tools likely speed developers up, though its measurements can no longer isolate by how much.<sup 2 user-content-fn-2 </sup It shows that AI redistributes work toward verification, not that verification is everyone's bottleneck; whether it is yours depends on your volumes and controls. Measure it. Do not assume it. Klarna is the version everyone knows. In February 2024 it said its assistant had held 2.3 million conversations in a month, the workload of 700 full-time agents, and estimated a $40 million profit improvement.<sup 3 user-content-fn-3 </sup Company numbers, unaudited. By May 2025 its CEO told Bloomberg it was hiring human agents again because cost had dominated the evaluation and quality had fallen.<sup 4 user-content-fn-4 </sup Klarna proves nothing universal. It shows that aggregate throughput and per-case adequacy are different measurements, and one can look excellent while the other degrades unseen. Two poor answers dominate. Keep a human approving every action, and you have rebuilt the queue automation was meant to dissolve. Grant broad autonomy because the demos are impressive, and you have converted a capability claim into an authorization decision. Between them sits the missing operating model, calibrated trust made operational: human review as temporary support for a precisely defined class of agent work, retired as evidence accumulates, kept where evidence cannot reach. Five terms, kept apart five-terms-kept-apart Reliability is a measured property: how often, and how badly, a specific configuration fails at a specific action class. Trust is the calibrated belief you form from that measurement, and it is exactly as wide as the measurement. Autonomy is the review regime you apply: how much human attention per action. Authorization is the technically enforced permission set: credentials, allowlists, transaction and rate limits, data boundaries, isolation, logging, automatic suspension. Accountability is a named role that answers for outcomes and holds the authority, resources and duty to halt the system, investigate and remedy harm. Evidence changes trust and, through trust, autonomy. It never changes authorization; humans do that, separately. The unit that earns the-unit-that-earns The agent does not earn anything. The unit that earns is a combination: agent configuration × action class × operating environment × permission scope × verification system. Change one materially and the record no longer describes what is running. An agent trusted to triage internal IT tickets has earned nothing about external refunds; that is the obvious half. The less obvious half is that the same refund workflow on a new model version, or a new market's payment rails, is a new unit whose record must be rebuilt. Reliability does not transfer on its own. A ceiling, and a position below it a-ceiling-and-a-position-below-it Autonomy has two determinants that should never be computed together. The ceiling is set by the nature of the action: reversibility, risk-calibrated blast radius, regulatory status, and whether something other than the model can verify the outcome. Authorization bounds what the agent can do; the ceiling bounds how little review it may receive. The ceiling is fixed for the current system design: routine performance cannot raise it, and it moves only when the system changes a rollback path, say, or an independent verifier , through its own assessment and approval. Reversibility is defined at the level of consequence, not storage: restoring a database does not recall an email to a vendor. Authorization is enforced outside the model from the first rung on which anything executes, and it is never widened as a consequence of promotion. An executing agent is a richer prompt-injection target than a drafting one; its permission scope caps the payoff. Below the ceiling, the position is earned. The mixed-autonomy ladder the-mixed-autonomy-ladder The ladder changes the form of review so that the supervision ratio, how much agent output one person can responsibly stand behind, rises with the strength of the verification beneath it instead of with headcount. | Rung and review regime | Evidence required to promote to the next rung | |---|---| | Observe and recommend. Agent proposes; human decides and acts | Correct on independently audited samples, including ambiguous and adversarial inputs | | Draft for human action. Agent drafts; human edits and executes | Low correction rate; escalations flag what the agent did not know | | Execute with approval. Human approves each action | Severity-weighted miss rate within the owner's declared limit; separate limit for unacceptable harms; restricted autonomous trial passed | | Execute bounded, verifiable actions. Narrow limits; every exception to a human; random audit continues | Audit findings within limit at the required sample size; rollback drills pass; controls shown to fire | | Execute at expanded scope. Wider limits; exceptions to automated hold; humans review aggregates and audit samples | None. Only a ceiling change goes further | Most action classes should stop below the top. The last two rungs differ in exposure, not enforcement, and reaching the fifth is two decisions: an autonomy promotion earned by evidence, and a separate authorization expansion that a human approves on their own assessment of the larger capped loss. Evidence never earns the second. Two principles govern the upper rungs. A guardrail on every action covers more than a human who inspects a sample, but it finds only what it was built to recognize. Exception-driven attention therefore replaces routine approval; it does not replace independent discovery. If humans see only what the controls flag, errors missed by both agent and detector vanish from oversight, and the apparent success rate rises because the organization stopped looking. The audit is the check on the checker. Second, sampling finds common failures and misses rare severe ones. That blind spot is why the ceiling exists. What counts as promotion evidence what-counts-as-promotion-evidence Approval rate is the metric most often cited and least informative: even a 98% approval rate cannot distinguish a reliable agent from a tired reviewer without independent evidence. The EU AI Act names the tendency in Article 14: automation bias. 5 user-content-fn-5 Promotion evidence should be hazard-specific and tied to a limit the owner declared in advance, per action class. The signals that justify promotion are severity-weighted miss rate on audited samples, manual-intervention rate, and how often guardrails catch a problem before a human has to. Verification time per agent task belongs in the investment case, not the safety case. Limits and standing autonomy are re-reviewed on a schedule. Sample sizes follow from the limit and are larger than intuition suggests: zero detected failures in 3,000 independent, representative trials, with perfect detection, bounds the failure rate at roughly 0.1% with 95% confidence.<sup 6 user-content-fn-6 </sup That is not proof of safety, and correlated trials or imperfect detection prove less. A harm defined as unacceptable gets its own limit, and no success rate offsets it. Supervised success is not autonomous performance: shadow mode records what the agent proposed, not what would have happened, and approve-each conceals how much depended on human repair. Test the autonomous arrangement itself in a restricted trial before promoting. One action class, end to end one-action-class-end-to-end Consider a hypothetical: goodwill refunds under a fixed amount on a consumer platform. The unit is one refund-agent build on one model version, for orders below the threshold in one market, holding refund-only credentials, verified by sampled audit and daily ledger reconciliation. A completed refund is not reliably recoverable, so nothing rests on reversibility. What permits a high rung is that the potential loss is capped and explicitly accepted, and the outcome is independently verifiable. The caps are enforced outside the model: a per-transaction limit, a daily aggregate, one refund per order, payment only to the instrument the customer used. Promotion: the owner, a named head of customer operations, declared a severity-weighted miss limit and one unacceptable harm a refund to an account flagged for fraud with its own limit. After a restricted trial with full reconciliation, the class moved from approve-each to bounded, verifiable execution, plus a random audit sized to detect a miss rate at the declared limit. Demotion: the platform team upgraded the foundation model. Execution suspended automatically on the change trigger; credentials and caps did not move. The owner then decided where the class would resume. The ceiling and controls were unchanged, so the assurance case stood; what had been invalidated was the behavioral evidence on the old model. The class resumed at approve-each until that evidence was rebuilt. A prompt wording tweak the week before had been judged cosmetic and logged. Demotion and reset demotion-and-reset Autonomy is revocable or it is not earned. Performance triggers: severity events, drift, a rising manual-override rate. Change triggers: model, prompt, tools, permissions, knowledge sources, input population, downstream systems, regulation, threat environment. One rule serves: identify which claims the change invalidates, restrict the affected execution, and let the named owner choose the interim regime. A change that invalidates only behavioral evidence sends the class down until it is rebuilt. A change that invalidates the assurance case itself a new hazard class, a model swap arriving with new tools or wider permissions warrants a reset to the bottom; a replaced model alone may, at the owner's judgment. Suspending execution and resetting permissions are separate acts, and restoring execution needs the owner's decision. Two further boundaries. Correlated failure: instances of one configuration share a failure mode and can fail together, so a fleet is closer to one unit of risk than many. And the cost floor: for low-volume classes, evaluation, audit and rollback infrastructure can cost more than the review it replaces. Where human approval stays where-human-approval-stays Irreversibility, high blast radius, safety-criticality, effects on people's rights, regulation and weak verifiability each trigger an assessment of whether per-action approval must stay. None settles it alone. The rule is the one this article has been building toward: retain review where it supplies judgment, authorization or demonstrated protection. And where irreversibility, high consequence and the absence of an independent verifier coincide, approval stays, or the action is not delegated at all. Regulation shapes the rule without replacing it. For high-risk systems in scope, Article 14 of the EU AI Act requires oversight commensurate with risk, autonomy and context, including the ability to override and halt, not approval of every action.<sup 7 user-content-fn-7 </sup Singapore's agentic-AI framework and NIST's AI RMF treat bounded agent powers and defined human checkpoints as controls to be designed; both are voluntary. 8 user-content-fn-8 Rarity alone is not a reason. Low volume means observed frequencies cannot justify autonomy; it does not mean a human gate protects anyone. Some gates protect nothing because the reviewer cannot evaluate what they approve. Where neither humans nor controls can reliably assess or contain a dangerous action, the right outcome is no deployment. Human-in-the-loop is neither universally necessary nor universally sufficient. What remains unsettled what-remains-unsettled How far evidence transfers across minor model versions, how to set tolerable failure rates, and whether sampled audit detects drift fast enough remain unsettled. This framework says what to decide and who decides, not what the numbers are. The leadership implication the-leadership-implication Accountability is not approval, and the confusion between them is what turns agent governance into a queue. Before deployment, a named role owns each action class, with the authority to halt it, narrow it and refuse its promotion. Naming an owner does not settle legal responsibility. It creates the chain of command. The objective is not to eliminate review. It is to convert repeated human judgment into proportional, scalable controls guardrails, limits, audits, rollback so that human attention concentrates on exceptions, novelty and consequential decisions. The test for a leader is simple. For each agent workflow, can you name the unit, the ceiling, the owner, the declared limit and the trigger that demotes it? If not, you have not granted autonomy. You have lost track of it. Footnotes footnote-label 1. J. Becker, N. Rush, E. Barnes, D. Rein, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," METR, July 2025. https://arxiv.org/abs/2507.09089 https://arxiv.org/abs/2507.09089 . Randomized controlled trial; 16 developers, 246 tasks in their own repositories, February to June 2025; point estimate a 19% increase in completion time with AI allowed, with the added time spent prompting, waiting on and reviewing AI output, and idle. ↩ user-content-fnref-1 2. METR, "We are Changing our Developer Productivity Experiment Design," 24 February 2026. https://metr.org/blog/2026-02-24-uplift-update/ https://metr.org/blog/2026-02-24-uplift-update/ . Raw follow-up results point toward speed-up, but selection effects developers and tasks withheld from the no-AI condition make the estimate unreliable, and METR's time measurements were unreliable for developers using multiple agents concurrently. ↩ user-content-fnref-2 3. Klarna, "Klarna AI assistant handles two-thirds of customer service chats in its first month," press release, 27 February 2024. https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/ https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/ . Figures are company-reported estimates. ↩ user-content-fnref-3 4. Bloomberg, "Klarna Turns From AI to Real Person Customer Service," 8 May 2025. https://www.bloomberg.com/news/articles/2025-05-08/klarna-turns-from-ai-to-real-person-customer-service https://www.bloomberg.com/news/articles/2025-05-08/klarna-turns-from-ai-to-real-person-customer-service . CEO Sebastian Siemiatkowski's statements on hiring and on cost as an evaluation factor. ↩ user-content-fnref-4 5. Regulation EU 2024/1689 AI Act , Article 14. Paragraph 3 requires oversight measures commensurate with risk, level of autonomy and context of use; paragraph 4 b addresses automation bias; paragraph 4 e requires the ability to intervene or halt the system. Official text: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L 202401689 https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L 202401689 . The Digital Omnibus on AI, Regulation EU 2026/1744 published 24 July 2026 , defers obligations for stand-alone high-risk systems under Annex III to 2 December 2027. ↩ user-content-fnref-5 6. The "rule of three": with zero events in n independent trials, the approximate 95% upper confidence bound on the event rate is 3/n. J. A. Hanley and A. Lippman-Hand, "If Nothing Goes Wrong, Is Everything All Right? Interpreting Zero Numerators," JAMA 249, no. 13 1983 . ↩ user-content-fnref-6 7. Regulation EU 2024/1689 AI Act , Article 14. Paragraph 3 requires oversight measures commensurate with risk, level of autonomy and context of use; paragraph 4 b addresses automation bias; paragraph 4 e requires the ability to intervene or halt the system. Official text: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L 202401689 https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L 202401689 . The Digital Omnibus on AI, Regulation EU 2026/1744 published 24 July 2026 , defers obligations for stand-alone high-risk systems under Annex III to 2 December 2027. ↩ user-content-fnref-7 8. Infocomm Media Development Authority Singapore , Model AI Governance Framework for Agentic AI , version 1.0, launched 22 January 2026, which recommends bounding agents' autonomy and access to tools and data, and defining checkpoints at which human approval is required; NIST, Artificial Intelligence Risk Management Framework AI RMF 1.0 , NIST AI 100-1, January 2023, https://doi.org/10.6028/NIST.AI.100-1 https://doi.org/10.6028/NIST.AI.100-1 . ↩ user-content-fnref-8