Approve every agent action and your reviewers drown. Approve none and your customers find the mistakes for you. A three-factor score shows where the line belongs.
- Score each agent action from 1 to 4 on reversibility, dollar exposure and measured error rate, then multiply the three numbers.
- Scores of 6 or below run automatically, 7 to 17 run with a 10% sample reviewed afterward, and 18 or above need approval before the action happens.
- In the worked example, a support agent doing 1,000 actions a day drops from 600 pre-approvals to 78 reviews, about 15 hours of reviewer time down to roughly 2.
- Irreversible actions and bursts of repeated small actions need their own override rules, because a per-action score misses both.
- Error rates must come from logged outcomes, and any action the agent has never done before gets the worst error score until it earns a better one.
Every team that ships an agent ends up choosing an ai agent human in the loop approval threshold, and most choose it by accident. They start by approving every step, watch the review queue grow past what anyone can read, and then flip a switch to approve nothing. Both settings are wrong, and the second one usually gets found out on a bad Tuesday.
The fix is a number, not a mood. Score each action your agent can take on three things you can write down: whether it can be undone, how much money it can move, and how often the agent gets it wrong. Multiply them, set two cut-offs, and the review queue sorts itself. The worked example further down shows how small the queue gets.
Look at how Claude Code ships. By default it reads files without asking, but it stops to ask before it edits a file or runs a shell command. That's a tiering by action type, and it's a sensible start. It's also why the tool lets you allowlist specific commands: if every prompt gets the same weight, people click yes without reading, and the prompt that mattered gets the same reflex as the thousand that didn't. A reviewer who approves 99 of 100 items isn't reviewing. They're a slow, expensive rubber stamp.
The other direction has a body count. In July 2025, according to reports from Fortune and The Register, Replit's coding agent deleted a live production database belonging to SaaStr founder Jason Lemkin during a declared code freeze. Replit's CEO apologised publicly and the company moved to separate development and production environments. A deletion of production data is the textbook case of an action that should never have run without a person looking at it, and nothing in the setup forced that look.
Why AI Agent Approval Workflow Design Fails When Founders Rush It AI agent approval workflow design is the specific system of checkpoints that decides which actions an autonomous agent can take alone and which need a human to confirm first. This piece breaks down how those permission tiers actually work in tools like Claude Code, Devin, and Operator, and why startups that strip the gates out early get burned... - why AI agents delete production databases in startups - how founders rush approval workflows and lose data
So the question isn't whether to have a human in the loop. It's how to decide which agent actions need human review, and to decide it before the incident rather than after.
Score every action on three numbers #
Give each action type in your agent's toolbox a score from 1 to 4 on each factor. Do it per action type, not per agent. 'Send a refund' and 'look up an order' are different actions even when the same agent does both.
Reversibility comes first because it sets the cost of being wrong. A 1 means you can undo it with one click and nobody notices, like reopening a ticket. A 2 means you can undo it, but it takes someone's time, like clawing back a card refund. A 3 means the damage is partly permanent, like an email that's already in a customer's inbox. A 4 means it can't be undone at all: a deleted account, a wire sent, a production table dropped.
Dollar exposure is the most you could lose on one run of the action, not the average. Use 1 for under $50, 2 for $50 to $500, 3 for $500 to $5,000 and 4 above that. Pick bands that fit your business, but write them down. If the action has no direct dollar cost, price the worst case anyway, such as a regulatory fine or a lost contract, and be honest about it.
Error rate is where most teams cheat, so the rule is strict: it comes from logged outcomes, not from how the agent feels in a demo. Under 1% measured errors scores a 1, 1% to 3% scores a 2, 3% to 10% scores a 3, and anything above 10% scores a 4. An action the agent has never performed in production has no measurement, so it scores a 4 by default. That single rule keeps new capabilities from slipping into the automatic lane.
Multiply the three. The product runs from 1 to 64. Then set two cut-offs. Anything at 6 or below runs on its own. Anything from 7 to 17 runs on its own too, but a person reviews a random 10% sample afterward. Anything at 18 or above waits for approval before it happens. Multiplication matters here: a 4 on any one factor drags the whole score up fast, while a row of 1s and 2s stays cheap.
A worked example: 1,000 actions a day #
This one is illustrative, with invented volumes, but the arithmetic is real and you can rerun it with your own logs. Say a customer support agent performs 1,000 actions a day across six action types.
How to Build an Eval Set for AI Agents Before You Trust One in Production Air Canada was ordered to pay about C$812 in February 2024 after its chatbot invented a refund policy. This guide covers where to find test cases, how to score them with code before an LLM judge, and why each case should run three to five times. - build an eval set for AI agents - AI agent test cases before production
Lookups of orders and account details run 400 times a day. They score 1 on reversibility, 1 on exposure and 1 on error rate, so the product is 1. Automatic.
Standard reply emails run 300 times a day. Once sent, an email can't be recalled, so it scores a 3 on reversibility, 1 on exposure, and say a measured 2 on errors. The product is 6, which sits right on the automatic line.
Refunds under $50 run 150 times a day, scoring 2 for effort to reverse, 1 for exposure and 2 for errors. That's 4. Automatic, and the worst case on any one is under $50.
Refunds between $50 and $500 run 80 times a day at 2 by 2 by 2, which is 8. These get the sampled lane: the agent acts, and a reviewer checks 8 of the 80.
Refunds over $500 run 20 times a day at 2 by 3 by 3, which is 18. These wait for a person every time.
Account closures run 50 times a day. They score 4 for reversibility, 2 for exposure and 2 for errors, a product of 16, which would land in the sampled lane. That's a trap, and the next section explains why a hard override sends them to pre-approval anyway.
Now count the reviews. The old policy of approving every action that changes something means 600 pre-approvals a day, since the 400 lookups change nothing. The scored policy means 20 large refunds plus 50 closures needing approval, plus 8 sampled refunds, for 78 reviews. That's an 87% cut against the 600, and 92% against approving literally everything. At a minute and a half per review, the reviewer workload falls from 15 hours a day to just under 2.
The risk side matters just as much. In the automatic lane, nothing can cost more than $50 in a single action, so even if the agent botches three small refunds in a day, you're out about $150. You haven't removed the humans. You've pointed them at the 7.8% of volume that can hurt you.
The two things a score can't see #
The closures show the first gap. A score of 16 looks moderate because the dollar exposure is small, but an account that's gone can't be restored. So add a hard override: any action scoring 4 on reversibility needs approval, whatever the product says. Deleting a production database belongs here, and so does anything that touches backups.
The second gap is volume. A per-action score judges each run in isolation, and agents repeat themselves very fast. Knight Capital lost about $440 million in roughly 45 minutes on August 1, 2012, after faulty deployed code sent a flood of unintended orders. Each order alone looked ordinary. The disaster was the count. Put a velocity cap beside the score: if the agent does more than a set number of the same action within an hour, or total exposure in the automatic lane passes a daily ceiling, everything after that point drops into the approval queue. For the refund agent above, a cap of $1,000 in automatic refunds per hour would do it.
A third check is cheaper still. Give approvers a way to send back the whole class of action, not just the single item. If a reviewer rejects three sampled refunds in a row, that action type should lose its automatic status until someone looks at the cause.
Feeding the score: measure, promote, demote #
The error factor only works if you collect it. Log every action the agent takes with its inputs, the result and, where you can get it, a later verdict: did the customer reopen the ticket, did finance reverse the refund, did the reviewer overrule it. Run the agent in shadow mode for a few weeks first, proposing actions that a person executes, so you start with real error rates instead of guesses.
Then let actions move. A reasonable promotion rule is to drop an action one band after a long enough clean run, say 200 reviewed actions with no errors, and to push it up a band after a single costly miss. Treat those numbers as a starting guess for your own volumes. Demotion should be fast and promotion slow, because one bad week costs more than a month of extra review.
The approval step needs the same care as the score. Show the reviewer what the agent saw, what it plans to do and what it would cost, on one screen, so a decision takes seconds. Batch similar items. And decide in advance what happens when nobody answers: the action should wait or fail, never go ahead after a timeout. An approval that approves itself is not a control. Frankly, this is where many workflows quietly break, because a timeout that fires at 3am turns the whole gate into decoration.
Watch your approval rate too. If reviewers accept 99% or more of an action type, either the agent has earned a lower tier or the reviewers have stopped looking, and the sampled lane will tell you which. Send a few deliberately wrong items through the queue now and then. If nobody catches them, the review isn't working.
Start with one week of your agent's logs and a spreadsheet with six rows. Score each action, pick your two cut-offs, and count what the queue would have looked like. If the new count isn't at least half the old one, your bands are too tight or your error data is too thin to trust.
Also read: How to Calculate Token Cost Per Customer for AI Startup Pricing • How to Evaluate an AI Agent Pilot Before the Vendor Trial Even Starts • How does prompt caching reduce LLM costs? Most agent teams get it wrong
This article is posted in AI News, check it out for more related stories.
Join the discussion #
Open in the community → Almost there. Sign in and your reply posts straight away.