{"slug": "the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything", "title": "The Rubber Stamp Problem: When a Human in the Loop Is Not Overseeing Anything", "summary": "A new analysis examines the \"rubber stamp\" failure mode of human-in-the-loop oversight, arguing that a logged approval click does not constitute meaningful review when reviewers face hundreds of decisions a day, see only summaries, and are measured on throughput. The piece cites the EU AI Act's Article 14, which explicitly names automation bias and requires high-risk systems to be designed so people can effectively oversee them, with Annex III obligations applying from 2 December 2027 after the Digital Omnibus deferral. It points to Ben Green's 2022 survey of 41 government human-oversight policies, which found most research suggests people cannot reliably perform the oversight those policies expect, and to a meta-analysis of 106 experiments by Vaccaro, Almaatouq and Malone finding human-AI combinations underperformed the better of the two alone, especially in decision tasks.", "body_md": "An insurer runs an AI agent that prepares claim decisions. Every proposal goes to a human reviewer before any money moves, which is exactly what the company's policy says and exactly what its auditors like to hear. The reviewer sees about four hundred proposals a day. Each one arrives as a short summary and a green button. By afternoon, the average review takes a few seconds.\n\nOne day a payout goes out that should never have been approved. The board asks the obvious question: was a human in the loop? The answer is yes. The click exists, it has a timestamp, and it has a name next to it.\n\nThe next question is harder, and it is the subject of this article: did that human oversee anything? We start with the problem, because it is larger than most teams assume, and only then look at a way of handling it. This is not legal advice, so check specifics with counsel.\n\nWhat oversight is supposed to do\n\nHuman oversight is the main safety net that regulation reaches for. The EU AI Act's Article 14 requires high-risk AI systems to be designed so that people can effectively oversee them. Among the capabilities it lists, overseers must be enabled to remain aware of the possible tendency to rely automatically, or over-rely, on the system's output. The text names this tendency automation bias. It is unusual for a law to name a psychological effect, and the reason is that its drafters knew where oversight tends to fail.\n\nCommentators draw a plain conclusion from this: a procedure that says a human will check the output is not enough, because the system and its surroundings must make a real check possible. A review that is nominal, in the sense that the box is ticked but nobody could have caught anything, does not meet the purpose. For Annex III high-risk systems, these obligations apply from 2 December 2027 after the Digital Omnibus deferral, but the problem exists today in every agent that asks a person to approve its work.\n\nWhy people rubber-stamp\n\nNone of what follows is about lazy people. It is about what attention does under certain conditions.\n\nVolume. A reviewer who must decide hundreds of times a day cannot give each decision a fresh mind. Attention is a budget, and it runs out.\n\nHabituation. If the agent is right nearly every time, approving becomes a reflex. High accuracy breeds trust, and trust lowers vigilance. Researchers have described this as an accuracy and trust paradox, where the better the system, the less likely people are to notice its rare failures.\n\nMissing context. A summary and a button give the reviewer a conclusion, not the reasoning behind it. Without the why, the only available check is whether the answer looks plausible, and plausible is what language models are good at.\n\nWrong incentives. If reviewers are measured on throughput, every override costs them time, and the system rewards the click.\n\nPassive monitoring is hard. Watching a reliable system and waiting for it to fail is one of the things people do worst. Safety researchers have documented this for decades, under names such as automation complacency.\n\nWhat the evidence shows\n\nThe evidence is broader than AI, and it is not flattering to the human in the loop.\n\nPolicy review. Ben Green surveyed 41 government policies that require human oversight of algorithms and concluded, in a 2022 paper, that the vast majority of research suggests people cannot reliably perform the oversight functions the policies expect. He added a second flaw: such policies can legitimize flawed systems by creating a false sense of security, and reduce the accountability of the vendors who built them.\n\nMeta-analysis. A meta-analysis of 106 experiments by Vaccaro, Almaatouq, and Malone, as summarized in a behavioral science reference, found that human and AI combinations performed worse on average than the better of the two alone, with the losses concentrated in decision tasks. This measures combined performance and not oversight as such, so read it as a related warning.\n\nA fatal case. In the 2018 Uber self-driving crash in Arizona, the NTSB attributed the cause in part to Uber's inadequate safety risk assessment, ineffective oversight of vehicle operators, and lack of mechanisms for addressing operators' automation complacency. The investigators described passive monitoring of a highly reliable system as producing lower vigilance and slower response. The human was there. The oversight was not.\n\nApproval data from coding agents. Anthropic reported that users approve 93 percent of Claude Code permission prompts, a figure it described as approval fatigue, and later coverage cites 97 percent as of August 2026. Reported details include a study of 1,053 actions in which human reviewers caught 13.6 percent of harmful actions while an automated classifier caught 89 percent, and a finding that people blocked about 17 percent of dangerous commands early in a session but about 5 percent after more than 50 prompts. One more detail is useful: users reportedly rejected 39 percent of plans presented for approval, far more than the 3 percent of individual prompts. People can engage when asked the right question at the right level. These figures come from a vendor's own telemetry and tests, reported through trade press, in a coding setting, so treat them as an illustration of the pattern and not as a measurement of your system.\n\nTaken together, the honest claim is this: where humans are asked to approve a high-volume stream of mostly correct decisions, one at a time, with little context, the evidence says they will often approve nearly everything, and miss a large share of the errors. It does not say humans are useless. It says the design of the oversight decides whether it works.\n\nWhat it costs\n\nA false sense of security. The organization believes it has a control. The control is not working, and nobody finds out until after an incident.\n\nBlame lands on the person. When an incident occurs, the reviewer who clicked is the visible human. Researchers have warned that this moves accountability from the system's designers onto an individual who was set up to fail.\n\nThe evidence is empty. A regulator or auditor asking for proof of oversight will be shown clicks. A click records that a button was pressed, not that anything was reviewed. Practitioners note that a human approving everything in seconds is the pattern regulators are most likely to treat as performative.\n\nReal money is wasted. Paying people to approve what they do not evaluate adds cost and friction with none of the protection.\n\nThe incident is harder to investigate. When the question becomes what the human saw and knew, there is usually no record.\n\nWhy the usual tooling does not catch it\n\nApproval logs store the decision, the user, and the time. They rarely store what was shown, how long the person looked, or what the agent's reasoning was.\n\nThroughput dashboards reward speed and show override rates only as a nuisance.\n\nTraining and policy documents describe what should happen, not what does.\n\nAudits typically check that a review step exists, and not whether it works.\n\nAlmost nobody measures oversight itself. A commentator observed that a system where the reviewer approves 99.8 percent of decisions is showing automation bias, and a team that never measures the approval rate cannot see it. That figure is an illustration and not a legal threshold, but the point stands.\n\nWhat good looks like\n\nEffective oversight is designed and measured, not declared. Practices that recur across guidance and research:\n\nTier by risk. Reserve human review for consequential, irreversible, or ambiguous actions, and automate or sample the rest. Asking a person to approve everything guarantees they will approve everything.\n\nAsk the right question. Review plans and outcomes, not a stream of tiny actions. The plan versus prompt rejection gap above suggests why.\n\nShow the why. Give the reviewer the inputs, context, and reasoning behind the proposal, not just the result.\n\nAdd friction where it counts. For consequential decisions, require a reason, a second reviewer, or a confidence check.\n\nMeasure the oversight. Track approval rate, time per review, and override rate by reviewer and action type, and investigate both extremes. Zero overrides suggest automation bias, and frequent overrides in one category suggest the system is failing there.\n\nTest the reviewers. Seed known-bad proposals and see whether they are caught.\n\nProtect independence. Do not punish overrides with throughput targets.\n\nKeep a record of what the human actually saw and did. Otherwise you cannot show oversight happened.\n\nHow ZizkaDB helps\n\nZizkaDB is an open source causal database for AI agents. It records activity as linked events and makes the record tamper-evident. It cannot make a person pay attention, and it does not build your review screen. What it does is record the oversight step as data, so you can measure it, test it, and prove it.\n\nRecord oversight as linked events. Log the agent's proposal, what was presented to the reviewer, and the human decision, each linked to the one before:\n\npython\n\nproposal = await db.log(\n\n    agent=\"claims-agent\",\n\n    event=\"proposal\",\n\n    data={\"claim_id\": \"C-7731\", \"action\": \"approve_payout\", \"amount\": 4200},\n\n)\n\nshown = await db.log(\n\n    agent=\"claims-agent\",\n\n    event=\"review_presented\",\n\n    data={\"claim_id\": \"C-7731\", \"context_ref\": proposal.id},\n\n    parent_id=proposal.id,\n\n)\n\ndecision = await db.log(\n\n    agent=\"claims-agent\",\n\n    event=\"human_decision\",\n\n    data={\"claim_id\": \"C-7731\", \"reviewer\": \"r-204\", \"outcome\": \"approved\", \"reason\": \"\"},\n\n    parent_id=shown.id,\n\n)\n\nThe event names and fields above are illustrative, so check the SDK reference for exact signatures.\n\nMeasure the oversight. Because timestamps and outcomes are events, you can compute approval rate, time between presentation and decision, and override rate by reviewer and action type, by iterating over events with the SDK. Watch those numbers over time and compare against a stored baseline, so a creeping approval rate shows up as a measured change.\n\nGive the reviewer the why. The lineage call returns the chain of inputs, retrieved context, and tool results behind a proposal, which you can surface on the review screen so the human checks reasoning and not just plausibility.\n\nTest whether reviewers catch errors. Log seeded proposals with a canary flag, and compare the catch rate against the decisions recorded. That gives you an effectiveness measure, not a claim.\n\nProve it afterwards. Each event carries a SHA-256 checksum and a chain hash linked to the previous one, so edits or deletions are detectable, and you can show an auditor exactly what was presented and decided. Time travel can reconstruct what the system and its configuration looked like at the time of a disputed decision.\n\nPractice by practice. Here is how the eight practices above map onto what the record can support:\n\nGood practice   What supports it\n\nTier by risk    An action class field on each proposal, so you can compare review rates by tier\n\nAsk the right question  Plan-level and action-level events kept separate, so their approval rates can be compared\n\nShow the why    The lineage chain surfaced on the review screen\n\nAdd friction    A logged reason or second-reviewer event on consequential decisions\n\nMeasure the oversight   Approval rate, time to decision, and override rate computed from events\n\nTest the reviewers  Canary proposals logged and matched to decisions\n\nProtect independence    Reviewer data used to improve the design, with redaction and retention set\n\nRecord what the human saw   A presented event linked to each decision, verified by the chain\n\nThree habits make this work. Log what was presented, not just what was decided. Log the time the review screen opened and the time the decision came back. And keep canary cases clearly flagged, so you can separate them from real traffic in analysis and never in the reviewer's view.\n\nBack to the insurer. With oversight recorded, the board's question changes from was a human in the loop to what did the record show. The record could show that the reviewer had seconds per case, that the approval rate had crept up for weeks, and that the faulty proposal's reasoning was available but never displayed. That is a finding about the design, and it points to a fix, instead of ending at one person's click.\n\nHandle reviewer data carefully. Records of reviewer behavior are personal data about employees. Decide what to keep, use redaction and retention settings, involve your works council or HR where required, and measure the process, not the person. The aim is to find where the design fails, not to rank people.\n\nThe honest scope: ZizkaDB does not make oversight effective on its own, and it cannot tell whether a decision was right. It makes oversight visible and testable. The open source edition records one agent per install, so tracing across several cooperating agents is a Cloud capability.\n\nStart this week\n\nPick one agent that asks a person to approve its work. For the last month, compute three numbers from whatever data you have: the approval rate, the median time per review, and how many overrides there were. If the approval rate is near total and the time is a few seconds, you have found a rubber stamp before an auditor did. Then record what the reviewer is shown and what they decide as linked events, add a handful of seeded test cases, and look again in a month.\n\nWant to test ZizkaDB on your agent? Try our open source version here: [https://github.com/ZIZKA-AI-SL/ZizkaDB](https://github.com/ZIZKA-AI-SL/ZizkaDB)\n\nInterested in a design partnership? Fill in the form on our site or reach me directly at [founder@zizka.ai](mailto:founder@zizka.ai)\n\nSources: Ben Green, The flaws of policies requiring human oversight of government algorithms · Green 2022, ScienceDirect · On the Quest for Effectiveness in Human Oversight, arXiv · Automation Bias in the AI Act, arXiv · EU AI Act human oversight guide, euaiact.com · Article 14, SpringLex · Article 14 conformity guide, Presencis · Article 14 analysis, DeepInspect · Automation bias reference, Becisions · NTSB findings on the Uber crash, VisualExpert · DevOps.com, Claude Code auto mode default · TechWyse, auto mode and approval rates · NeuralTrust, human in the loop for AI agents · Beri, Human-in-the-loop approval gates · ZizkaDB changelog · ZizkaDB repository", "url": "https://wpnews.pro/news/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything", "canonical_source": "https://dev.to/mir_arshadalitalpur_1b3/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything-gil", "published_at": "2026-11-10 10:29:38+00:00", "updated_at": "2026-11-10 10:51:57.173218+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "ai-agents", "ai-ethics"], "entities": ["EU", "Ben Green", "Vaccaro", "Almaatouq", "Malone", "Digital Omnibus"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything", "markdown": "https://wpnews.pro/news/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything.md", "text": "https://wpnews.pro/news/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything.txt", "jsonld": "https://wpnews.pro/news/the-rubber-stamp-problem-when-a-human-in-the-loop-is-not-overseeing-anything.jsonld"}}