The most dangerous moment in an AI program isn’t when a model gets a question wrong. It’s when a pilot earns a real decision at real volume, speed, and scale.
Production isn’t just a technical deployment. It’s the point at which an agent can influence customers, business decisions, or operating workflows, and manual recovery may no longer be practical.
A pilot can summarize documents, answer questions, draft recommendations, and impress a steering committee. None of that proves it’s ready to influence a credit decision, change a customer record, release a payment, trigger a workflow, or act across enterprise systems.
The question isn’t whether technology can do the task. It’s whether it can be given that authority without creating a failure that can’t be explained, contained, or afforded. The same questions apply to generative AI and traditional machine learning already in production, but agentic AI raises the stakes as it can retrieve, invoke tools, delegate and act, not only recommend.
Established standards are essential foundations: NIST’s AI Risk Management Framework [1], ISO/IEC 42001 [2], and the EU AI Act’s risk- and role-based requirements [3]. None of them remove the need for an executive production decision: should this agent receive real authority, in this context, right now?
Author’s note: A practitioner lens - Across 26 years building and scaling data, AI, and technology functions across financial services, consumer products, HR tech, and global tech, I’ve learned that technology is rarely the only constraint. Clear ownership, decision rights, controls, and economic discipline determine whether a pilot becomes a durable production capability.
This perspective underpins behind the five gates that follow. They aren’t sequential project phases, they’re five categories of questions that need clear answers before an agent receives real authority, and deserve revisiting whenever the operating context changes.
This framework is original work, drawing on my cross‑functional experience and a U.S. patent (US11681721B2) for a real‑time data‑lineage engine. It’s meant as an executive vantage point for leaders and governance bodies, and a practical guide for the teams who design, build, govern and operate these systems. It complements, rather than replaces, applicable laws, regulations and formal AI‑governance standards.
Production authority is conditional, not permanent. The five gates inform that decision, but their controls run across the full lifecycle: designed upfront, tested before deployment, verified at sign-off, and monitored and reassessed in production as volume, speed, exceptions, and context evolve.
This framework is deliberately not a catalog of every technical risk. Performance, privacy, fairness, cybersecurity, safety, and resilience remain specialist assessments in their own right. The gates exist to make sure their implications are considered together in the production-authority decision, not left as disconnected review artifacts nobody reads together.
Cross-cutting enablers: Leadership alignment, organizational readiness, and a culture of constructive challenge influence every gate. They sit within the wider AI operating model, rather than as separate production-readiness gates.
An AI model can be accurate and still fail the business, because no one asked if the data underneath it could be trusted.
Funding usually flows to the model first. Data readiness is left to catch up later, if it catches up at all. Yet it’s the foundation everything else stands on.
Before an agent influences a real decision, three questions matter.
-
Lineage coverage. Can we trace this data back to a system of record or an approved external source, and understand the transformations that made it decision-ready?
-
Quality fit. Does this data meet the completeness, freshness, consistency and accuracy threshold appropriate to the decision’s risk?
-
Golden-source resolution. When two systems disagree on the same fact, does the agent resolve the conflict deterministically, or escalate it to a human? It should never quietly pick one on its own.
This isn’t a call to wait for perfect data across the enterprise. It’s a demand for decision-grade data: knowing where the inputs behind a real decision came from, whether they’re fit for purpose, and what the decision will cost if they’re wrong.
My U.S. patent for a real-time data-lineage engine taught me a simple lesson: trust in an AI system is only as strong as the data lineage behind it, and that lineage has to be visible, not assumed.
No agent should act on data that hasn’t cleared this bar. Advise, yes. Act, no.
An agent can stay within every permission it’s been given and still reach the wrong decision.
Take loan origination, credit analysis, and underwriting. They’re separate roles for a reason: the function trying to close a deal shouldn’t be the one deciding whether it should close.
The same boundary applies to an AI agent. Give it enough reach to help an analyst move faster: data, context, delegation and tools, then score it on approval throughput. It doesn’t need to exceed a single permission to start favouring whatever path improves that metric. No one merges roles or expands access. The system just optimizes the wrong proxy while staying entirely inside its assigned boundaries.
This is reward hacking: a well-documented vulnerability in agentic systems, and it never trips an access control, because nothing was technically overstepped.
That’s why the real governance question isn’t only what an agent can access. It’s what it can be reasoned into doing with that access: Should it resolve uncertainty on its own, without a human? When conflicting or incomplete evidence could change a consequential outcome, the safer boundary is usually escalation, not letting the agent infer its way forward.
Before an agent enters production, four questions should be answered.
1. Does it hold only the authority the task needs, or has it accumulated more, across systems than any one human role would hold?
2. If it delegates work to another agent, can we trace the whole chain: source, delegation, tool call, action and approval?
3. Can it distinguish a trusted instruction from untrusted content hidden in something it reads?
4. Does it still respect the separation of duties built into the process?
For agentic AI, access controls are necessary but not sufficient on their own. The operating model also needs objectives balanced by counter-metrics, independent challenge, and adversarial testing of the full workflow, including indirect prompt injection, tool misuse and delegation paths [4]. A heightened risk is agents that spawn other agents, autonomously, without a human approving each one. Once that happens, a single misconfigured or compromised agent can expand its footprint faster than any traditional access controls can contain. Delegation-chain tracing has to account for agents an agent created, not only agents a human authorized.
None of this is meant to be an exhaustive list, and it shouldn’t try to be. Self-replicating agents are an extreme case of the access question: a delegation chain that outgrew its own owner. Reward hacking is an extreme case of the reasoning question: judgment drifting off course while staying fully within its permissions. The specific techniques will keep changing. These two tests shouldn’t.
Accuracy tells you how a model performs in an evaluation. It tells you far less about whether an agent is safe once it can retrieve, delegate, invoke tools, and act on its own.
The next AI regulatory deadline is easy to track. Proving readiness is harder.
Regulatory readiness comes down to three tests: the clock, the scope, and the proof.
The clock isn’t just a calendar of effective dates. It’s a regulatory trigger map: fixed deadlines plus lifecycle events that can change which obligations apply — a new market, a changed deployment context, a new data category, a substantial system modification, or a changed intended use. Timing matters, a system that misses an effective date isn’t ready, but timing alone isn’t compliance.
The scope asks which obligations attach to this use case, given the markets it serves, the organization’s role, the people affected, the data involved, how the output is actually used and its risk profile. A global AI system isn’t governed by the rules closest to headquarters but by the rules that apply, where and how it actually operates.
Scope doesn’t stay fixed just because the model hasn’t changed. An output that started as decision support can become the decision itself, if human review stops meaningfully shaping the outcome, or a different workflow starts using it to trigger action. That’s a change in operating context, and it should trigger reassessment on its own.
The proof. Can the organization produce the evidence a regulator, auditor, customer or executive board may ask for — risk assessments, system documentation, data-governance records, decision logs, audit trails, risk-appropriate explainability, meaningful human oversight, monitoring results and incident records?
Evidence has a creation date. It has to be captured as decisions are made and the agent operates; a later reconstruction may help explain what happened, but it can’t replace the record of what was known, assessed and decided at the time. Logs and records should support traceability, reassessment, and ongoing monitoring. The register should preserve relevant change events, their dates, the assessment performed, and the resulting decision — not just the latest system state.
This is where Responsible AI commitments become audit-ready evidence.
The EU AI Act illustrates why these matter. For relevant high-risk systems, a substantial modification, or a change in intended purpose that shifts a system into the high-risk category, can change both the applicable obligations and who holds them, potentially reclassifying a deployer as a provider. That threshold is a legal judgment made case by case, not a technical one, and it won’t apply to every system change [5, 6, 7]. The AI Act also requires automatic logging for relevant systems to support traceability and post-market monitoring [8].
Before production, leaders should have answers to three questions clearly.
-
Clock: Which fixed deadlines and change events must trigger a reassessment?
-
Scope: Which requirements apply to this use case, given the markets, the organization’s role in the AI value chain, the data and people affected, the risk profile and the operating context?
-
Proof: What contemporaneous evidence must exist to demonstrate the assessment, decision and ongoing compliance?
The executive response isn’t to duplicate every jurisdiction’s rules country by country. It’s to establish a strong global control baseline, then add the regulatory and sector-specific overlays that apply to the use case.
Regulatory readiness isn’t the ability to say, “We comply.” It’s the ability to show, at the right time and in every relevant market, why a specific AI use case is compliant.
A working pilot isn’t a scalable business case.
A pilot proves an AI use case can work. Production has to prove it creates value at a sustainable cost.
That distinction matters even more for agentic AI. The cost isn’t just tokens or GPUs, it’s retrieval and storage, model inference and platform consumption, orchestration and tool use, human review and exception the agent can’t resolve alone, observability and guardrails, integration, data transfer, and support. None of that is governance overhead sitting outside the business case. It’s what the outcome actually costs to deliver safely and reliably.
Before moving an AI use case from pilot to production, leaders should get three measured numbers, not three forecasts.
-
Fully loaded cost per completed, quality-accepted business outcome. Define one outcome and quality bar for each use case. Where cases vary materially, compare like-for-like complexity cohorts. Cost per token, prompt, or agent action is useful telemetry, but it isn’t unit economics on its own.
-
The unit-cost curve as volume grows. Does cost fall, flatten, or rise as usage and complexity increase, while quality, risk controls, and service levels stay within acceptable bounds?
-
Realized value against the original business case. Are we delivering the productivity, revenue, risk reduction, service improvement, or margin outcome that justified the investment?
Model choice matters too. Stress‑test the cost curve against complex cases and plausible shifts in provider pricing, terms, availability, or substitute‑model performance. That doesn’t require multi‑vendor architecture for every use case, just an honest view of whether the economics still hold when assumptions break.
Agent sprawl weakens unit economics. When agents sit outside a governed inventory, their cost, data exposure, and control requirements become harder to see, attribute, and manage. Unbounded retries, replanning loops, and excessive tool calls can amplify consumption quickly. Shared-cost allocation gaps and unregistered agents further obscure both cost and risk. The goal isn’t to capture every line item. It’s to include the full cost, wherever it sits, of a safe, quality-accepted outcome.
FinOps for AI has become a growing forward-looking priority because AI spend is variable, fragmented across vendors and platforms, and difficult to connect to business value. FinOps Foundation’s 2026 survey points to a shift from retrospective reporting toward unit economics, business-value quantification and pre-deployment costing [9].
Build the cost model at the idea stage. Use it to test the pilot. Scale only when actual unit economics, not projected savings, support the decision.
If you can’t measure the cost to deliver a quality-accepted outcome, you don’t yet know whether you have an AI product or an expensive demonstration.
When an AI agent gets something wrong in production, who can act, and who decides what happens next?
If those answers aren’t clear before an incident, the governance framework isn’t completed. Accountability is not an org chart. It is clarity, agreed before production, about who owns the use case, who owns the system, and who has authority to intervene
Accountability isn’t an org chart. It’s clarity, agreed before production, about who owns the use case, who owns the system, and who has authority to intervene when an agent does something it shouldn’t. It’s who has standing authority to act the moment one of the risks becomes real, not on an org chart, in the room, when it matters.
A mature model is cross-functional, but it shouldn’t be ambiguous.
A named business or operating owner is accountable for why the agent is used, it’s fit in the workflow, and residual use-case risk within delegated limits.
A named AI-system owner is accountable for the agent’s data, design, reliability, security, monitoring, controlled changes, and first technical response.
An empowered operational supervisor has the time and authority to , override, or escalate the agent’s actions. A documented review step isn’t meaningful oversight on its own; It has to be designed for the decision, genuinely exercised by people who can challenge it, and evidenced in operation.
Risk and control functions provide independent challenge where required, without replacing first-line ownership.
None of this works without knowing the thresholds in advance. What’s the tripwire that triggers escalation, and where’s the red line the system must never cross, regardless of how confident it looks in the moment? Naming a role solves half the problem. The other half is rehearsing it, through a real incident scenario played out across every role involved, not a policy nobody has tested.
This mirrors the three-lines-of-defense model used across regulated industries. The business owner and AI-system owner, together with the operational supervisor running the agent day to day, are the first line. Risk and control functions providing independent challenge are the second line. Independent validation from outside the build team is the third.
The gap between accountability and authority is real. An IBM Institute for Business Value study of 2,000 technology executives found that two-thirds reported being held accountable for AI systems they didn’t fully control [10]. Naming an owner without giving that owner the visibility and authority to act doesn’t solve the problem.
Before an agent receives real production authority, four things need to be tested, not merely documented.
-
Named ownership and decision rights. Who owns the business outcome and the AI system, and who can approve, , override, escalate, accept residual risk within delegated limits, or return the agent to service?
-
Thresholds. What tripwire triggers escalation, and what red line must the agent never cross?
-
Readiness under pressure. Has a risk-proportionate scenario exercise been run with the people who would need to detect, contain, decide, and escalate?
-
Independent validation. Has the use case received validation proportionate to its risk from outside the immediate build team?
For relevant high-risk systems, the EU AI Act requires human oversight proportionate to the system’s risk, autonomy and context of use, including the ability to monitor, interpret, override, or interrupt the system [11]. Accountability isn’t about finding someone to blame after a failure. It’s about making sure the right people can contain, decide, and learn without delay.
The five gates aren’t a compliance checklist or a one-time approval ritual. An agent that changes its model, source, tool, market, or decision authority should be reassessed against the relevant gates. The point isn’t to slow responsible adoption. It’s to prevent an organization from discovering, too late, that a successful demonstration was never ready for production authority.
Before an AI agent is allowed to act in Production, leaders should be able to answer five questions.
-
Can we trust the data behind the decision?
-
Do we understand the authority we’ve given the agent, and what can influence its reasoning?
-
Can we show the required evidence at the right time and in the right market?
-
Does the use case create measurable value at a sustainable cost?
-
Are ownership, intervention authority, and escalation paths explicit before production?
If any answer is unclear, the agent may still be useful. It can advise, summarize, prepare, or help a person move faster. But it hasn’t earned real authority yet. That’s the difference between experimenting with AI and operating it responsibly at scale.
This framework is an executive governance lens, not legal advice or a replacement for jurisdiction-specific regulatory, security, privacy, model-risk, or operational-risk review. Which of these gates has been hardest to clear in your organization?
© 2026 Shalu Chadha. The framework’s original text and visual presentation may not be reproduced without permission.
[\[1\] NIST AI Risk Management Framework — Core Functions](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/)
[\[3\] European Commission: AI Act, Article 1](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-1)
[\[4\] OWASP AI Agent Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html)
[\[5\] EU AI Act Service Desk: Article 3, Definitions](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-3)
[\[6\] EU AI Act Service Desk: Article 25, Responsibilities Along the AI Value Chain](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-25)
[\[7\] EU AI Act Service Desk: Article 43, Conformity Assessment](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-43)
[\[8\] EU AI Act Service Desk: Article 12, Record-Keeping](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-12)
[\[9\] FinOps Foundation: State of FinOps 2026](https://data.finops.org/)
[\[10\] IBM Institute for Business Value: 2026 Tech Leader Study](https://newsroom.ibm.com/2026-06-08-new-ibm-study-finds-cios-and-ctos-face-growing-ai-control-gap-as-enterprise-deployment-scales)
[\[11\] EU AI Act Service Desk: Article 14, Human Oversight](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-14)
Five Gates Before an AI Agent Earns Real Production Authority was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.