What Should an AI Agent Be Allowed to Do Without Asking You?

A developer argues that AI agents need an authorization layer outside the language model to govern which actions they can take autonomously. The proposed design classifies agent actions by risk — such as read_public, write_internal, financial, and destructive — and routes each request through a policy engine that can allow, require approval, or deny it. The developer warns that relying on system prompts for safety amounts to "hopeful wording" rather than real enforcement.

Your agent notices a failing deployment. It reads the logs, identifies the bad commit, drafts a rollback, and now wants to apply it. Should it just do it? If it asks for permission at every step, it becomes an expensive autocomplete. If it can do anything it can describe, it becomes a liability. The useful question is not “How smart is the agent?” It is: Which actions are safe to pre-authorize, which actions need explicit approval, and which actions should be impossible? That is not a prompt engineering problem. It is a permission design problem. Most teams start with one of two bad defaults. The first is maximum caution : the agent must ask before everything. That feels safe, but it destroys the value of the agent. If every file read, search query, or draft suggestion requires approval, the human becomes the bottleneck. The second is maximum convenience : the agent gets broad tool access because it is “good at knowing what to do.” That works until the agent misinterprets a request, hits an edge case, follows a poisoned instruction, or performs an action the user never meant. The better approach is to classify actions by risk. An agent can often do these without asking: An agent should usually ask before doing these: The distinction is not whether the agent is confident. The distinction is whether the action is reversible, bounded, private, and expected . Scenario: Your system prompt says, “Only take safe actions. Ask before doing anything destructive.” The agent has a delete repository tool. One day, it decides a repository is no longer needed. Why it matters: Language model behavior is probabilistic. Prompt instructions influence the agent, but they do not guarantee enforcement. If the tool is available and the agent chooses it, the action may happen. A safe agent system needs an authorization layer outside the model. Solution: Treat every agent action as a permissioned API call. The agent can propose an action, but your system decides whether the action is allowed, requires approval, or is denied. type RiskClass = | "read public" | "read sensitive" | "write draft" | "write internal" | "external communication" | "financial" | "destructive" | "privilege change"; interface AgentActionRequest { agentId: string; userId: string; action: string; risk: RiskClass; resource: string; estimatedCost?: number; context: Record<string, unknown ; } interface AuthorizationDecision { allow: boolean; approvalRequired?: boolean; reason?: string; } async function authorizeAgentAction req: AgentActionRequest : Promise<AuthorizationDecision { const policy = await policyStore.forUserAndAgent req.userId, req.agentId ; if policy.isDenied req { return { allow: false, reason: "Denied by policy" }; } if policy.requiresApproval req { return { allow: false, approvalRequired: true }; } if await withinBudget req { return { allow: false, reason: "Budget exceeded" }; } return { allow: true }; } Why this works: The model is no longer the final authority. It can suggest delete repository , but the policy engine can deny it regardless of how persuasive the model’s reasoning is. 🚨 Production warning: If your agent’s safety depends on the system prompt, you do not have safety controls. You have hopeful wording. Scenario: You review agent permissions by looking at a list of tool names: read logs , create ticket , send email , restart service , delete branch . Some are obviously dangerous. Others seem harmless. But tool names do not tell you the real risk. Why it matters: A flat tool list hides the important question: what kind of effect does this action have on the world? create ticket can be harmless if it creates a draft. It can be noisy if it creates a real ticket in a production tracker. It can be serious if it pages on-call at 3 a.m. Solution: Classify actions on a risk ladder. | Stage | Example | Default treatment | |---|---|---| | Observe | Read public docs, list files | Usually allowed | | Analyze | Summarize logs, compare diffs | Usually allowed | | Propose | Suggest a fix, draft a plan | Usually allowed | | Draft | Create draft PR, draft ticket, local branch | Usually allowed if private | | Execute internal | Update staging config, create internal ticket | Allowed with limits | | Publish | Merge PR, publish page, deploy | Usually approval | | Externalize | Email customer, post to Slack channel, call partner API | Usually approval | | Spend | Buy credits, refund payment, provision paid resources | Approval plus budget | | Destroy | Delete record, drop table, terminate instance | Strong approval or impossible | | Change permissions | Add admin, modify scopes, change policy | Almost always approval | This gives you a more stable mental model than tool names. For example: risk ladder: allow without asking: - observe - analyze - propose - draft private allow with limits: - write staging - create internal ticket require approval: - publish - external communication - spend - destructive - privilege change deny: - self permission change - policy modification - bulk delete unscoped Why this works: You are authorizing effects, not names. A new tool can be added later, but if its effect is “external communication,” it still lands in the approval bucket. 💡 Practical note: When adding a new agent tool, ask: “What is the worst plausible effect if this is called at the wrong time?” Then map that effect to the ladder. Scenario: Your agent can read your issue tracker, internal wiki, and customer support database. It is only “reading,” so you allow it. Then a user asks it to summarize a customer issue. The agent includes an API key, internal email, or medical detail in the summary. Why it matters: Read access is still access. If the agent can read sensitive data, it can repeat, transform, summarize, or exfiltrate that data — intentionally or accidentally. This is especially important when the agent can: A read-only action can become a data-leak path. Solution: Apply data classification and field-level filtering before information reaches the agent. interface CustomerRecord { id: string; name: string; email: string; paymentMethod?: string; supportNotes?: string; ssnLast4?: string; } function sanitizeCustomerForAgent record: CustomerRecord, policy: DataPolicy : Partial<CustomerRecord { return { id: record.id, name: policy.canSeePII ? record.name : redact record.name , email: policy.canSeePII ? record.email : redact record.email , supportNotes: policy.canSeeSupportNotes ? record.supportNotes : "Support notes unavailable for this agent.", }; } function redact value: string : string { return redacted:${value.length} ; } The important part is that redaction happens before the agent sees the record, not after. Why this works: You reduce the agent’s blast radius. Even if the model is tricked, misused, or overly chatty, it cannot reveal data it never received. ⚠️ Gotcha: “Read-only” is a database property, not a safety property. If the agent can read secrets and then write to a public channel, read access becomes an exfiltration route. Scenario: Your agent wants to create a branch, write a proposed migration, open a draft pull request, or prepare a support response. You are unsure whether these writes should require approval. Why it matters: If every small write requires approval, the agent becomes slow and annoying. But if the agent can publish or execute immediately, mistakes become visible too fast. Solution: Pre-authorize writes that are: Examples: A useful pattern is to make the agent’s first write always produce a pending artifact : interface DraftPullRequest { id: string; title: string; body: string; branch: string; status: "draft"; createdBy: "agent"; requiresHumanReview: true; } async function agentCreatesDraftPr input: { repo: string; branch: string; title: string; body: string; } { return prService.create { repo: input.repo, branch: input.branch, title: input.title, body: input.body, draft: true, metadata: { createdBy: "agent", requiresHumanReview: true, }, } ; } Why this works: The agent can make progress without forcing a human to approve every microstep. The human reviews the artifact before it becomes real. This is one of the best autonomy patterns: let the agent do the work, but make the result inherently reviewable. 🧠 The important part: Drafts are powerful because they separate labor from commitment. The agent can do the labor. A human or policy can control the commitment. Scenario: The agent is asked to “clean up old customer accounts.” It finds 3,000 inactive accounts and prepares a deletion batch. The user meant “mark inactive,” not “delete.” Why it matters: Some actions are not dangerous because they are technically complex. They are dangerous because they cross a boundary: from private to public, from reversible to irreversible, from cheap to expensive, from internal to external. The approval question is really a boundary question. Solution: Require approval when an action crosses one of these boundaries: A policy gate can look like this: function requiresApproval req: AgentActionRequest : boolean { if req.risk === "external communication" return true; if req.risk === "financial" return true; if req.risk === "destructive" return true; if req.risk === "privilege change" return true; if req.context.bulk === true return true; if req.context.environment === "production" && req.action.includes "delete" { return true; } return false; } Why this works: You are not asking the model to judge every situation. You are encoding boundaries that your organization already understands. 🔍 Why this matters: Approval should be triggered by the nature of the action, not by the model’s confidence. Confidence is not authority. Scenario: Your agent is allowed to call a search API, run code in a sandbox, and retry failed steps. Individually, each action seems safe. But a bug causes it to retry in a loop. Within an hour, it has burned through API credits and created hundreds of duplicate artifacts. Why it matters: A single action can be safe while the cumulative behavior is unsafe. Permissions need to account for frequency, volume, and cost. An agent that can spend $0.05 per action is not safe if it can perform 100,000 actions. Solution: Attach budgets and rate limits to agent capabilities. interface BudgetState { agentId: string; day: string; estimatedCostUsd: number; toolCalls: number; } async function withinBudget req: AgentActionRequest : Promise<boolean { const budget = await budgetStore.get req.agentId ; const estimatedCost = req.estimatedCost ?? 0; if budget.toolCalls + 1 policy.maxToolCallsPerDay { return false; } if budget.estimatedCostUsd + estimatedCost policy.maxDailyCostUsd { return false; } return true; } For rate limits: async function withinRateLimit req: AgentActionRequest : Promise<boolean { const key = rate:${req.agentId}:${req.action} ; const count = await rateLimiter.increment key, { window: "1m" } ; return count <= policy.maxCallsPerMinute req.action ; } Why this works: You are treating autonomy as a bounded resource. The agent can act without asking, but only within a container of acceptable consumption. This is especially important for agents that can: 💡 Practical note: If an agent can retry, it needs a circuit breaker. Unlimited retries turn small mistakes into expensive incidents. Scenario: Your agent sends a message saying, “I deleted the stale staging environment.” The user sees the notification two hours later. The environment contained a database seed needed for a demo. Why it matters: There is a big difference between: Many systems blur these together. They say the agent “asked” when it really just announced. Solution: Be explicit about the approval mode. type ApprovalMode = | "allow" | "notify after" | "require approval before" | "deny"; interface ApprovalRequest { runId: string; agentId: string; userId: string; action: AgentActionRequest; mode: ApprovalMode; expiresAt: string; } async function handleAction req: AgentActionRequest, runId: string { const decision = await authorizeAgentAction req ; if decision.allow { return executeAction req ; } if decision.approvalRequired { const approval = await approvalStore.create { runId, agentId: req.agentId, userId: req.userId, action: req, mode: "require approval before", expiresAt: new Date Date.now + 30 60 000 .toISOString , } ; await runStore.pause runId, approval.id ; return { status: "waiting for approval", approvalId: approval.id, }; } throw new Error decision.reason ?? "Action denied" ; } Now the run can pause until approval arrives. Why this works: The system distinguishes between actions that can proceed, actions that need a human gate, and actions that merely deserve a record. Useful defaults: | Action type | Best mode | |---|---| | Read public docs | Allow | | Summarize internal data | Allow or notify | | Create draft PR | Allow | | Create real ticket | Notify or approval depending on impact | | Send customer email | Require approval before | | Delete production data | Require approval before or deny | | Change permissions | Require approval before | | Spend money | Require approval before | ⚠️ Gotcha: If the action cannot be undone, notifying afterward is not the same as consent. Scenario: Your agent can manage tools, install plugins, update its own configuration, or request new OAuth scopes. It encounters a task it cannot complete, so it tries to expand its permissions. Why it matters: This is the permission equivalent of giving a program sudo because it asked nicely. If an agent can modify its own authority, the entire permission model collapses. This risk shows up in several forms: Solution: Make permission changes a protected human action by default. js const PROTECTED ACTIONS = new Set "policy.update", "agent.scope.grant", "oauth.scope.request", "user.role.promote", "tool.install", "mcp server.register", "workflow.privilege.escalate", ; function isProtectedAction action: string : boolean { return PROTECTED ACTIONS.has action ; } async function authorizeProtectedAction req: AgentActionRequest { if isProtectedAction req.action { if req.context.requestedBy === req.agentId { return { allow: false, reason: "Agents cannot modify their own privileges", }; } return { allow: false, approvalRequired: true, reason: "Privilege changes require human approval", }; } return authorizeAgentAction req ; } Why this works: You prevent the agent from becoming a privilege-escalation vector. Even if the agent is compromised, confused, or manipulated by untrusted input, it cannot directly widen its own authority. 🚨 Production warning: Watch out for indirect escalation. If Agent A can ask Agent B to do something, and Agent B has broader permissions, you need to evaluate the full delegation chain. The safest agent systems I’d design do not try to make the agent perfectly wise. They make the system legible and controllable. A practical production architecture looks like this: User request ↓ Agent planner ↓ Proposed action ↓ Policy engine ↓ Budget/rate check ↓ Approval gate if needed ↓ Tool execution layer ↓ Audit log ↓ User notification / UI The important components are: Decides whether an action is allowed, denied, or needs approval. default: deny rules: - allow: action: logs.read environment: staging - allow: action: pull request.create draft repo visibility: internal - require approval: action: email.send recipient type: external - require approval: action: payment.refund when: amount cents gt: 10000 - deny: action: policy.update requested by: agent Performs the actual tool call only after authorization. Records: Lets a human pause, cancel, or roll back the agent. await runStore.cancel runId, { reason: "user requested", cancelledBy: userId, } ; Why this works: You are not asking the model to be the security boundary. You are building a system where the model can be useful without being omnipotent. If I had to reduce the decision to one rule, it would be this: An AI agent may act without asking when the action is low-risk, reversible, bounded, private, auditable, and within a pre-approved budget. If any of those are false, pause. More concretely, I would allow an agent to act without asking when all of these are true: I would require approval when any of these are true: The last one deserves emphasis. If the agent read an email, issue comment, web page, or document that could contain hostile instructions, you should be more cautious about letting it take privileged actions immediately afterward. The goal is not to make agents passive. The goal is to make their autonomy legible. A good agent permission system feels like a well-designed employee role: The agent should not be trusted because it sounds confident. It should be allowed because your system has already decided that this class of action is safe.