I watched a test agent try to delete a PostgreSQL table. It had the credentials. It had the intent. The only thing stopping it was a single line in the system prompt that said "you are a read-only assistant."
That line held. But I've seen enough close calls to know that a prompt is not a safety net.
Every week there's a new horror story. An agent that wiped a production database. An agent that mailed 10,000 customers the wrong message. An agent that ran up a $50,000 API bill in an hour. The common thread is not malicious intent. It's architecture that treats the LLM as a trusted operator instead of a powerful but unreliable intern.
The answer is not "don't use agents." The answer is "use agents with the right guardrails." I've built systems that use LLMs for scoring, generation, and structured extraction at production scale. Here are the patterns that keep agents from doing damage.
Every agent I build starts life as a read-only system. It can observe, analyze, and report. It cannot write, delete, or execute.
This is not a feature you add later. It is the default state. You promote the agent to write access only when you have proven that it needs it and that you can control what it writes.
The implementation is straightforward. Give the agent a database connection with read-only credentials. Wrap all write operations behind a separate service that requires explicit approval. Never let the agent call a mutation API directly.
Here is what that looks like in practice with OpenAI function calling:
const readOnlyFunctions = [
{
name: "query_database",
description: "Execute a SELECT query against the database. Read only.",
parameters: {
type: "object",
properties: {
sql: { type: "string", description: "The SELECT query to execute" },
},
required: ["sql"],
},
},
{
name: "search_records",
description: "Search for records matching criteria.",
parameters: { /* ... */ },
},
];
// Write operations live in a separate system, not exposed to the agent.
// The agent can REQUEST a write, but a human must approve it.
const writeFunctions = [
{
name: "request_write",
description: "Request a write operation. Requires human approval.",
parameters: {
type: "object",
properties: {
action: { type: "string", enum: ["update", "delete", "insert"] },
target: { type: "string" },
reason: { type: "string" },
},
required: ["action", "target", "reason"],
},
},
];
The agent can ask for a write. It cannot perform one. This forces a human-in-the-loop gate at the architectural level, not just the prompt level.
The classic "human in the loop" is a yes/no confirmation dialog. It works until users get fatigue and start clicking "yes" without reading. I've seen it happen.
The better pattern is to require the human to re-express the intent in their own words. Instead of "approve this action," show a summary and ask the user to type or confirm the specific change. For example, rather than a button that says "Delete user 1234," show the user a preview of what will happen and require them to type "delete user 1234" to proceed.
For high-risk actions, add a cooldown period. The agent cannot execute the same action twice within a window. This prevents a runaway loop where the agent floods the approval queue with identical requests.
Consider a social media engagement agent that autonomously replies to conversations. Suppose you give it the ability to generate replies but not to post them. A human reviews each generated reply, edits if needed, and clicks publish. That extra step catches hallucinations, tone mismatches, and the kind of inappropriate joke an LLM thinks is funny. The agent never touches the posting API at all. That separation is what saves you.
Agents fail. They time out. They retry. The worst thing you can do is let an agent retry a non-idempotent action.
Every mutation your agent performs should be safe to call twice. Use idempotency keys. Generate a unique key for each action, and if the agent retries with the same key, the system deduplicates it.
Here is a pattern I use for job application agents that need to submit applications:
async function submitApplication(application: Application, idempotencyKey: string) {
// Check if this key was already processed
const existing = await db.applications.findUnique({
where: { idempotencyKey },
});
if (existing) {
return existing; // Already submitted, return the result
}
// Submit the application
const result = await atsApi.submit(application);
// Store the result with the key
await db.applications.create({
data: {
idempotencyKey,
status: result.status,
submittedAt: new Date(),
},
});
return result;
}
The agent always passes an idempotency key. If the API call times out and the agent retries, the second call is a no-op. No duplicate applications, no double charges, no double deletions.
The agent should never have direct access to your production database or any production API. This is non-negotiable.
Create a proxy layer that sits between the agent and every external system. This proxy validates every request against a policy. The policy defines what the agent is allowed to do, what data it can see, and what it cannot touch.
For databases, I run agents against a replica that is read-only and lag-tolerant. The agent can see near-real-time data but cannot write to it. If the agent needs to write, it goes through the proxy which logs every mutation and enforces rate limits.
For external APIs, the proxy strips dangerous capabilities. If the agent is calling a CRM API, the proxy removes the DELETE endpoint from the available routes. The agent never sees the full API surface. It sees only what the proxy exposes.
When an agent's action fails, the default behavior should be to stop, not to retry blindly. A circuit breaker pattern works well here: suppose the agent gets three errors in a row from the same service. The circuit opens. The agent cannot call that service again until a human resets it.
This prevents a single misbehaving agent from hammering a downstream system during an outage. It also prevents the agent from making things worse by trying to "fix" a failed operation with a destructive alternative.
I also log every action the agent takes, including the full reasoning chain. When something goes wrong, I can replay the agent's decision-making process and see exactly where it went off the rails. This is invaluable for debugging and for improving the guardrails.
The pattern that works across all agent systems I've designed is this: the agent is a proposal engine, not an execution engine. It generates ideas, arguments, and plans. A separate, tightly controlled execution layer decides whether to carry them out.
The agent never touches the production database. It never calls a mutation API directly. It never has admin credentials. It is given the minimum permissions needed to do its job, and those permissions are enforced at the infrastructure level, not the prompt level.
If your team is wrestling with AI agent safety and shipping slower because you're afraid of what the agent might do, that is the kind of thing I help with. I build production AI systems that are powerful enough to be useful and safe enough to ship without fear. Happy to compare notes.
Written by Abdul Rehman, full-stack AI engineer building production SaaS, MVPs, and AI automation. More at PrimeStrides.