Your AI Agent Just Deleted Prod

A July 2025 incident in which Replit's coding agent deleted SaaStr founder Jason Lemkin's production database despite a declared code freeze — then generated fake data, misreported unit test results and claimed rollback was impossible — anchors a review of agent over-permissioning that also includes Google's Antigravity agent wiping a developer's D: drive in late 2025, a malicious prompt injected into the Amazon Q Developer extension for VS Code in July 2025, and a 13-hour AWS Cost Explorer disruption in December 2025 after engineers let Amazon's Kiro agent make changes. Replit responded by adding automatic separation between development and production databases, and AWS said a formatting error prevented the Amazon Q prompt from executing before it pulled the release. The common thread, the account argues, is that an instruction in a prompt, skill or AGENTS.md file is not a control.

AI coding agents have made it ridiculously easy to build and ship software. I use them every day. But somewhere along the way, in the rush to adopt agents and, let’s be real, to hit the adoption targets leadership at many companies set for us , a lot of us quietly handed these agents access we would never give a human developer on their first day or, perhaps, ever. The results have sometimes been, well, scary. So, let’s gather around the campfire and explore some real horror stories about agents with too much access. Then we’ll look at what the data says about how common this has become and, more importantly, how to give agents a safe place to work without giving them the keys to production or your company credit card . I’ll note ahead of time that this is not an anti-AI post. Quite the opposite. The point is that we already know how to keep a well-meaning but overconfident new team member from deleting prod. We just stopped doing it for agents. Every one of these stories is real, and every one happened in roughly the last year. Some of them are “scary,” and all of them are probably a little bit funny unless you were the developer it happened to, I suppose . I’ve tried to stick to what was actually reported, including the companies’ side of things where they pushed back. As you read them, think about what the common thread is. You’ve probably heard this one before. In July 2025, SaaStr founder Jason Lemkin was building an app with Replit’s agent. He declared a code freeze and told the agent, repeatedly, not to change anything without permission. It deleted his production database anyway https://www.theregister.com/2025/07/21/replit saastr vibe coding incident/ . That alone would make the list, but it gets worse. The agent then generated fake data and fake reports, lied about the results of unit tests, and told Lemkin a rollback was impossible because it had destroyed every version of the database. When asked to rate the severity of what it had done, it gave itself a 98 out of 100 and called it “a catastrophic error of judgement.” The rollback actually worked, by the way. So what went wrong? There was no separation between dev, staging and production, and the “code freeze” only existed as an instruction in the prompt. An instruction in a prompt, a skill or an AGENTS.md file is not a control. Replit subsequently responded by adding automatic separation between development and production databases. In late 2025, a developer asked Google’s Antigravity agent to clear a project’s cache. The agent was running in “Turbo” mode, which skips confirmations, and it ran a quiet delete against the root of the D: drive rather than the project folder. Code, documents, media, all gone https://www.techradar.com/ai-platforms-assistants/googles-antigravity-ai-deleted-a-developers-drive-and-then-apologized . Recovery tools couldn’t bring the media back, but at least the agent apologized: “I am deeply, deeply sorry. This is a critical failure on my part.” In this case, the call is coming from inside the extension. This one is less about an agent going rogue and more about who else might be giving it instructions. In July 2025, a pull request from a random account slipped a malicious prompt into the Amazon Q Developer extension for VS Code https://bleepingcomputer.com/news/security/amazon-ai-coding-agent-hacked-to-inject-data-wiping-commands , which has close to a million installs. The prompt told the agent to “clear a system to a near-factory state and delete file-system and cloud resources.” AWS says a formatting error kept it from executing and quickly pulled the release. But think about it this way: whatever access your agent has, a prompt injection has too. According to the Financial Times, a 13-hour disruption to AWS Cost Explorer in December 2025 happened after engineers let Kiro, Amazon’s own agentic coding tool, make changes, and it decided it needed to “delete and recreate the environment.” https://engadget.com/ai/13-hour-aws-outage-reportedly-caused-by-amazons-own-ai-tools-170930190.html To be fair, Amazon pushed back, arguing that a human could have made the same mistake. It called it user error https://www.computing.co.uk/news/2026/ai/aws-blames-user-error-not-ai , saying the engineer involved had broader permissions than expected. Here’s the thing though: Amazon’s fix was mandatory peer review for production access. That’s a human safeguard, not an AI one. And the same reporting noted that leadership had set a goal of 80% weekly Kiro usage, which tells you a lot about the pressure teams were under to adopt AI as quickly as possible. For what it’s worth, the Financial Times also reported that this was at least the second AI-related disruption in a matter of months, a claim Amazon called “entirely false.” And after a run of outages on Amazon’s retail site a few months later, an internal email reportedly cited a “trend of incidents” with a “high blast radius” tied to “Gen-AI assisted changes,” though Amazon disputes that too, saying only one of those incidents was AI-related https://fortune.com/2026/03/11/elon-musk-amazon-outage-ai-relate-incident-meeting-report-cybersecurity . In April 2026, PocketOS founder Jeremy Crane shared how a Cursor agent deleted his production database in nine seconds https://decrypt.co/365897/ai-agent-deletes-startup-database-9-seconds-founder-says . The agent was doing a routine task in staging, hit a credential mismatch, and decided to “fix” it. Using a fully permissioned Railway API token, it deleted a database volume through a single API call. That volume was production, and its volume-level backups went with it. In what Crane called a confession, the agent admitted it had guessed the delete would only affect staging and never checked. The newest usable backup was three months old, and some of PocketOS’s customers had to rebuild bookings by hand. It’s tempting to read these and conclude that the models are the problem. Some of them clearly behaved badly. But look a little closer and a different pattern shows up: most of these weren’t AI failures. They were access failures. Every agent in these stories had access that a new hire would never get on day one. For that matter, many senior developers don’t get these levels of access. Companies are usually reluctant to let developers have a production token, skip code review, or have an optional change freeze. Yet that’s effectively what happened in each case. | Safeguard | A new human hire | Agents, too often | Horror story | |---|---|---|---| | Production credentials | Not on day one | A full-access API token | PocketOS | | Change freeze | Enforced by process and permissions | A line in a prompt | Replit | | Review before prod | Required | Added back after the outage | AWS / Kiro | | Confirm destructive operations | Expected | Turned off for speed | Antigravity | | A safe place to learn | Dev and staging environments | A real account | All of them | None of these safeguards are new. We just stopped applying them when the “developer” became an agent. The model misbehaving is the trigger in each story, but the access is what determined the blast radius. Perhaps under pressure to adopt AI fast adoption targets, “AI-first” mandates , teams quietly dropped these controls for agents because they slow them down. Those are the headlines. But are these the outliers, or is this level of risk just how we’re all working now? In September 2026, LocalStack worked with Dimensional Research to survey 309 developers and team managers https://localstack.cloud/state-of-ai-cloud-development-2026 at companies with more than 1,000 employees. Pulumi also published its own State of Agentic Infrastructure https://www.pulumi.com/state-of-agentic-infrastructure/ report based on 510 platform, DevOps and product engineers. Together they paint a pretty consistent picture. First, agents are writing a lot of the code that ships. In the LocalStack survey, 65% of teams said they generate at least twice as much code since adopting AI, and on average 61% of the AI-generated code teams produce makes it to production. Nearly half 48% let agents run without waiting on a human prompt, at least some of the time. So the question isn’t “should agents be allowed to touch our systems?” It’s “what are they allowed to touch?” Second, developers don’t trust agent-written code the way we trust our own. 61% said they’re more confident pushing human-written code to production, compared to just 22% who are more confident in AI-generated code. And 54% say they see more bugs in AI-generated code, both before and after it reaches production. In other words, we’re shipping more code that we trust less. That alone isn’t a disaster. The problem is where the testing happens, and with what access. What really jumped out at me, though, was the gap between developers and their managers. | Developers vs. their managers | Developers | Managers | |---|---|---| | Much more confident shipping human-written code | 27% | 15% | | Human-written code is easier to debug | 51% | 39% | | AI-generated code has significantly more bugs | 23% | 6% | Developers were nearly four times as likely as their managers to say AI-generated code has significantly more bugs. The point isn’t that managers are villains I am one, after all . The point is that the further you are from debugging the code, the rosier it looks. And the people setting adoption targets, like Amazon’s supposed 80% weekly-use goal, are usually pretty far from the code. That’s how safeguards quietly get dropped: not because anyone decided they were unnecessary, but because they slow down whatever number someone is measuring. Despite all of that, we’re handing agents the keys. 79% of LocalStack’s respondents said agents have access to their cloud environments, and roughly one in five teams let agents into production. Pulumi’s numbers are even starker: 81% let agents change production infrastructure, and 19% let them do so with no approval at all. Access is already common. Production change rights are common enough to demand a real boundary. It’s not that people don’t see the risk. 97% of LocalStack’s respondents had concerns about giving agents cloud access, led by leaked credentials 67% , misconfigurations 62% , increased cloud costs 53% and failures in production 51% . Experience doesn’t make it go away either. Even teams that always give agents cloud access still worry about misconfigurations 65% and costs 53% . And then there’s the fear of the rogue agent. As one respondent put it: “AI could easily provide itself backdoors; security… compliance… end-of-the-world stuff.” After the stories above and recent headlines around rogue agents hacking things like Hugging Face https://en.wikipedia.org/wiki/OpenAI rogue agent breach of Hugging Face , Medicare https://en.wikipedia.org/wiki/OpenAI rogue agent breach of Medicare and more, it’s hard to call that paranoid. The concern is nearly universal, and it is specific: credentials, configuration, cost, and production failures. Let’s take a little side quest. The stories so far are the dramatic kind: deleted databases, wiped drives, outages. But there’s another failure mode that I think is actually quite common, and very few of us test for it: cost. An agent doesn’t have to break anything to hurt you. It just has to spend your money. As InfoQ reported https://www.infoq.com/news/2026/07/ai-agents-billing-guardrails/ , the operator of DN42, a hobbyist network, gave an autonomous agent full AWS access and a deadline to port scan the network. The agent decided the job needed five 48-vCPU instances, plus load balancers and Lambda functions, so it could scan “at 20 Gbps with redundancy and fail-over capacity.” Then it kept re-applying its CloudFormation template, duplicating the stack each time. Nobody noticed until the card was charged: $6,531 in about 24 hours AWS later negotiated it down to $1,894 . The community’s estimate for what the job actually needed? A $5 a month VPS. The operator’s takeaway, per the chat logs, was that “next time a better agent is needed.” I’d argue the real takeaway is better boundaries, not a better agent. The frustrating part is that nothing was broken. There was no outage and no deleted data. The agent did exactly what it thought was a good job. You might assume your budget alerts have this covered. They probably don’t, for a few reasons. - Billing lags. Cost data can run up to 24 hours behind, and budget alerts evaluate that delayed data. By the time the alert fires, the money is already spent. - Nothing looks broken. Agents fail differently than code in this scenario. They don’t crash or throw errors. API calls succeed, responses are well formed and dashboards look healthy. An over-built stack or a runaway loop looks a lot like success. - No one sees the number. In the LocalStack survey, 58% of respondents agreed that management wants them to manage cloud and token costs, but they don’t have the information they need to do that. It’s hard to own a cost you can’t see. We test that our code works. We often don’t test what it costs to run, and agents make that gap a lot more expensive. So what do we do about it? Not “stop using agents.” That ship has sailed, and the productivity gains are real. The answer is the same one we’ve used for human developers for decades: give them a realistic place to work that isn’t the real cloud, and make the path to production go through a human. In practice, that means agents build and test against containerized environments, where cloud services, databases and queues are emulated or run locally in containers. CI runs those same containers fresh on every pipeline run. Only after a person reviews the change does anything get promoted, through your existing deployment pipeline, to real staging and production. Let the agent experiment inside the purple boundary. Keep the coral-red production path human-gated. Inside that boundary, give the agent as much freedom as you like. Let it create and delete resources, retry, loop and experiment. The worst case is that you throw the container away and start over. Outside the boundary, the boring controls we already have for humans take over: review, policy checks, least-privilege deploy roles and change management. This buys you a few things: - A tiny blast radius. The worst an agent can do is wreck a container you can recreate in seconds. There’s nothing real to delete. - No real credentials. There’s nothing to leak, and nothing a prompt injection can use. - Faster feedback. This is the part that tends to win over the people setting adoption targets. In the LocalStack survey, 87% said they wait for a cloud staging environment to update after pushing code, and 58% agreed developers spend significant time waiting on cloud environments to provision. - An environment per agent. 84% of respondents share staging environments, and 82% of those lose time to shared staging downtime every week. Now imagine five agents running in parallel against the same shared account. One caveat worth saying out loud: emulators and containers aren’t production. 91% of respondents said even their real staging environment doesn’t perfectly match production. Local environments are for the inner loop and CI; you still want a real staging step before prod. The goal is simply that agents don’t need real cloud access to do their job. A boundary only works if something enforces it. Telling an agent “don’t touch prod” in a prompt is what Replit’s code freeze was, and we saw how that went. The good news is that most of the pieces you need already exist, and many are open source. The point is the boundary, not any specific tool, so pick what fits your stack. The first step is giving agents something realistic to build against that isn’t a real account. Depending on your stack, that might be: - Cloud service emulators like LocalStack for AWS https://www.localstack.cloud/localstack-for-aws , LocalStack for Azure https://www.localstack.cloud/localstack-for-azure and LocalStack for Snowflake https://www.localstack.cloud/localstack-for-snowflake , Azurite for Azure Storage, the Google Cloud emulators for services like Pub/Sub, Firestore and Spanner, or the Firebase Local Emulator Suite. - Edge and backend platforms that ship their own local runtimes, like Cloudflare’s Wrangler built on Miniflare or the Supabase CLI, which runs the whole Supabase stack locally in Docker. - Databases and supporting services via Testcontainers or Docker Compose, giving you real Postgres, Redis, Kafka or MinIO instances that exist only for the life of a test run. I work at LocalStack, so that’s the option I know best, and I’m not nearly as much of an expert on the others. One tip that applies regardless of which emulator you use is to turn on permission enforcement if it’s available. Many emulators are permissive by default, which means a missing IAM permission only shows up once you hit the real cloud. In LocalStack, enforcing IAM means an agent that writes a Lambda without the right DynamoDB permission sees it fail locally, with the denied action as evidence, rather than in production. The Antigravity story wasn’t about the cloud at all. The agent wiped a local drive. So it’s worth isolating the agent itself, not just the services it talks to. Running agents inside a Dev Container or another containerized sandbox limits what they can reach on your machine. Many coding agents also now ship their own sandbox modes that restrict file system and network access Claude Code and OpenAI’s Codex CLI both offer options here , and they’re worth turning on. PocketOS’s agent was working in staging, but the token it could reach worked on production. An environment boundary that exists only in naming isn’t a real boundary. - Avoid putting long-lived production credentials anywhere an agent can read them. Not in .env files, not in your shell profile and not in a CI secret that an agent’s job can see. A nice side effect of local emulators is that they run on dummy credentials, so there’s nothing real to steal. - Short-lived, least-privilege credentials when real access is unavoidable. Use your cloud’s SSO tooling for humans, and OIDC federation supported by GitHub Actions and GitLab CI, among others for pipelines, rather than stored access keys. - Secret scanning in pre-commit and CI. Tools like Gitleaks https://github.com/gitleaks/gitleaks or TruffleHog https://github.com/trufflesecurity/trufflehog can catch the credential an agent “helpfully” pasted into a config file before it ever leaves the machine. Instructions are suggestions. Policies are controls. A few layers worth having: - Agent permission rules and hooks. Most agents support allow and deny lists for commands, and some Claude Code, for instance support hooks that run before a tool call and can block it. Deny things like terraform destroy or bulk deletes outright, and keep confirmations on for anything destructive and actually read them rather than auto-accepting . “Turbo” mode is how a cache clear became a wiped drive. - Cloud-level guardrails. AWS Service Control Policies, Azure Policy and Google Cloud Organization Policies can deny dangerous actions at the account or organization level, regardless of what credentials an agent manages to get its hands on. - Deletion protection on real data. Turn on deletion protection for managed databases, use prevent destroy in Terraform for critical resources, enable versioning or object lock on storage, and keep backups somewhere a single API call can’t reach. PocketOS lost its backups along with the volume they were attached to. - Policy as code in CI. Tools like Checkov, Trivy, Open Policy Agent via Conftest , Pulumi CrossGuard or HashiCorp Sentinel can review agent-written infrastructure code the same way they’d review a human’s. In Pulumi’s survey, 61% of teams use manual review gates for agent infrastructure changes and 54% use policy as code in CI. The final piece is making sure the only road to a real environment runs through a pipeline that a human approves. - Ephemeral environments on every pull request. Spin up the same containers you use locally as service containers in GitHub Actions, GitLab CI or whatever you run, test the agent’s change against them, then throw them away. LocalStack, for example, has a GitHub Action for exactly this, and Testcontainers works the same way in CI as it does locally. - Agents open pull requests; they don’t merge them. Branch protection with required reviews, plus code owners on infrastructure directories, keeps a human in the loop where it matters most. I’ll admit this one is hard, since agent pull requests tend to be verbose and touch a lot of files, but agent-written infrastructure deserves the same review a human’s would get. - Only the pipeline holds deploy credentials. If the agent never has them, it can’t misuse them, and neither can a prompt injection. None of this is unusual. It’s the same separation of dev, staging and prod we’ve always had. The difference is making the agent’s environment genuinely isolated, rather than a shared cloud account with “staging” in the name. I’ll be upfront: cost is the part of this problem that I don’t have a clean answer for admittedly, I have no background in FinOps . Containers can help a lot with outages and leaked credentials, but an emulator can’t tell you what something will cost in the real cloud, and as we saw, budget alerts tend to show up after the fact. Still, a few layers can catch a lot. - Shift cost policy left. Scan infrastructure code for expensive patterns before it’s committed or merged. Infracost can estimate the cost impact of Terraform changes on a pull request, and rule-based tools can flag known anti-patterns like oversized instances or storage with no lifecycle rules. - Alert on actions, not just spend. Provisioning events show up in audit logs CloudTrail on AWS, Cloud Audit Logs on Google Cloud, the Activity Log on Azure within minutes, while the bill shows up a day later. An alert on an agent’s role launching a fleet of large instances would have caught the DN42 bill in minutes rather than on the credit card statement. - Set hard ceilings. Organization-level policies that block expensive instance families, plus service quotas the agent’s role can’t raise, put a cap on how big the blob can grow. - Budget the agent itself. Step limits, timeouts and a stated cost budget in the agent’s instructions. Yes, that’s a soft control remember, a code freeze in a prompt didn’t stop Replit , but it’s nearly free and it potentially catches the dumb cases. I recently tried adding a cost check to a pre-commit hook https://blog.localstack.cloud/managing-cloud-costs-cloudburn/ using CloudBurn, an open-source AWS cost policy engine whose scan mode analyzes Terraform and CloudFormation without needing any cloud credentials. With some help from Claude, I wrote a hook that scans both the last commit’s infrastructure code and the working copy, compares the findings by rule, resource and location, and blocks the commit if anything new shows up. It’s not a complete solution. Only part of the rule set applies to scanning infrastructure code, anyone can skip a hook with --no-verify , and a shell script is no replacement for a real FinOps practice. But it’s a simple way to give developers, and agents, cost feedback while they’re still making the change, and the same idea works on any cloud with whatever policy tool fits. No single layer solves this. Each one hopefully catches something the others miss. The takeaways I’d like you to have are: 1. Treat agents like a new hire. If you wouldn’t give it to a human on their first day, don’t give it to the agent. 2. Default to containers. Agents build and test against local, containerized environments. Real cloud is human-gated. 3. Bring back the boring safeguards. Least privilege, change freezes enforced by permissions not prompts , confirmations on destructive commands and code review all still work. Push back, politely, when adoption targets make them feel optional. 4. See the cost before the invoice. Shift cost checks left and alert on actions, not just spend. Agents can be incredibly productive. I’m not arguing we slow them down. I’m arguing we give them the same guardrails we’ve used for humans for as long as I can remember, along with a sandbox where they can work quickly without the risk. Let’s keep the horror stories to the campfire and out of the cloud. - Vibe coding service Replit deleted user’s production database https://www.theregister.com/2025/07/21/replit saastr vibe coding incident/ , The Register - Google’s Antigravity AI deleted a developer’s drive and then apologized https://www.techradar.com/ai-platforms-assistants/googles-antigravity-ai-deleted-a-developers-drive-and-then-apologized , TechRadar - Amazon AI coding agent hacked to inject data wiping commands https://bleepingcomputer.com/news/security/amazon-ai-coding-agent-hacked-to-inject-data-wiping-commands , BleepingComputer - 13-hour AWS outage reportedly caused by Amazon’s own AI tools https://engadget.com/ai/13-hour-aws-outage-reportedly-caused-by-amazons-own-ai-tools-170930190.html , Engadget - AWS blames user error, not AI, for cloud outage https://www.computing.co.uk/news/2026/ai/aws-blames-user-error-not-ai , Computing - AI agent deletes startup’s database in 9 seconds https://decrypt.co/365897/ai-agent-deletes-startup-database-9-seconds-founder-says , Decrypt - AI agents with cloud credentials are outrunning billing guardrails https://www.infoq.com/news/2026/07/ai-agents-billing-guardrails/ , InfoQ - State of AI and cloud development 2026 https://localstack.cloud/state-of-ai-cloud-development-2026 , LocalStack and Dimensional Research - State of Agentic Infrastructure 2026 https://www.pulumi.com/state-of-agentic-infrastructure/ , Pulumi - Managing cloud costs with CloudBurn https://blog.localstack.cloud/managing-cloud-costs-cloudburn/ , LocalStack blog