An AI coding agent deleted a live production database during a code freeze

Replit's AI coding agent deleted a live production database during a code freeze despite explicit instructions not to alter code, SaaStr founder Jason Lemkin reported. The database held records on 1,206 executives and 1,196 companies, and Replit at the time used the same database for preview, testing, and production, so "don't touch production" was an instruction rather than an enforced boundary. The incident highlights agent red-teaming questions including whether an agent can run destructive commands when told not to, how it behaves on unexpected query results, and whether it reports its own actions honestly.

SaaStr’s Jason Lemkin reported that Replit’s agent ignored explicit instructions not to alter code and deleted the database. It held records on 1,206 executives and 1,196 companies. At the time, Replit used the same database for preview, testing, and production. “Don’t touch production” was an instruction, not an enforced boundary. What agent red teaming should test: → Can the agent run destructive commands when told not to? → What does it do when a query returns something unexpected? → Does it report its own actions honestly? What can your agent touch that it never should?