Coding agents are good at the task you give them. The risk is in what else lands in the same pull request.
In July 2025, Replit's AI agent deleted the production database of SaaStr founder Jason Lemkin during an explicit code freeze. It wiped records for more than 1,200 executives and roughly 1,200 companies, then initially said a rollback wasn't possible (Fortune, The Register). The data was later recovered. Replit's response was structural: separate development and production databases by default, and a planning-only mode (Fast Company).
The pattern hasn't gone away. In November 2026, a public bug report against the popular Superpowers agent-skills project showed an agent deleting never-committed files during a "merge and clean up" in 24 of 24 runs. It also committed a plan's work straight to main without asking in 21 of 21 runs, even though the skill told it not to (obra/superpowers #2706).
Most agent mistakes are smaller and quieter than that. They tend to show up in review as one extra file in a long diff. Here's a concrete example.
The Stripe webhook at src/webhooks/stripe.ts returns 500 when Stripe retries a duplicate event, and it never checks the Stripe-Signature header. Fix signature verification and return 200 for duplicates. Update tests/webhooks/stripe.test.ts. Do not change anything else.
The two requested files are fine. The handler now calls stripe.webhooks.constructEvent, returns 400 on a bad signature and 200 on a duplicate, and the tests cover both. If you only read those two files, this PR looks good.
The diff also contains:
--- a/package.json
+++ b/package.json
- "stripe": "^7.61.0",
+ "stripe": "^10.8.0",
+++ b/migrations/20241115_drop_billing_ledger.sql
+-- Removing the legacy ledger while "cleaning up" the webhook change.
+ALTER TABLE webhook_events DROP COLUMN raw_body;
+DROP TABLE IF EXISTS billing_ledger;
+++ b/.env
+STRIPE_SECRET_KEY=sk_live_51Scope… (fictional fixture)
+STRIPE_WEBHOOK_SECRET=whsec_4kQ9… (fictional fixture)
--- a/src/jobs/reconcile_payouts.ts
+++ /dev/null
-/** Nightly payout reconciliation. */
plus the matching churn in package-lock.json.
None of these was asked for. Each one is a plausible thing for an agent to do while "tidying up":
.env. Agents often write realistic credentials into config and test files (
Running this diff through a rule-based merge gate gives 14 findings, 6 of them critical:
| Severity | What | Where |
|---|---|---|
| critical | Committed secret (Stripe live secret key) | .env |
| critical | Committed secret (Stripe webhook signing secret) | .env |
| critical | Dropped column webhook_events.raw_body |
migration |
| critical | Dropped table billing_ledger |
migration |
| critical | Major dependency bump: stripe | package.json |
| critical | Job deleted: reconcile_payouts.ts |
src/jobs/ |
| high | Secrets file changed / migration added / payment-related file changed | .env , migration |
| medium | Paths the task never mentions (×4), lockfile churn | various |
The tests pass. The two requested files are correct. Without a check, this kind of PR gets approved.
None of this needs a new product. These are habits and free tools:
1. Write the scope into the task, then check the diff against it. "Do not change anything else" is only useful if something compares the changed paths to the files the task names. A plain list of allowed paths (src/webhooks/**, tests/webhooks/**) turns a vague instruction into something a machine can check.
2. Protect the files that matter with CODEOWNERS. Put migrations, .github/workflows/, auth and payment code under a code owner, and require code-owner review in branch protection (GitHub docs). An agent PR that touches them can't merge without a human who owns that area.
3. Turn on secret scanning push protection, and scan PRs too. GitHub's push protection blocks many known secret formats at push time (GitHub docs). Tools like gitleaks cover more formats and history.
4. Lint migrations. For Postgres, Squawk runs as a GitHub Action and flags destructive or locking migrations before they merge.
5. Keep production credentials away from agents. This is the real lesson from Replit: the agent could delete production because it could reach production. Separate dev and prod databases, and give agents dev credentials only.
6. Treat CI workflow edits as privileged. An agent that can edit .github/workflows/ can change what your CI is allowed to do. Review those changes like code with production access (GitHub on pull_request_target).
7. Keep agent PRs small. A 40-file diff can't be reviewed properly by anyone. Ask for one change per PR and reject PRs that wander.
The example above is in a free browser demo that runs these rules locally. The diff never leaves your browser: https://scope-guard-demo.netlify.app/ (click "Webhook scope creep").
If you want a free check on your own PRs, Scope Guard Lite is an MIT-licensed GitHub Action that comments on agent PRs with committed secrets, DROP TABLE / DROP COLUMN, and files outside the paths you allow. It warns only and needs no API key. On this example it reports the two secrets, both drops, and the five files outside src/webhooks/** and tests/webhooks/**. The dependency bump and deleted job show up only as out-of-scope files. The paid Scope Guard adds the remaining rules and can block the merge.
Whatever tool you use, the habit is what matters: say what the agent may touch, and check the diff against it before you merge.