{"slug": "how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail", "title": "How My AI Agents' Mistakes Become Permanent Rules (And Why I Want Them to Fail)", "summary": "A solo developer running SaaS products for German golf clubs has built a system where AI agents' mistakes crystallize into permanent guard rules. After an agent pushed a hotfix directly to production at 2 AM, the developer implemented a shell-level guard that blocks direct pushes to main, and over two months, 211 rules have emerged from 1,448 autonomous agent sessions. The system uses learnings.md files to capture failures and promote high-quality learnings into enforceable bash scripts that cannot be overridden by prompts.", "body_md": "I run SaaS products for German golf clubs. Solo founder. 85 containers, 24 databases, one server. No team.\n\nMy AI agents handle deployments, database migrations, code reviews, content pipelines, and infrastructure monitoring. They run autonomously, 24/7. And they make mistakes.\n\nAt 2 AM on a Tuesday, one of my agents pushed a hotfix directly to the production branch. No review. No tests. No human in the loop. The app stayed up by luck. I woke up to a commit I never approved on a branch that should be protected.\n\nThat morning, I wrote a guard. Two months later, 211 rules have crystallized from 1,448 autonomous agent sessions. Not one of them was planned. Every single rule started as a failure.\n\nMost AI agent setups have no memory. Every session starts from zero. Your agent breaks something on Monday, learns nothing, and breaks the same thing on Wednesday.\n\nI tried prompt engineering. I tried longer system prompts. I tried telling the agent \"never push to main.\" It worked until it did not. Prompts are suggestions. Agents interpret them. Sometimes they interpret them wrong.\n\nWhat I needed was not better instructions. I needed a system that physically prevents the same mistake from happening twice.\n\nEvery skill in my system has a `learnings.md`\n\nfile. When an agent runs a skill and something goes wrong (or right, in a surprising way), the learning gets captured with context, a rule, and a quality score from 1 to 5.\n\n```\n## 2026-06-10: Agent pushed directly to main at 02:14\n\n**Context:** Autonomous deploy task, develop branch was \nbehind main, agent decided to \"shortcut\" the workflow\n**Learning:** Agents will find creative workarounds when \nthe intended path has friction\n**Rule:** Block git push to main/master/production at the \nshell level, not the prompt level\n**Score:** 5\n**Runs:** 1\n```\n\nOn subsequent sessions, when that skill runs again, the agent reads `learnings.md`\n\nfirst. If the learning helps, the run counter goes up. If it does not apply, it stays.\n\nWhen a learning reaches a quality score of 4 or higher AND has proven useful across 3 or more sessions, it crystallizes. It graduates from a soft note in a markdown file to a permanent guard rule: a bash script that fires on every command, every file edit, or every session end.\n\nThe learning stops being advice. It becomes law.\n\nHere is the actual `main_push_guard.sh`\n\nthat crystallized from that 2 AM incident:\n\n``` bash\n#!/bin/bash\n# Guard: main_push_guard\n# Origin: Crystallized from learning 2026-06-10 \n# (agent pushed to main at 02:14)\n\nhook_main_push_guard() {\n  # Only look at push commands\n  echo \"$CMD_SHELL\" | grep -qE \\\n    'git[^;&|]*push([[:space:]]|$)' || return 0\n\n  # Direct push to protected branches?\n  if echo \"$CMD_SHELL\" | grep -qE \\\n    'push[^;&|]*(main|master|production)'; then\n\n    # Log the block for audit trail\n    echo \"| $(date +%Y-%m-%d\\ %H:%M) \\\n    | Main-Push-Guard | blocked \\\n    | $SESSION_ID |\" \\\n      >> /opt/audit/gate-audit-log.md\n\n    deny \"MAIN-PUSH-GUARD: Direct pushes to main \\\n    are blocked. Use: gh pr create\"\n  fi\n}\n```\n\nNo prompt can override this. No creative agent reasoning can work around it. The `deny`\n\nfunction kills the command before it executes. It fires on every `git push`\n\nacross every session, every agent, every skill.\n\nThe guard that came from PII leaking into container logs works the same way. An agent dumped a database query result that contained email addresses into stdout. The PII scanner now runs on every command output, checking for email patterns, phone numbers, and German address formats. It does not ask the agent to be careful. It blocks the output.\n\nThe loop itself is enforced by a guard. After every skill execution, `learnings_loop_guard.sh`\n\nfires:\n\n```\nhook_learnings_loop_guard() {\n  # Was a skill just executed?\n  echo \"$CMD\" | grep -qE 'skills/.*SKILL\\.md' \\\n    || return 0\n\n  local skill=$(basename \"$skill_dir\")\n  local learnings=\"/root/.claude/skills/$skill/learnings.md\"\n\n  if [ -f \"$learnings\" ]; then\n    add_context \"LEARNINGS-LOOP: Skill '$skill' detected. \\\n    REQUIRED: (1) Read learnings.md BEFORE execution, \\\n    (2) AFTER: increment runs if learning helped, \\\n    add new learning only for NEW insights, \\\n    (3) Check crystallization (Score>=4 + Runs>=3).\"\n  fi\n}\n```\n\nThe agent is reminded every single time. Read the learnings. Update the counters. Check if anything is ready to crystallize. This is not optional. The guard injects the instruction into the agent's context.\n\nThe Crystallization Loop is the R in what I call GRIP:\n\n**Guards** prevent known failures. 176 guard files fire on every shell command, every file edit, every session end. 96% of all rules are enforced by hooks, not prompts.\n\n**Resilient** means the system learns from failures it could not prevent. The Crystallization Loop turns agent mistakes into permanent guards. 211 rules crystallized so far. None were written proactively.\n\n**Isolated** means every customer has their own database. One breach affects one club. Deletion is DROP DATABASE. No shared risk.\n\n**Public** means full transparency. Every guard block is logged. Every AI feature is documented. Every model choice is traceable.\n\nThe feedback loop between G and R is what makes it work. Guards prevent known problems. When unknown problems slip through, they become learnings. When learnings prove stable, they become new guards. The system gets stricter with every session.\n\nMy agents have an 88% success rate across 1,448 autonomous sessions. That means 12% still fail.\n\nAnd that is fine.\n\nThose failures are the fuel. Every failed session is a potential new learning. Every learning that proves itself becomes a permanent rule. If the success rate ever hit 100%, the Crystallization Loop would stop producing new guards. The system would stop getting better.\n\nI do not optimize for zero failures. I optimize for zero repeated failures.\n\nThe difference matters. A system that never fails is fragile because it was never tested. A system that fails, captures the failure, and makes it structurally impossible to repeat is antifragile. It gets stronger under stress.\n\n61 skills in the system. Each one has its own learnings file. 73 active learnings are sitting in various stages of the loop right now. Some will crystallize next week. Some will fade because they were too specific to one situation.\n\n232 cron jobs run daily. 17,812 knowledge files in the vault. The agents operate in this environment around the clock, and every interaction with the system is a chance to discover a new edge case that no human would have anticipated.\n\nI did not design most of this. I designed the loop. The loop designed the rules.\n\nYou do not need 85 containers to start. You need three things:\n\n`learnings.md`\n\nfile next to your agent configurationStart with the prompt version. When you get tired of agents ignoring the prompt, graduate to shell hooks. That is exactly the path I took.\n\nThe book covers the full system: the GRIP framework, the guard architecture, the Crystallization Loop, and how to build autonomous agent operations as a solo founder.\n\nGet the book: Paperback (24.99 USD) [https://amazon.com/dp/B0HDMVKRMG](https://amazon.com/dp/B0HDMVKRMG) | E-Book (9.99 USD) [https://amazon.com/dp/B0HDMK7QJ1](https://amazon.com/dp/B0HDMK7QJ1)", "url": "https://wpnews.pro/news/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail", "canonical_source": "https://dev.to/frederikvonderheyden/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail-4moi", "published_at": "2026-08-14 21:44:36+00:00", "updated_at": "2026-08-14 22:25:04.716600+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "developer-tools", "mlops"], "entities": ["German golf clubs"], "alternates": {"html": "https://wpnews.pro/news/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail", "markdown": "https://wpnews.pro/news/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail.md", "text": "https://wpnews.pro/news/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail.txt", "jsonld": "https://wpnews.pro/news/how-my-ai-agents-mistakes-become-permanent-rules-and-why-i-want-them-to-fail.jsonld"}}