{"slug": "delegating-to-ai-means-governing-the-environment", "title": "Delegating to AI Means Governing the Environment", "summary": "An engineer argues that as AI agents write more code, developers must shift from reviewing the writing process to verifying results against contracts. The author warns that rules in prompts are requests, not guarantees, citing an experiment where an agent deleted files using rsync --delete after rm was removed.", "body_md": "In the [previous article](https://dev.to/manufosela/the-shifting-role-of-the-software-engineer-4ph8), I argued that AI isn't simply changing the tools we use to develop software, but shifting our work to a new level of abstraction. In this one, I want to address the problem that immediately follows: if we're going to write less and less code directly and agents are going to produce an increasingly larger part of it, how the hell do we know whether what they code is actually right?\n\nBecause the answer obviously can't be “trust the AI, it's very smart”.\n\nEven though I personally develop code with AI today with practically no review, I don't blindly trust AI. Just as I don't blindly trust an engineer on my team. I don't even blindly trust myself.\n\nBlind trust is a security hole. And not blindly trusting someone doesn't mean distrusting them, it means having mechanisms to prevent their mistakes, or mine, from causing problems.\n\nThat's why we've spent decades building mechanisms and methodologies around software development to detect, and avoid as much as possible, our mistakes. XP. Scrum. Tests. Code reviews. Pair Programming. CI. Static analysis. Permissions. Observability. Environments. Containers. Auditing...\n\nThe question, therefore, shouldn't be whether we can trust an AI. The question should be **what system do we need to build so we can use it without needing to blindly trust it?**\n\nOne of the first objections is usually that if you ask it the same thing twice, it generates two different pieces of code.\n\nTrue.\n\nBut if you give the same task to two different programmers, or to the same programmer with enough time in between, we'll very probably get two different implementations too, depending on the complexity of what we're asking.\n\nAnd if we've never required two developers to produce exactly the same code, why do we expect AI to produce exactly the same code from the same request?\n\nIsn't it enough for the result to satisfy the requested requirements?\n\nThat it does what it's supposed to do. That it passes all kinds of tests. That it satisfies the acceptance criteria. That it meets security requirements. That it respects certain constraints.\n\nThe problem appears when we keep evaluating the **writing process** while this new level of abstraction increasingly forces us to evaluate the **result against a contract**.\n\nAnd when I say contract, I'm not talking about a huge *spec* telling the agent how it has to build every single thing. I'm talking about the conditions the result must satisfy and having mechanisms that can verify them.\n\nWe haven't reached compiler-level reliability yet, and I don't know whether we ever will, or even whether we need to. What is happening is that agents are becoming more capable and we're giving them more and more autonomy.\n\nAnd the more autonomous they become, the less sense it makes to think we'll be able to control them by constantly looking over their shoulder.\n\nThis is, for me, one of the important parts.\n\nA rule written only in a prompt isn't a rule. It's a request.\n\nYou can have an `AGENTS.md`\n\n, a skill or whatever you want telling it never to do a certain thing. It helps enormously. I use them, obviously. But we've already seen that the agent forgets them more often than we'd like, sometimes interprets them incorrectly, can even lose that information among the ever-growing amount of context it's handling, or simply finds another way to achieve what it was trying to do.\n\nAnd that last one isn't theoretical.\n\nWhile experimenting with all this, I configured a mini PC with Ubuntu to run a model's CLI. I removed the `rm`\n\ncommand from the system so it wouldn't be able to delete anything.\n\nThen one day I discovered that it had deleted files. How? I couldn't believe it. I asked, and it explained that since `rm`\n\ndidn't work, it had used `rsync --delete`\n\n.\n\nI was surprised and scared at the same time.\n\nI had prevented it from using `rm`\n\n. And even though I had explicitly told it never to delete a file without my approval, at one point it realized it had made a mistake and needed to rebuild a structure, so it decided it needed to delete things. It interpreted my prohibition as meaning it shouldn't delete anything critical and that, given the context it had, it could decide that deleting those files wasn't going to cause any problem. After all, it was code that hadn't existed before and that it had created itself, right?\n\nIt disobeyed an instruction in order to achieve its goal.\n\nThat's why important restrictions have to live outside the agent, in mechanisms it doesn't control. Hooks. CI. Permissions. Tests. Mutation tests. Policies. Gates...\n\nIf I want every change to have tests, I can tell it to do TDD. But I also have to verify that those tests exist, that they run and that they actually pass.\n\nIf I don't want any change to get in without review, I can't let the same agent decide that “this time” the review isn't necessary.\n\nIf I want a security vulnerability to block a change, no second agent should be able to casually decide “well, it doesn't look that serious”.\n\nIf I want all work to be traceable, it can't depend on an agent remembering to create a ticket afterwards.\n\nImportant rules have to become mechanisms.\n\nNot good intentions.\n\nThe agent can decide how to solve the problem. What it shouldn't be able to decide is whether it can bypass the mechanisms that determine whether that solution is acceptable.\n\nAnd here I think we're usually mixing two different problems.\n\nOne is how we **orchestrate agents**. Who writes. Who researches. Who reviews. How we share context. How several agents work in parallel. Which model we choose for each task.\n\nWe can have one writing, another from a different family reviewing, and a third arbitrating when they disagree. We can even launch the same task to several models and then choose between the solutions that satisfy our conditions.\n\nThis already looks much more like building an artificial team than using a programming assistant.\n\nBut there's another different problem: **who governs those agents?**\n\nWho establishes the rules? Who decides which actions are allowed? Who checks that they were actually followed? Who prevents an agent from granting itself an exception?\n\nOrchestration gives capability.\n\nGovernance gives guarantees.\n\nAnd we need both.\n\nFor a while it seemed that the answer to AI development was the prompt engineer.\n\nIt lasted months.\n\nThen the idea emerged that development with agents was going to be about writing better specs. Right?\n\nIf the AI made mistakes, it was because we hadn't specified clearly enough what we wanted. So we started writing increasingly detailed documents explaining which endpoints to create, which classes to use, which structure to follow, which cases to handle...\n\nI think that's also starting to become “the past”.\n\nIf today I tell a good frontier model “build me a REST API to access my recipe database”, and it has access to the repository, the database and the project context, I don't need to explain that I probably need to list recipes, retrieve one, create one, modify one or delete one, or how to do any of that.\n\nIt will explore. It will understand the schema. It will look at how the rest of the project is built, its stack, its style, the conventions we use, even previous conversations we've had, and it can make many of those decisions by itself surprisingly well.\n\nIf I've also told it that we work with TDD, that APIs must be documented and that it should generate a Postman collection, I don't have to repeat that in every request either.\n\nBut, and here we go back to the beginning of the article, telling it something is one thing and guaranteeing that it does it is another.\n\nInstructions guide.\n\nThe environment governs.\n\nThat changes where we put our effort again.\n\nI don't need to translate my need into an almost technical specification first so that the AI can then translate that specification into code.\n\nI can express the intention at the highest possible level, my need, my desire, my idea, nothing more: “I want to access my recipes through a REST API”.\n\nIt's practically the same thing I do as a manager when I ask a senior engineer. I don't have to specify the details, because they know how we work, they know the conventions, the rules and the context.\n\nThe detailed *spec* was, in a way, another layer of human translation between the idea and the code. And AI is starting to eat that layer too.\n\nWhat doesn't disappear, and actually becomes more important, are the rules, the constraints and the definition of what really matters.\n\nAre we moving to RDD? Rules-Driven Development? Are they going to start selling us courses on how to govern agents?\n\nOf course 😝\n\nJokes aside, one thing is telling the agent exactly how to build the solution, which is something I don't do with a senior engineer either, and another very different thing is giving it enough context to make good decisions. A senior engineer can work from a high-level request because they know the domain, understand the product, know how the team works and have references they can use to compare their decisions. And if they don't know something, they know where to look or who to ask.\n\nThe same should happen with an agent. If it has access to the domain, the code, the documentation, previous decisions, examples and the team's rules, I increasingly need less to turn my need into a detailed implementation spec.\n\nWhat is still necessary is defining the conditions that any solution must satisfy. Those can live in team agreements, a playbook, a developer kit, ADRs, architecture rules, security criteria or verification mechanisms.\n\nFewer specs doesn't mean less context. It means stopping using the spec as a manual translation layer between the need and the implementation.\n\nThat's where we're getting to, if we're not already there.\n\nLess implementation specification. More intention. More available context. More mechanisms around it.\n\nBut here's another problem I don't think we've solved yet.\n\nGuardrails work, but many times they work **after the agent has already smashed into them**.\n\nIt already happens to me.\n\nOne of my rules may say that I don't want PRs larger than 200 lines.\n\nThe agent knows it.\n\nAnd even so, far too often, it ends up trying to create a PR with more than 200 lines.\n\nThe hook blocks it.\n\nPerfect.\n\nGovernance worked.\n\nBut now the agent has to go back. Redo part of the work. Split it. Maybe create another file. Reorganize imports. Divide the PR. Run things again.\n\nAnd all of that consumes tokens.\n\nA lot of them.\n\nThe final result may comply perfectly with the rules, but we got there in an obviously inefficient way.\n\nThe system is safe, but the agent has been banging its head against the walls until it found the right path.\n\nAnd this happens because the rules that live in the model's context don't have the same weight as the real restrictions that live outside it. The model may “know” that a rule exists and still forget it during execution.\n\nI use tools to try to reduce token consumption, such as RTK or Squeezr, but you're only optimizing one part of the problem.\n\nThe underlying problem is still there.\n\nBuilding better guardrails isn't enough. We need agents to get better at understanding and maintaining those constraints throughout the work so they don't waste time, tokens and money attempting solutions that we already know the system is going to reject.\n\nI think this is one of the areas where companies building models and agents have a huge amount of room for improvement.\n\nBecause fully delegating development doesn't just require the result to be correct.\n\nGetting there also has to be reasonably efficient.\n\nAnd this efficiency problem leads me to something else.\n\nIf we're going to have agents working, reviewing, arbitrating, retrying after hitting a guardrail and handling increasingly large amounts of context, always using the most powerful frontier model for absolutely everything doesn't seem particularly smart either.\n\nOr cheap.\n\nI think companies should seriously start considering running their own models in their infrastructure too.\n\nAnd individuals as well.\n\nNot necessarily to replace Claude, Codex or whatever frontier model happens to be the one next week. But to complement them.\n\nThere are tasks a local model can do well enough. Repetitive tasks. Classification. Context retrieval. Some reviews. Simple generation. Work involving code that, for security reasons, you don't want leaving your infrastructure.\n\nAnd when the task requires more capability, when the local model can't handle it, when it gets stuck or when you need someone to come to the rescue, you escalate to a frontier model.\n\nJust as I wouldn't put my best engineer on absolutely every task in the team simply because they're the best engineer.\n\nWe have to find the balance.\n\nWhich model we use for what. What stays inside. What goes outside. When it's worth paying more and when a smaller model is enough.\n\nAnd this reinforces again the idea that we're not heading towards “using an AI”.\n\nWe're heading towards building **teams of AIs**, each one doing what makes the most sense for it, under a common set of rules that none of them controls.\n\nAfter thinking about it a lot, experimenting and taking a few hits along the way, I more or less know what my ideal framework looks like for developing software with AI without having to blindly trust it, but still having enough governance guarantees.\n\nI want to start by telling the model what I'm trying to do, at the highest level possible. My need, my idea, the problem I want to solve. Sometimes I may even give it a pile of documents that have been written about the idea, for example, a role-playing game with its rules, its objective, its cards... And once I give it that high-level instruction and some details, I want it to generate a first plan.\n\nThat plan I do want to review. Me as an engineer, other engineers if necessary, or even other agents from different families that challenge it, look for problems, suggest alternatives and debate the decisions with it.\n\nOnce the plan is approved, I want the system itself to turn it into an actual working plan. Epics. Tasks. Dependencies. Estimates. Acceptance tests. Everything initially generated by AI, but something we can discuss, correct and complete with it before starting.\n\nIn other words, I don't want to write a huge *spec* myself telling it how to do things. I want to express what I need and discuss with the AI the plan it proposes to achieve it.\n\nAnd once that plan is approved, then yes, let the machinery start.\n\nLet an orchestration of agents pick up the tasks. Let them architect when architecture is needed. Let them code. Let other agents review. Let them debate when they disagree. Let us pit several models against the same problem and run tournaments if it makes sense. Let there be specific security reviews. Let tests, mutation tests, static analysis, SonarQube or SonarCloud, linters and all the tooling we consider necessary run.\n\nBut all of that inside an environment that doesn't depend on the agent remembering our rules.\n\nPre-commit and pre-push hooks. CI gates. Permissions. Real limits. Policies that prevent certain things from happening even if the agent wants to do them.\n\nThe agent can decide how to get there.\n\nThe system decides where it is allowed to go.\n\nAnd I want incremental updates on every merge to `main`\n\nto keep the codebase RAG up to date, plus a full reindex on every release so that when we deploy software, the system's own knowledge is refreshed as well. And also update the wiki and the documentation so that the next agent that arrives isn't working from how the system looked three months ago, but from how it looks now.\n\nAnd that RAG, that wiki, that task board, all of it should be inspectable by the engineer, so it can be corrected or understood.\n\nFor me, that's the framework.\n\nHuman intention. A generated and debated plan. Decomposed and verifiable work. Teams of specialized agents executing it. Other agents reviewing and challenging it. Deterministic tools checking it. External guardrails preventing them from stepping outside the rules. And knowledge updated after every change.\n\nNot trusting that the agent will do it right.\n\n**Building an environment where it's very difficult for it to do it wrong and even harder for an error to reach production.**\n\nAnd even then, it will make mistakes.\n\nThat's why we need to build a governance system for our agents.\n\nToday's AI still isn't that modern compiler we trust without practically ever looking at what it generates.\n\nIt's more like the compiler from decades ago. Good enough to change the profession. Not good enough to blindly trust.\n\nAnd that's precisely why I think we're making a mistake when we conclude that we should keep manually reviewing every line.\n\nThe answer should be something else: **learning to build better systems around it.**\n\nSystems that limit. And verify. And block. That allow mistakes without causing a disaster.\n\nBut also systems that are efficient and don't force the agent to discover the rules by smashing into them again and again.\n\nBecause the goal shouldn't be to make AI never make mistakes.\n\nWe can't even achieve that with humans. I wish.\n\nThe goal should be that when it makes a mistake, the system detects it before it matters. And, if possible, before it burns through a few million tokens discovering it.\n\nAnd even if we solved all of this, we'd still be left with probably the hardest problem.\n\nGetting an engineer to actually want to work this way.\n\nBecause changing the tools is one thing.\n\nAnd telling someone who has spent years programming that part of what defines their profession, and something they probably love doing, is no longer going to be done by them is something very different.\n\nAnd CI doesn't fix that.\n\nThis new path has to be led.", "url": "https://wpnews.pro/news/delegating-to-ai-means-governing-the-environment", "canonical_source": "https://dev.to/manufosela/delegating-to-ai-means-governing-the-environment-ef3", "published_at": "2026-08-13 15:54:17+00:00", "updated_at": "2026-08-13 16:20:15.665239+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-safety", "developer-tools"], "entities": ["manufosela", "Ubuntu", "rsync"], "alternates": {"html": "https://wpnews.pro/news/delegating-to-ai-means-governing-the-environment", "markdown": "https://wpnews.pro/news/delegating-to-ai-means-governing-the-environment.md", "text": "https://wpnews.pro/news/delegating-to-ai-means-governing-the-environment.txt", "jsonld": "https://wpnews.pro/news/delegating-to-ai-means-governing-the-environment.jsonld"}}