The first meeting of the new agent platform team is going well. The whiteboard already shows the whole answer: a gateway for identity, a sandbox for isolation, an orchestration layer for the agents themselves. Everyone at the table can defend their box, and everyone is right. Forty minutes go to token exchange and network egress, and the conversation is genuinely good.
Then someone asks a small question. What will the agent actually do first? Which workflow, in which system, and who checks the result when it gets things wrong?
The room goes quiet. Not because anyone is incompetent, but because nobody present has ever done, or owned, the work being automated.
That meeting is a composite. Every piece of it is real, and I have sat in more versions of it this year than I want to count. The silence is the most expensive sound in enterprise AI, because the platform that ships six months from now was decided right there, and its most important failure had a seat at the table. The seat was empty.
Let me be precise about what this piece claims, because the lazy version is false. Plenty of agent projects die on technology: the data is a mess, the model is unreliable at the task, the system that matters has no API and never will. This piece is not about those. It is about the other class, the pilots that work perfectly in the demo and die at the question “whose credentials does this thing run on”, and the platforms that pass every review and still produce agents nobody trusts with real work. For that class, the cause is not in the stack. It was in the room, and a programmer named Melvin Conway explained why in 1968: “Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.”
Melvin Conway, “How Do Committees Invent?”, 1968
The agent version is sharper: your platform inherits the blind spots of the meeting that designed it.
every expert in the room is right #
Three kinds of people are selling you an agent platform right now, and it matters that all three are telling the truth.
The security story: agents are identities, and ungoverned identities are how companies burn down. You need to know who the agent is and what it is entitled to, and you need one place to see and revoke all of it. Every word of that is correct.
The infrastructure story: agents are workloads, and untrusted workloads belong in isolation. Sandboxes, egress controls, resource limits, kill switches. Blast radius is a real concept and limiting it is real engineering. Every word of that is correct too.
The harness story: the interesting behaviour lives in the loop, in how the agent plans, calls tools, recovers, and asks for help. Whoever owns the harness owns the quality of the outcome. Also correct.
These are not caricatures, and the people who carry these stories are not fools. These are perspectives, not badges: plenty of security engineers write agents, and one person often carries two of these stories at once. The error is never in any layer’s claim. The error is in the silent suffix each story carries: “and that is the whole problem.” All three decks in your inbox can be true at once, and your pilot can still die, because the thing that kills it does not live in any of their layers.
the damage travels through doors you built #
Here is the mechanism.
An agent that causes damage in production almost never does it by breaking something. It does it through channels you authorised: an approved API, a valid token, a sanctioned scope, a tool it was given on purpose. The sandbox does not need to be escaped, because the actions that matter were never inside it. They travel out through doors you built, using keys you cut. And every door in that list exists because a piece of work needed it.
This problem was named before most of your stack existed. In 1988, Norm Hardy described what he called the confused deputy: a compiler that held legitimate permission to write files in its home directory, took the name of the billing file stored there as a debugging output path from a user, and obediently overwrote the billing records. Nothing was breached. Every permission checked out. A program with authority it legitimately held was steered by input it trusted, and that was enough. An agent is that deputy at machine speed, taking input from everywhere.
Simon Willison gave the modern version a name in 2025: the lethal trifecta. An agent becomes robbable the moment three things are true at once:
- it can read private data
- it takes in content an attacker can influence
- it has any channel that leads back out
Nothing needs to break. The agent can be robbed while every part of the stack behaves exactly as designed. Three legitimate capabilities, individually approved, jointly fatal.
Now look closely at what those three legs are made of. Which data the agent can read, what content reaches it, which channels lead back out: the gateway and the sandbox enforce that list, but they do not write it. It is written by the work you gave the agent to do. The trifecta is an application-layer fact wearing a security costume, and the only person who can walk a room through its legs for a real workflow is the person who owns that workflow.
Watch what this does to the guards-only room. In this class of incident, every property the security and infrastructure chairs measure reports green. Sandbox intact. Tokens valid. Every call authorised. Outcome wrong. Their controls are not failing. They are measuring the wrong object, because “was this action permitted” is a different question from “was this action right”, and only one of those questions can be answered by someone who has never seen the work.
the bill for the empty chair #
I used to think of this as safe pairs and a dangerous one, some combinations of perspectives you could trust and one you could not. That framing is wrong. The honest version is simpler: whichever chair was empty on the day the platform was decided, that is the failure you have already bought. You just have not received the bill yet. Read each row as one sentence: this chair was empty, so this is what goes wrong, and this is when you find out.
| the empty chair | what goes wrong | when you find out |
|---|---|---|
| infrastructure | Outages, latency, a cloud bill that becomes a meeting. | Immediately. Loud failures generate their own urgency, so they get fixed. |
| security | The audit, the pen test, the review that kills two use cases a week before launch. | Late and angry. But you do find out. |
| the work-owner | The agent does the wrong work correctly, through authorised channels. | You do not. No dashboard was ever built to notice. |
Credit where due on the middle row: the reviewers were right. Given what they were shown, a vague agent with broad grants, refusal was the correct call. The failure happened earlier, in what they were shown.
The bottom row is the one to sit with. The agent resolves the ticket that should have been escalated, approves the edge case a human would have caught, files the thing in the wrong place with perfect confidence, and nobody built an alarm for it, because the people who built the alarms did not know what wrong looks like for this work.
The agent does the wrong work correctly. That is the failure no dashboard was designed to catch.
One distinction makes this usable. Platform rooms make two kinds of decisions, and they need different chairs.
| the decision | examples | the room it needs |
|---|---|---|
| substrate: where agents run | sandboxes, egress, secrets, kill switches | Security and infrastructure. This room is already right, and the paved roads it builds are genuinely valuable. |
| capability: what an agent may do | which workflow, which actions, how much access | The work-owner, no exceptions. |
Most platform teams never notice the moment they cross from the first row into the second.
And to be fair in the other direction: a room made only of work-owners fails too. It ships a demo that cannot be deployed, secured, or explained to an auditor. That failure is real. It is also cheaper, because it dies loudly before production instead of silently inside it.
You can recognise a badly composed room by what it produces. Four patterns, each with a tell you can test this week.
| the pattern | what it looks like | the tell |
|---|---|---|
| the shell game | Containment keeps improving around an agent whose damage never touches the walls. Hardening continues because it is measurable, not because it addresses the risk. | Across the whole agent programme, the roadmap is all nouns of containment: gateway, sandbox, proxy, vault. Nothing anywhere on it names a workflow. |
| the single keyring | Every credential the agent might ever need hangs on one identity, one vault role, one broker that hands it anything it asks for. One successful manipulation converts into total authority. | One service account whose scope list reads like an org-wide access review, with keys to tools, data, and security systems on the same ring. |
| the proxy operator | The work-owner’s chair is technically filled, by someone who has never done or closely watched the work. Requirements arrive as use-case titles, and access gets defined by guesswork. | Ask the room for the ten specific actions the agent will perform and the exceptions that make each one hard. Watch who answers, and watch who looks at their laptop. |
| the prompt as policy | The authorisation rules live in the system prompt, so the layer being defended against is also the layer doing the defending. An instruction is not a control. | Any part of the incident-response story that begins with “the agent is instructed not to”. |
A nuance the keyring row deserves, because it is regularly sold as the solution, not least by gateway-shaped products. One place to see and revoke everything is a control plane, and you want it. One principal that can reach everything is a target, and you do not.
the application layer is still the king #
Networking people settled this argument decades ago. The OSI model has seven layers, the top one is the application, and the other six exist to deliver it. Nobody ships a packet for the packet’s own sake. Your agent stack has the same shape. Infrastructure decides where things run. Security decides who may enter. But execution, the thing that actually happens to a customer, an invoice, a record, lives in the application layer, and the application layer is still the king of execution. Every layer below exists to keep the king safe. None of them can know what the king is supposed to do.
And the king has one problem the other layers do not: it cannot send one person to the meeting. Infrastructure sends someone who talks subnets, machines and operating systems. Security sends someone who talks identity and policy. The application layer is not one team with one badge. It is the work itself, plus a crowd of builders scattered across the org chart, and nobody in that crowd has platform in their title. So the layer needs three things sent on its behalf: a chair, a rule, and a test.
The chair belongs to the operator, the work-owner from the tables above: the person who can narrate the workflow the agent is replacing, end to end, exceptions included, and who is personally embarrassed when the agent gets it wrong. That is the whole definition. No job title, no team, no requirement to be an engineer. In most enterprises the operator sits two levels below the people currently in the meeting. If they can walk the room through the work, they are the king’s voice at the table. If they cannot, the chair is still empty, whoever is sitting in it.
The rule is the two-chair rule. No agent platform decision gets made by one perspective alone. Substrate decisions, where it runs and how it is contained, need the guards. Capability decisions, what it may do, need the operator in the room, no exceptions. And two chairs is a floor, not a governance model: every decision still has one named owner, and each chair vetoes only inside its own scope. The operator does not veto egress policy. The guards do not write the verb list.
The test is the Tuesday test, and you can run it tomorrow. Before your next capability decision, ask the room: can anyone here narrate one real task this agent will complete next Tuesday, end to end? That means naming:
- the workflow it sits in
- the systems it touches
- the tool and access each action needs
- what a plausible-but-wrong outcome looks like
- who checks that result today, when a human produces it
If someone can, carry on, whatever their job title is. The chair is filled. If nobody can, you have found the empty chair, and the honest move is to adjourn the capability decision, not the meeting, until someone who owns that work is in the room. That is the entire framework: a chair, a rule, a ten-minute test. No new product, no new team, no transformation programme. The expensive way to learn this lesson is silently, eighteen months from now, in production. The cheap way is an extra chair.
A quick translation, for when you buy help instead of building the room: listen to the first question a forward deployed engineer (FDE) or a consultancy asks. If it is about your stack, they are filling a chair you already have. If it is about your work, which workflow, who owns it, what wrong looks like, they might be worth the rate.
And for the platform team you already have: nothing here says to rebuild it. Keep it on substrate decisions, where it is exactly the right room, and borrow the operator into capability decisions one use case at a time, a chair per decision, not a new headcount.
The reorganisations keep missing a job. Whatever AI does to your org chart, the agent’s application layer still needs a home and a keeper: somewhere the tools live, the integrations run, and the glue between the model and your systems of record gets looked after. That job belongs to the same lineage that kept your Tomcat and JBoss servers alive through every previous wave, the unglamorous middle of the stack where the business actually runs.
Your security vendor cannot host the king. Your infrastructure provider cannot. No governance product can. Hosting the application layer was your job before agents, and it is still your job after them.
the architecture a good room chooses #
Here is what happens when the operator’s chair is actually filled. The vague grant, “the agent needs access to the CRM”, becomes a verb list: look up this, update that, never touch the other thing. The verb list becomes a short list of named tools, each carrying no more access than its verb needs. Least privilege has been preached since the seventies and skipped every time for the same reason: the units were too coarse to enforce. A tool call is the first unit of authority narrow enough to make it stick. And this time the security review has something in front of it that can be approved rather than merely refused. That is the payoff the guards get from this arrangement: requests precise enough to say yes to. The room that killed your last two use cases was not wrong. It was under-informed, and this is the mechanism that informs it.
Notice where that architecture came from: not from a product, from the work. The gateway enforces the rule and the sandbox contains the blast, but neither can know what the rule should be, because application knowledge cannot be bought from a layer below. Be politely suspicious of any pitch that ends with you not having to think about the workflow.
We have run this experiment before. Coding agents were supposed to make application engineering redundant, and what they made redundant was the typing. The knowing, what to build, what correct looks like, what must never happen, became the whole job. Agents in production are the same trade at higher stakes: execution gets cheap, and judgement about execution stays exactly where it always lived.
So here is the concrete version of that judgement. Before anything gets bought or built, count the application bench in the conversation:
- the people who write your MCP servers and tools
- the ones who have deployed microservices and carried the pager for them
- the integration and middleware crowd who glued your systems together the last time round
- the backend engineers who own the services the agent will call
- the workflow and rules-engine people who encoded your decisions before agents could read them
- the ones who build the screens where a human approves what an agent proposes
That bench is where the unglamorous words live: manageability, upgrade path, support rota, integration debt, the whole hidden cost of owning anything. Letting the guards win a round and buying the shell can be the right call. But if nobody from that bench was in the conversation, you have not avoided the application layer’s bill. You have quietly assigned it to them, and they will end up building an unfunded product of their own around whatever you bought.
The principle underneath is old and it fits in one line.
An agent’s power should be the sum of the tools in its hand, not the identities on its badge.
And even that line is a people claim in disguise. The operator names the verbs. The bench turns them into tools. Bind authority to those tools and even a fully authorised agent cannot act outside them, and keep the keys apart while you are at it: tools, data, security systems and platform should never hang on one ring, for the agent or for anyone selling you convenience.
But keep the order of causation straight. Good architecture is what survives a bad day; the room is what decides whether it gets chosen at all. No product, whatever layer it lives in, can substitute for the person who can say “that is not how invoicing works here”.
Which is why the cheapest fix in enterprise AI right now is a meeting invite. Before you approve the platform, before you sign the FDE contract, before the whiteboard fills up with boxes, look around the room and ask who here has done the work this thing is about to do. If the answer is silence, you already know how the pilot ends.
Send the invite.