Imagine a payment service halfway through a migration. A traffic layer can route requests to a backend on a VM or to another in Kubernetes. Both are healthy, and either can handle the current traffic on its own.
During this phase, the Kubernetes workload uses a stop-and-start deployment. The old instance goes offline before its replacement starts. The VM is expected to carry payments through that interruption.
An infrastructure agent prepares to reboot the VM after maintenance. It checks that the Kubernetes backend can keep serving payments. At the same time, a deployment agent prepares its release and checks that the VM can keep serving payments.
Both have permission. Both choose the intended tool. Both inspect live state.
Then they execute at the same time. For a short period, neither backend can serve a payment.
Each plan was acceptable on its own. Each depended on the other plan not happening.
Both agents counted on the same redundancy.
This is a hypothetical scenario. It raises a practical question for anyone building agents that can change production: how does one piece of automation find out that another has already committed to a change that conflicts with its own?
Uber Engineering’s article on its MCP Gateway provides a useful starting point. It describes a platform hosting more than 800 MCP servers and 5,000 tools. A registry manages discovery and ownership; a proxy connects agents to existing services. Tools start disabled and require owner review and enablement. Authorization and sensitive-data redaction are also built into the gateway. [1]
Reading that design made me think about the next decision in an infrastructure workflow. Once an agent has found a tool and has permission to call it, what establishes that this particular change can proceed now?
The proposal here concerns that admission decision. The payment example is separate from Uber’s implementation; their gateway article does not establish what coordination mechanisms exist elsewhere in their systems.
In our example, the infrastructure agent touches the VM and the deployment agent touches Kubernetes. Their credentials can be correctly scoped and their resource locks completely separate. Both changes still affect the same payment service.
That shared condition needs an owner.
The failure resembles write skew in a database: two actors read a condition involving multiple records, update different records, and leave the condition false. PostgreSQL’s transaction-isolation documentation describes related serialization anomalies, where individually successful transactions produce a result that no serial ordering would have produced. [2]
Here, the shared condition is simple: at least one backend must remain able to serve payments. Checking it independently gives both agents a reason to proceed.
Kubernetes already has a related mechanism in PodDisruptionBudgets. A PDB can limit eligible voluntary pod evictions, but it does not coordinate maintenance on an external VM. Kubernetes also documents that Deployment and StatefulSet rolling updates are not constrained by PDBs; their rollout behavior is configured separately. [3]
Maintenance coordination predates AI. A script and a human operator could cause the same collision. Agents make the problem relevant to more workflows because they can assemble actions across tools and systems. The execution platform needs a way to express the application condition those actions share.
For this payment service, the admission rule can be stated precisely:
After accounting for work already admitted and the proposed change, at least one verified backend capable of carrying the current load must remain available.
The first admitted change reserves the service’s allowance for disruption. That reservation starts before the action, while both backends may still look healthy.
The following sequence is illustrative:
Checking for conflicts and recording the reservation must be atomic. Two separate calls would recreate the original race.
One implementation could use a PostgreSQL row keyed by tenant and service identity. A short transaction locks that row, reads outstanding reservations, evaluates the proposed change against the relevant evidence, and records the reservation and an execution request before committing. Every admission and reservation update for that service must follow the same coordination rule. PostgreSQL’s row-locking facilities provide the serialization point for those transactions. [4]
The database lock ends when the transaction commits. The durable reservation remains while the external work proceeds. A reboot may take minutes; there is no reason to hold the database transaction open for that duration.
This simple design serializes disruptive work within one service. A change affecting several services would need to acquire all its required reservations together, or acquire none and wait. Otherwise it could protect one application while leaving another exposed.
The core race is small enough to reproduce without a cloud account. This Python model has two workers, one for each backend. A barrier makes both admission decisions happen before either worker disrupts its target.
Health observations deliberately remain unchanged during the actions. That models a monitoring delay and makes the role of reservations visible.
from concurrent.futures import ThreadPoolExecutorfrom threading import Barrier, Lockclass Service: def __init__(self): self.observed = {"vm", "k8s"} self.serving = {"vm", "k8s"} self.reserved = set() self.lock = Lock() def admit(self, target, reserve=True): with self.lock: if not (self.observed - self.reserved - {target}): return False if reserve: self.reserved.add(target) return True def disrupt(self, target): with self.lock: self.serving.remove(target) # Health observations deliberately lag behind the action. def reconcile(self, target, verified_serving): with self.lock: if verified_serving: self.serving.add(target) self.observed.add(target) self.reserved.discard(target)def concurrent_changes(reserve): service, checked = Service(), Barrier(2) def worker(target): admitted = service.admit(target, reserve) checked.wait() # Both admission decisions precede either action. if admitted: service.disrupt(target) return admitted with ThreadPoolExecutor(max_workers=2) as pool: admitted = list(pool.map(worker, ("vm", "k8s"))) return sum(admitted), len(service.serving)for reserve in (False, True): admitted, remaining = concurrent_changes(reserve) print(f"reserve={reserve}: admitted={admitted}, serving={remaining}")service = Service()assert service.admit("vm")service.disrupt("vm")# Model a worker that stops here; the coordinator retains its state.service.reconcile("vm", verified_serving=False)print("after worker loss:", service.admit("k8s"))service.reconcile("vm", verified_serving=True)print("after verified recovery:", service.admit("k8s"))
reserve=False: admitted=2, serving=0reserve=True: admitted=1, serving=1after worker loss: Falseafter verified recovery: True
The first run admits both changes and leaves no serving backend. The second admits one change and leaves the other backend available. Notice that both runs protect the check with a lock. The reservation is what makes the first decision visible to the next one.
The final case models a worker stopping after it disrupts the VM. The coordinator retains the reservation and refuses the Kubernetes change until a verified recovery is supplied.
This is a model of one interleaving, with one request per target and enough capacity on either backend. It does not test a distributed coordinator, persistence across process crashes, or real health checks. The recovery boolean represents evidence that a production system would have to collect. Its purpose is to expose the admission error and show the state that prevents it.
For two known backends, explicit service membership is enough. A shared reservation can solve this example without a graph database.
The harder question is how to establish the affected service when the target is a VM process, a Kubernetes workload, a load-balancer target, or a shared dependency. Those relationships change during migrations and deployments. Keeping them in a manually maintained list becomes an operational responsibility of its own.
This is where a compiled system graph can help. By compiled, I mean a model that resolves identities across collected sources and turns their metadata and observations into explicit relationships. It connects a VM to its processes, workloads to their serving endpoints, and those endpoints to the applications they support. Each relationship carries its origin and the time it was last supported by evidence.
The graph helps determine which service conditions a proposed change could affect. The coordinator uses those conditions to evaluate reservations. The executor enforces the admitted plan and gathers the evidence needed to complete it.
A graph can be accurate about the present while two approved plans are about to make it obsolete. Admission therefore needs both the dependency model and the record of work already committed.
Service membership can also change while a reservation is active. I would tie each reservation to a stable service identity within its tenant and record the dependency revision and evidence used at admission. If a migration changes the service’s backends or critical dependencies, the coordinator should reevaluate active plans before admitting more work. Updating the graph must not make an existing reservation disappear or allow a conflicting change to enter under a new identity. If the required serving condition can no longer be verified, further disruptive steps should . The reservation remains active until recovery is verified or reconciliation establishes that releasing it is safe.
For the payment example, a structural simulation can remove the proposed backend from the model, account for existing reservations, and check whether a serving path remains. That is a useful, bounded result. It does not establish that the remaining backend has enough capacity, that its credentials work, or that its payment requests will succeed. Those conclusions require measurements and checks beyond topology.
Missing evidence needs equal attention. If a collector stops reporting and the graph shows no consumers for a VM, the absence of edges cannot establish that restarting it is harmless. Essential relationships need freshness and provenance checks. When those checks fail, the execution policy needs to hold the change or request further evidence.
Otherwise we have given the agent a more convincing way to be wrong.
Consider the instruction, “Restart the VM.”
An approver would reasonably expect it to refer to a particular VM and plan, under the conditions presented for review. If the Kubernetes backend fails ten minutes later, that approval should not silently authorize the restart in a different situation.
I would bind approval to the target, the plan, its permitted impact, and the conditions that justified it. Immediately before a consequential step, the executor should refresh the relevant evidence. A material change should invalidate the admission or send the plan back for review.
Permission can remain valid for months. The evidence supporting a particular action may remain valid for seconds.
The person supervising the work should be able to understand the constraint without reconstructing tool calls. For the waiting deployment, a useful status message would be:
The deployment is waiting for VM maintenance to finish. That VM must be serving payments again before this deployment can start. I will recheck both backends and confirm that the existing approval still applies.
The reservation only coordinates execution paths that honor it. Human operators, deployment pipelines, autoscalers, and scheduled jobs need to participate where possible. Changes outside that boundary still require detection and a response when assumptions fail.
Reserving the VM’s disruption allowance also cannot prevent the Kubernetes backend from failing independently. The guarantee is narrower: cooperating controllers will not deliberately consume the same redundancy twice. Unexpected failures still require recovery.
Suppose the VM reboot completes but the payment process fails to start. Releasing the reservation at that moment would permit the deployment to remove the only functioning backend.
Completion must therefore include the checks the next change depends on: the process is running, the traffic path is usable, and the required application checks pass. The definition should be agreed before execution begins.
An executor crash creates another unresolved condition. A timeout tells us that a worker has gone quiet. It does not tell us whether the reboot happened or whether the service recovered. The reservation should remain while a reconciler inspects the actual state and takes responsibility for the unfinished work.
Production execution also needs to handle a worker that resumes after its authority has been replaced. Where supported, the mutation boundary should reject commands carrying an old execution generation. If the downstream system cannot enforce that, takeover must account for an old command that may still be in flight. A fresh owner alone does not make retrying an action safe.
The change record becomes useful here. It holds the intended outcome, the approved plan, the evidence used for admission, the reservation, and the observed result. It gives the next executor enough information to investigate an unfinished operation without treating a successful API response as proof that the service is ready.
For teams building infrastructure agents, the payment example makes a useful execution test. Let two workers propose individually acceptable changes against the same initial state. Verify that their shared service condition survives, including when the first worker disappears before reporting completion.
A system trusted to carry work through production has to account for the work happening around it. In this example, the second agent can complete its task only after the first backend has recovered.
That is part of doing the job correctly.
Disclosure: I work on AI agents and Agentic coworkers, infrastructure engineering, and service-boundary compilation (Knowledge Graph) at Empirik AI (formerly ManifestIT), and have filed patent applications related to this work. The payment-service scenario is hypothetical, and the Python example is a simplified illustration of the coordination problem.
[1] A. Srivastava et al., Designing MCP Gateway Uber’s MCP Management Platform (2026), Uber Engineering.
[2] PostgreSQL Global Development Group, Transaction Isolation, PostgreSQL documentation. Accessed October 11, 2026.
[3] Kubernetes contributors, Disruptions, Kubernetes documentation. Accessed October 11, 2026.
[4] PostgreSQL Global Development Group, Explicit Locking, PostgreSQL documentation. Accessed October 11, 2026.
The Next AI Outage Could Begin With Two Approved Changes was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.