# AI Systems Need Coordination Planes, Not Just Control Planes > Source: > Published: 2026-07-11 09:31:13+00:00 In the previous article, *The Future of AI Is Stateful Infrastructure*, I argued that memory, retrieval, workflow checkpoints, execution lineage, and KV cache layers increasingly shape enterprise AI system behavior long after inference completes. If inference generates outputs and state shapes behavior, the next architectural question becomes how distributed execution is coordinated once intelligent work crosses system boundaries. As organizations move from isolated AI applications toward distributed AI systems that span retrieval, memory, agents, policies, and humans, a second challenge emerges. State rarely remains confined to a single system component. Execution context moves across retrieval systems, memory stores, workflow engines, models, agents, APIs, approval processes, and human decision-makers. Preserving information is only the beginning of the challenge. Organizations must also coordinate how work progresses as responsibility, authority, and operational control move between participants across those boundaries. This distinction appears in production when the cluster, inference endpoint, and model serving infrastructure are all healthy. Yet, a workflow still fails because a retrieval result is stale, an external tool is unavailable, an approval remains pending, or a policy engine blocks progress mid-execution. The infrastructure functions correctly, yet the overall system still fails to complete its intended objective. The underlying reason is straightforward. These platforms are beginning to resemble distributed operational systems rather than traditional software applications. Control planes coordinate infrastructure, but distributed AI systems require mechanisms that coordinate execution itself. That responsibility may become one of the most important architectural concerns of the next generation of enterprise AI platforms. Enterprise AI systems increasingly encounter operational failures that cannot be resolved through infrastructure orchestration alone. The challenge is no longer deploying intelligent services. It is governing execution as work moves across organizational, technical, and human boundaries. The cloud-native ecosystem owes much of its success to the emergence of control planes. Kubernetes demonstrated the power of declarative infrastructure management through desired-state reconciliation. Operators define an intended outcome, and the platform continuously works to align actual conditions with that objective. Workloads are scheduled, failed components are replaced, resources are provisioned, and cluster state remains synchronized. Similar patterns appear throughout modern infrastructure platforms. Service mesh control planes coordinate network behavior. Infrastructure automation platforms coordinate provisioning activities. Distributed schedulers coordinate resource placement. Control planes are remarkably effective because infrastructure management is fundamentally a reconciliation problem. The system understands what should exist and continuously works to move the environment toward that desired outcome. Over the past decade, as Kubernetes and declarative platforms saw widespread enterprise adoption, this architectural model has enabled organizations to operate highly complex distributed environments with a degree of consistency that would have been difficult to achieve through manual operations alone. Modern AI platforms introduce a different architectural challenge. Provisioning resources does not ensure successful execution. A control plane can determine where a model runs, but it cannot determine whether an approval workflow should advance, whether a delegated task should escalate, whether an external tool response is trustworthy, or whether a policy exception requires human review. These concerns involve execution decisions rather than infrastructure state. Consider a common enterprise security workflow. An AI assistant identifies a potential vulnerability, gathers supporting evidence, creates a remediation recommendation, and proposes a production change. Kubernetes can ensure that the remediation service remains available and that the underlying infrastructure continues operating correctly. What it cannot determine is whether the recommendation should be approved, escalated, deferred, or rejected. Those decisions require interaction among workflow systems, governance processes, policy controls, and human participants. This pattern is especially visible in regulated financial services and security workflows, where approval gates and audit requirements are non-negotiable regardless of infrastructure health. The distinction may appear subtle at first, but it becomes more important as AI systems scale across more participants. Control planes reconcile infrastructure resources and operational state. Enterprise AI systems require architectural mechanisms that coordinate workflow progression across distributed participants. Architects should distinguish infrastructure coordination from workflow coordination during system design. Treating both responsibilities as a single concern often creates operational complexity as systems scale. When evaluating AI platform investments, assess not only model hosting and infrastructure capabilities but also how the platform coordinates approvals, recovery, escalation, delegation, and governance processes. Traditional enterprise applications typically follow predictable execution paths. Requests arrive, business logic executes, databases are queried, and responses are returned. Modern AI systems execute through collections of specialized services. A single workflow may span retrieval systems, memory services, orchestration engines, policy platforms, external APIs, multiple models, autonomous agents, and human reviewers. No single component owns the entire process. Instead, workflow progresses through a chain of distributed decisions. A customer-support workflow provides a useful example. A customer submits a request. A model classifies intent. A retrieval platform gathers historical information. An agent invokes an account-management API. A policy engine evaluates authorization requirements. A reviewer approves an exception. The workflow then continues. Every transition introduces a coordination boundary where information, authority, or responsibility must move between participants. As organizations adopt multi-agent architectures, coordination boundaries multiply rapidly. Agents invoke tools, delegate work, collaborate with other agents, revisit earlier decisions, and escalate requests when confidence thresholds are not met. The resulting architecture increasingly resembles a coordination graph rather than a traditional application. Reliability now depends less on the performance of individual components than on the system’s ability to coordinate decisions across every participant involved in execution. As workflows expand across models, retrieval systems, policy engines, tools, and human reviewers, interaction quality largely determines overall system behavior. This shift has important implications for architects and security teams. Distributed execution creates new decision dependencies. Those dependencies introduce operational relationships that become as important as the behavior of any individual model. Modern AI workflows resemble interconnected decision networks rather than linear software execution paths. Map coordination boundaries explicitly. Retrieval systems, agents, approval processes, policy evaluations, external tools, and human participants should be treated as workflow participants rather than implementation details. Expect operational complexity to grow faster than model complexity. Many production failures emerge from workflow interactions rather than model behavior. Distributed systems have long required mechanisms that coordinate work across independent participants. Transaction managers coordinate database operations. Message brokers coordinate communication. Workflow engines coordinate business processes. Distributed schedulers coordinate resource allocation. These technologies emerged because infrastructure coordination alone was insufficient once execution became distributed across independently managed participants. Enterprise AI systems require similar coordination capabilities, but across a broader scope and a wider range of participants. These responsibilities extend beyond software components to include models, policies, autonomous agents, external services, and accountable human participants. This transition is already visible across the enterprise AI ecosystem. Kubernetes reconciles infrastructure state, Temporal and Argo Workflows provide durable workflow execution, LangGraph coordinates multi-step agent reasoning, OpenTelemetry captures execution telemetry and lineage, Open Policy Agent enforces policy decisions, and event-driven architectures synchronize interactions among services, tools, agents, and human participants. Although each technology addresses a different aspect of distributed execution, together they illustrate a broader architectural trend: capabilities that were once embedded within individual applications are becoming shared operational services. As organizations deploy larger numbers of AI workloads, they increasingly require these shared capabilities to manage retries, timeouts, checkpoints, delegation, approvals, escalation, compensation, recovery, and lineage consistently across the platform. These capabilities are no longer implementation details. They increasingly distinguish experimental AI workflows from production-grade operational systems and demonstrate how execution coordination is evolving into a first-class architectural responsibility. This progression mirrors a familiar pattern in distributed systems history. Observability evolved from application logging into a dedicated operational discipline. Security evolved from isolated controls into platform-wide architecture. As organizations deploy more autonomous workflows, coordination appears to be following a similar path, becoming an architectural differentiator that often determines whether complex AI systems operate reliably in production. The next challenge in enterprise AI is no longer managing infrastructure alone. It is governing how decisions move safely and predictably across distributed execution environments spanning systems, reviewers, policies, agents, and execution engines. As AI workloads become increasingly autonomous, the emphasis shifts from generating intelligence to governing how intelligence progresses through operational systems. Architects should design coordination capabilities as shared platform services rather than embedding execution logic within individual applications. Durable workflow management, checkpointing, delegation, escalation, recovery, approvals, and execution visibility become more valuable when they can be applied consistently across many AI workloads rather than reimplemented for each solution. Evaluate AI platforms on their ability to coordinate complex execution, not solely on model performance or infrastructure scalability. As organizations deploy more autonomous and multi-agent workloads, capabilities such as durable execution, governance, recovery, operational visibility, and human oversight increasingly determine whether AI systems remain reliable, manageable, and suitable for enterprise use. Ask vendors to demonstrate workflow replay, checkpoint recovery, delegated approvals, and policy enforcement under realistic failure scenarios rather than isolated model benchmarks. Partial failures most clearly expose the tight coupling between preserved state and workflow progression. The previous article established that state enables continuity by preserving context, execution progress, and operational history. Execution coordination determines how that continuity is used. Consider a software delivery workflow involving an AI assistant, vulnerability scanner, policy platform, approval process, and deployment pipeline. The AI assistant identifies a vulnerability. The scanner validates the finding. The policy engine approves remediation. A reviewer authorizes deployment. The deployment service then becomes unavailable. At that moment, the central question is not whether information has been preserved. The question is how execution should proceed. Should execution restart from the beginning? Should execution resume from a checkpoint? Should previously completed approvals remain valid? Should compensation actions be triggered? Should escalation procedures begin? Infrastructure alone cannot answer these questions because they involve decisions about execution progression rather than resource availability. Distributed systems have spent decades developing mechanisms such as retries, checkpoints, replay, rollback, durable execution, and compensation to address partial failure. These patterns are closely related to long-running orchestration and saga-style recovery models, where systems must coordinate progress, compensate for completed steps, and preserve consistency without relying on a single distributed transaction. These environments often require the same architectural capabilities. As workflows span larger numbers of systems and participants, recovery becomes less dependent on individual components and more dependent on the mechanisms that govern how workflow progression resumes after disruption. Recovering distributed AI workflows is fundamentally a coordination problem rather than an infrastructure problem. Design recovery paths before designing optimization strategies. Reliable continuation frequently determines operational success more than execution speed. Evaluate AI initiatives based on recovery characteristics as well as functional capabilities. Failure handling often determines long-term operational viability. Require vendors to demonstrate recovery from partial failures, not simply successful execution under ideal conditions. Predictions about fully autonomous enterprise environments often underestimate an important reality. Many business processes involve authority rather than automation. Decision authority frequently remains with accountable human participants because organizations continue to require accountable decision-makers for financial approvals, procurement decisions, legal reviews, compliance assessments, security exceptions, and governance processes. As a result, enterprise AI workflows commonly resemble hybrid systems. Human decision-makers become first-class participants within distributed execution workflows alongside models, agents, tools, and policy engines. A workflow may begin with an agent, transition through policy evaluation, escalate to a reviewer, return to automation, and ultimately execute an operational action. Each transition introduces requirements involving accountability, escalation, delegation, timing constraints, auditability, and governance. In many enterprise environments, the most difficult challenge is not machine-to-machine interaction. It is managing how authority moves between automated and human participants while maintaining visibility, compliance, and operational control. AI systems may accelerate these workflows, but they rarely eliminate accountability requirements. Many regulated industries already require accountable human approval for financial transactions, healthcare decisions, security exceptions, and policy overrides. Those requirements are unlikely to disappear. Modern AI platforms are not eliminating human participation. They are increasingly orchestrating human participation. Design human interaction as a first-class workflow component rather than an exception path. Approval chains, escalation processes, and accountability requirements should be modeled explicitly. Expect governance requirements to increase as AI systems become more capable. Operational oversight should scale alongside automation. As AI environments incorporate more participants, the challenge extends beyond coordinating individual workflows. Organizations need a consistent way to coordinate execution across agents, people, policy engines, external services, and operational platforms. At that point, coordination becomes a shared platform responsibility rather than an application-specific concern. As enterprise AI systems become more distributed, architectural responsibilities are separating into distinct operational layers. This evolution resembles earlier transitions in distributed computing, where infrastructure platforms eventually separated control planes from data planes because each served a fundamentally different purpose. A useful way to understand this transition is through the Enterprise AI Operational Architecture Model, which organizes the major architectural responsibilities emerging as AI systems become increasingly distributed. The remaining articles in this series explore these responsibilities individually, beginning with coordination and continuing through trust, observability, policy, authorization, autonomy, and containment. Although illustrated as distinct architectural responsibilities, these capabilities operate together as an interconnected operational system. As enterprise AI systems become increasingly distributed, responsibilities that were once embedded within individual applications are emerging as shared platform concerns. State infrastructure preserves context. Coordination governs distributed execution. Trust enables delegated identities, authority, and decisions to propagate safely across organizational and technical boundaries. Observability explains behavior, policy constrains decisions, authorization governs execution rights, and containment limits the impact of failures or unexpected behavior. Together, these responsibilities form the Enterprise AI Operational Architecture Model, a conceptual framework for understanding how distributed AI systems preserve context, coordinate execution, propagate authority, explain behavior, constrain decisions, govern execution, and limit failure. Within this operational architecture, a coordination plane is an architectural layer responsible for governing distributed execution across workflows, participants, and operational boundaries. Figure 6 illustrates how these architectural responsibilities combine to govern distributed execution across enterprise AI systems. Unlike a control plane, which reconciles infrastructure state, a coordination plane governs workflow progression across agents, humans, tools, policies, and services. Its core responsibilities include workflow progression, checkpoint management, escalation, delegation, approvals, recovery, compensation, lineage tracking, and orchestration visibility. The term should not be interpreted as a formal industry category. Instead, it describes an emerging architectural responsibility that is becoming more visible across enterprise AI systems. Different organizations will implement this responsibility through workflow engines, event-driven architectures, orchestration frameworks, custom platforms, or combinations of these approaches. Implementation details will necessarily vary across organizations, platforms, and operational requirements. The underlying architectural trend remains consistent. As enterprise AI systems scale across distributed environments, coordinated execution is becoming a first-class operational concern. Begin treating execution coordination as a shared platform capability rather than embedding progression logic within individual applications. Durable coordination services become foundational as AI workloads scale across teams, environments, and business processes. Architects evaluating enterprise AI platforms should explicitly assess whether execution coordination is treated as a foundational platform capability rather than an application-specific concern. Useful evaluation questions include: Organizations investing heavily in AI should anticipate growing requirements for governance, workflow visibility, recovery management, and operational oversight. Investment decisions should include explicit evaluation of execution coordination capabilities alongside model quality and infrastructure maturity. Enterprise AI requires architects to separate two distinct operational responsibilities. Infrastructure platforms reconcile infrastructure state. Coordination mechanisms govern execution across workflows, services, agents, policies, and people. Treating these responsibilities independently improves reliability, governance, and operational resilience. The history of distributed systems is filled with examples of organizations discovering that infrastructure alone does not solve workflow management challenges. Enterprise AI is beginning to expose similar architectural limitations. Models continue to improve, platforms continue to mature, and state infrastructure continues to expand. Yet many of the hardest operational challenges arise while advancing work across distributed workflows spanning systems, services, policies, and people. The previous article argued that state is becoming a first-class architectural concern because it shapes system behavior. This article extends that foundation by arguing that once execution becomes distributed, it must also be coordinated. Preserving context is necessary, but it is not sufficient. Organizations must also determine how workflow progression continues, how failures are handled, how authority moves between participants, and how governance requirements are enforced throughout the process. State creates operational complexity, while coordination advances distributed execution. Once execution spans agents, workflows, people, tools, and services, organizations must answer an equally important question: who should be trusted to act, delegate authority, approve decisions, or execute work? Just as Kubernetes established a consistent operational model for infrastructure management, enterprise AI increasingly requires equally mature architectural mechanisms for governing distributed execution. Those mechanisms need not be implemented identically across platforms, but they should consistently coordinate execution across workflows, organizational boundaries, and accountable participants. State preserves context. Coordination advances execution. Trust determines whether delegated actions can be relied upon. Together, these architectural responsibilities increasingly form the operational foundation of enterprise AI platforms, governing how intelligent work is executed, supervised, delegated, recovered, and ultimately trusted across the enterprise. This article uses the term coordination plane as a conceptual architectural model rather than a formally standardized industry term. Workflow orchestration, durable execution, business process management, and distributed coordination are well-established disciplines. However, there is currently no universally accepted definition of a coordination plane within enterprise AI or cloud-native architecture. Throughout this article, the term describes an emerging architectural responsibility rather than a specific technology or product category. Not every AI workload requires sophisticated coordination capabilities. Many inference services remain relatively simple request-response systems that execute successfully without durable workflow management, approval chains, distributed recovery, or human participation. The architectural considerations discussed here become more important as AI systems span multiple models, retrieval platforms, memory services, autonomous agents, external tools, policy engines, and human decision-makers. Throughout this article, coordination refers to managing execution across distributed participants rather than provisioning infrastructure resources. Infrastructure control planes reconcile desired system state by scheduling workloads, provisioning resources, and maintaining operational consistency. Coordination mechanisms govern how execution progresses across workflows by managing approvals, delegation, escalation, retries, recovery, compensation, and operational continuity. Existing infrastructure platforms also coordinate certain execution activities within their own operational domains, such as workload scheduling, rolling updates, and controller reconciliation. The coordination responsibilities described in this article extend beyond infrastructure lifecycle management to encompass distributed business workflows, agent interactions, human approvals, policy decisions, and cross-system execution. The implementation examples discussed throughout this article illustrate common architectural patterns rather than prescribing specific technologies. Organizations may implement these capabilities through workflow orchestration platforms, durable execution frameworks, event-driven architectures, custom application services, or combinations of these approaches. The underlying architectural responsibility remains more important than the implementation approach. Human participation continues to play an essential role in many enterprise AI environments. Financial approvals, legal reviews, procurement decisions, security exceptions, compliance assessments, and governance processes frequently require accountable human decision-makers regardless of advances in AI capabilities. The article therefore focuses on coordination across both automated and human participants rather than assuming fully autonomous enterprise systems. This article intentionally distinguishes coordination from trust. Coordination determines how distributed work progresses. Trust determines whether identities, delegated authority, decisions, and actions should be accepted across distributed execution environments. That relationship becomes the focus of the next article in this series. Future enterprise AI platforms will likely implement these responsibilities through different architectural patterns. The intent of this article is to describe the responsibilities themselves rather than prescribe a single implementation model. This article builds upon concepts introduced in the earlier installments of this series. Traditional perimeter-oriented security models become less effective as AI systems become more distributed and runtime-oriented. Platform capabilities increasingly become the primary security boundary. AI-assisted vulnerability discovery changes defensive assumptions but reinforces the importance of resilient platforms, containment, operational governance, and defense-in-depth. Inference infrastructure is becoming increasingly stateful as KV cache locality influences routing, scheduling, latency, throughput, and infrastructure design. [https://medium.com/@mhockelberg/kv-cache-reuse-is-quietly-reshaping-llm-inference-architecture](https://medium.com/@mhockelberg/kv-cache-reuse-is-quietly-reshaping-llm-inference-architecture) Inference runtimes increasingly resemble distributed operational environments where identity, authorization, orchestration, workload trust, and runtime governance become primary security concerns. Enterprise AI systems increasingly resemble distributed systems whose primary architectural challenges involve orchestration, scheduling, telemetry propagation, workflow coordination, state management, trust propagation, and operational control. [https://medium.com/@mhockelberg/ai-systems-are-quietly-becoming-distributed-systems-3d644b75b3d](https://medium.com/@mhockelberg/ai-systems-are-quietly-becoming-distributed-systems-3d644b75b3d) State increasingly shapes the behavior of enterprise AI systems through memory, retrieval, workflow checkpoints, execution lineage, vector stores, conversation history, and KV cache infrastructure. This article established the foundation that state creates the conditions that make coordination difficult. [https://medium.com/@mhockelberg/the-future-of-ai-is-stateful-infrastructure-322139b7e493](https://medium.com/@mhockelberg/the-future-of-ai-is-stateful-infrastructure-322139b7e493) Kleppmann, Martin. *Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems.* O’Reilly Media, 2017. Lamport, Leslie. “Time, Clocks, and the Ordering of Events in a Distributed System.” *Communications of the ACM*, Vol. 21, №7, 1978. [https://lamport.azurewebsites.net/pubs/time-clocks.pdf](https://lamport.azurewebsites.net/pubs/time-clocks.pdf) Dean, Jeffrey, and Sanjay Ghemawat. “MapReduce: Simplified Data Processing on Large Clusters.” *Proceedings of the Sixth Symposium on Operating Systems Design and Implementation (OSDI)*, 2004. [https://research.google/pubs/mapreduce-simplified-data-processing-on-large-clusters/](https://research.google/pubs/mapreduce-simplified-data-processing-on-large-clusters/) Verma, Abhishek, et al. “Large-scale Cluster Management at Google with Borg.” *Proceedings of the Tenth European Conference on Computer Systems (EuroSys)*, 2015. [https://research.google/pubs/large-scale-cluster-management-at-google-with-borg/](https://research.google/pubs/large-scale-cluster-management-at-google-with-borg/) Gray, Jim, and Andreas Reuter. *Transaction Processing: Concepts and Techniques.* Morgan Kaufmann, 1992. [https://www.sciencedirect.com/book/9781558601901/transaction-processing](https://www.sciencedirect.com/book/9781558601901/transaction-processing) The Kubernetes Authors. *Kubernetes Documentation.* Cloud Native Computing Foundation. *Cloud Native Landscape.* Istio Authors. *Istio Documentation.* https://istio.io/ Google Cloud. *Architecture Center.* [https://cloud.google.com/architecture](https://cloud.google.com/architecture) Helland, Pat. *Life Beyond Distributed Transactions: An Apostate’s Opinion.* [https://queue.acm.org/detail.cfm?id=3025012](https://queue.acm.org/detail.cfm?id=3025012) Garcia-Molina, Hector, and Kenneth Salem. “Sagas.” *Proceedings of the 1987 ACM SIGMOD International Conference on Management of Data.* [https://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf](https://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf) Harel, David. “Statecharts: A Visual Formalism for Complex Systems.” *Science of Computer Programming*, Vol. 8, №3, June 1987, pp. 231–274. [https://doi.org/10.1016/0167-6423(87)90035-9](https://doi.org/10.1016/0167-6423(87)90035-9) Temporal Technologies. *Temporal Documentation.* Cadence Project. *Cadence Documentation.* The Argo Project. *Argo Workflows Documentation.* [https://argo-workflows.readthedocs.io/](https://argo-workflows.readthedocs.io/) LangGraph Documentation. [https://langchain-ai.github.io/langgraph/](https://langchain-ai.github.io/langgraph/) Ray Project Documentation. KServe Documentation. [https://kserve.github.io/website/](https://kserve.github.io/website/) Model Context Protocol Specification. [https://modelcontextprotocol.io/](https://modelcontextprotocol.io/) Beyer, Betsy, Chris Jones, Jennifer Petoff, and Niall Richard Murphy, editors. *Site Reliability Engineering: How Google Runs Production Systems.* O’Reilly Media. Dean, Jeffrey, and Luiz André Barroso. “The Tail at Scale.” *Communications of the ACM*, Vol. 56, №2, 2013. [https://research.google/pubs/the-tail-at-scale/](https://research.google/pubs/the-tail-at-scale/) OpenTelemetry Documentation. [https://opentelemetry.io/docs/](https://opentelemetry.io/docs/) SPIFFE Project. SPIRE Documentation. [https://spiffe.io/docs/latest/spire-about/](https://spiffe.io/docs/latest/spire-about/) Open Policy Agent Documentation. [https://www.openpolicyagent.org/docs/](https://www.openpolicyagent.org/docs/) Kyverno Documentation. [AI Systems Need Coordination Planes, Not Just Control Planes](https://pub.towardsai.net/ai-systems-need-coordination-planes-not-just-control-planes-fd10aeb93372) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.