{"slug": "the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem", "title": "The Agent Stack Is Filling In. The Judgment Layer Is Still an Open Problem", "summary": "Cloudflare has open-sourced Cloudflare OS, its internal platform for AI agents, which separates organizational context, skills, MCP-connected tools, isolated environments, and gatekeepers. However, a recent paper highlights a concurrency problem where agents can make decisions based on stale state, leading to violations. This points to an architectural shift where the industry is improving agent access and execution, but the judgment layer—deciding whether an action is justified under organizational rules—remains an open problem.", "body_md": "Over the last few days I have been looking at several developments around enterprise AI agents that initially seemed unrelated.\n\n[Cloudflare open-sourced](https://blog.cloudflare.com/cloudflare-os/) its internal agent workspace platform.\n\n[Researchers published work](https://arxiv.org/html/2608.02764v1) on what happens when an agent authorizes an action using state that changes before execution.\n\n[Enterprise AI leaders are starting to talk](https://www.reuters.com/commentary/breakingviews/corporate-ai-is-still-chasing-theoretical-benefits-2026-08-07/) less about how many agents they can deploy and more about whether those agents are producing measurable value.\n\nThere is also growing infrastructure around agent identity, tool access, observability, security, and runtime isolation.\n\nTaken separately, these are different stories.\n\nTogether, I think they point to a larger architectural shift.\n\nThe ecosystem is getting much better at answering:\n\nWhat can this agent access and execute?\n\nBut another question remains less well defined:\n\nGiven the available evidence, what should this organization actually decide?\n\nThose are not the same question.\n\nCloudflare recently open-sourced Cloudflare OS, the platform it uses internally for AI agents.\n\nThe architecture is interesting because the model is not treated as the entire system.\n\nThere is organizational context. There are skills. There are MCP-connected tools. There are isolated environments in which agents operate. And there are Gatekeepers that help control what an agent can do.\n\nThis seems like a reasonable direction for enterprise agents.\n\nAs agents move closer to production systems, companies need much more than an LLM and a collection of tools.\n\nBut imagine using this architecture for vendor onboarding.\n\nAn employee asks an agent to onboard Acme Corp.\n\nThe employee has permission to make the request.\n\nThe agent has permission to call the vendor-management API.\n\nThe request passes the security gateway.\n\nDoes that mean the vendor should be onboarded?\n\nNot necessarily.\n\nThe decision might depend on facts such as sanctions screening, tax documentation, annual contract value, country of registration, personal-data handling, or an internal risk score.\n\nThere may also be exceptions.\n\nA sanctions match could be a hard stop.\n\nA high contract value might require committee review.\n\nMissing evidence might mean that neither approval nor rejection is correct yet.\n\nThe authorization layer can answer:\n\nIs this agent allowed to perform this operation?\n\nThe business still needs an answer to:\n\nUnder our rules, is this operation justified?\n\nThat is the boundary I find interesting.\n\nA recent paper on stateful governance for concurrent agent systems describes another version of the problem.\n\nSuppose two agents share the same purchasing budget.\n\nAgent A checks the available budget and determines that a $30,000 purchase is allowed.\n\nBefore Agent A executes the purchase, Agent B spends $40,000.\n\nThe state has changed.\n\nAgent A may have made the correct decision using the facts it had at evaluation time. Executing that decision later can still violate the organization's rules.\n\nThis is easy to miss when we think about agent governance only as a policy check performed immediately before a tool call.\n\nFor deterministic judgment systems, I usually describe one useful property as:\n\n```\nsame facts + same judgment definition → same disposition\n```\n\nThat gives us reproducibility.\n\nBut the concurrency problem suggests another requirement:\n\n```\nthe facts that justified the disposition\nmust still be valid when the action commits\n```\n\nThose are different guarantees.\n\nA production workflow might therefore need to look more like this:\n\n```\nAcquire facts\n     ↓\nEvaluate judgment\n     ↓\nRecord disposition + relevant state\n     ↓\nValidate state / re-evaluate if necessary\n     ↓\nExecute\n```\n\nThis is an area I want to experiment with because it sits directly between decision semantics and runtime enforcement.\n\nAnother signal comes from enterprise adoption.\n\nOrganizations are starting to discover that deploying agents is easier than understanding the full operational footprint afterward.\n\nOne recent report described a company where the CIO believed roughly 40 agents were operating internally. An audit reportedly found more than 400.\n\nThat number caught my attention because it changes the governance problem.\n\nIf you have three experimental agents, reviewing their prompts and workflows manually may be perfectly reasonable.\n\nIf you have hundreds of agents making decisions across finance, procurement, customer operations, HR, and IT, it becomes much harder to answer basic questions consistently.\n\nWhich decisions are these agents allowed to make?\n\nWhat evidence must be available?\n\nWhich exceptions apply?\n\nWhen must the agent abstain or escalate?\n\nWho approved the decision logic?\n\nWhich version was active when a particular action happened?\n\nThese questions do not disappear because the underlying model becomes more capable.\n\nSome of them become more important.\n\nThis is the problem I am exploring with the open-source Judgment Pack Specification.\n\nA Judgment Pack represents organizational decision logic as a separate artifact.\n\nThe goal is not to replace the agent.\n\nThe agent can still gather information, interact with users, use tools, and reason where reasoning is useful.\n\nThe difference is that the criteria governing a consequential business decision do not have to live entirely inside the agent's prompt.\n\nFor a vendor-onboarding decision, for example, a pack might define the required evidence, thresholds, exceptions, treatment of missing facts, and possible outcomes.\n\nThe agent supplies facts.\n\nThe evaluator applies the reviewed judgment definition.\n\nThe result can then be used by another layer responsible for execution.\n\nConceptually:\n\n```\nAgent\n  ↓\nEvidence acquisition\n  ↓\nJudgment Pack\n  ↓\nDisposition + evidence\n  ↓\nGateway / policy enforcement\n  ↓\nBusiness system\n```\n\nI think this distinction becomes clearer as the rest of the agent stack matures.\n\nMCP is solving connectivity.\n\nAgent platforms are solving orchestration and execution.\n\nSkills are helping models use systems correctly.\n\nSecurity layers are controlling identity and access.\n\nObservability platforms are helping companies understand what agents actually did.\n\nNone of those necessarily owns the organization's definition of a correct business decision.\n\nA deterministic lookup does not need one.\n\nA simple permission check probably belongs in an authorization system.\n\nA low-risk conversational suggestion may be perfectly appropriate to leave to the model.\n\nThe interesting cases are decisions where the organization expects consistent treatment of evidence, rules, exceptions, uncertainty, and escalation.\n\nThat is where putting the judgment into a reviewable artifact starts to become useful.\n\nI am particularly interested in testing this boundary against real agent platforms rather than defining it only on paper.\n\nCloudflare OS looks like one possible integration environment.\n\nConcurrent state changes are another area I want to test.\n\nAnd the larger question is whether this separation still makes sense when applied to real production workflows rather than clean examples.\n\nThat is the part I am trying to learn now.\n\nIf you are building enterprise agents and have a workflow where the difficult part is not tool access but deciding **whether the action is justified**, I would be interested in hearing what that decision looks like.", "url": "https://wpnews.pro/news/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem", "canonical_source": "https://dev.to/kikashy/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem-40g2", "published_at": "2026-08-09 17:29:57+00:00", "updated_at": "2026-08-09 17:48:27.020586+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-safety", "ai-infrastructure", "ai-research"], "entities": ["Cloudflare", "Cloudflare OS", "Acme Corp"], "alternates": {"html": "https://wpnews.pro/news/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem", "markdown": "https://wpnews.pro/news/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem.md", "text": "https://wpnews.pro/news/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem.txt", "jsonld": "https://wpnews.pro/news/the-agent-stack-is-filling-in-the-judgment-layer-is-still-an-open-problem.jsonld"}}