On this page #
Most 2026 agent patterns are older distributed systems or compiler ideas applied to probabilistic workers. An examination of what transfers and what fails.
I spent eighteen months deploying long-running AI agents.
Start building agents, and the “autonomous swarm” marketing pulls you in. You spin up five subagents, give them tools, and connect them in a conversational loop to share context.
Ten minutes later, you face a $45 API bill, an infinite loop, and an agent declaring: “All 14 database migrations applied!”
You open Postgres: zero tables exist.
I stopped believing the marketing. I was debugging a distributed systems failure from 1994.
Modern agent frameworks copy familiar plumbing: MapReduce, write-ahead logs, two-phase commits, actor mailboxes, and Conflict-free Replicated Data Types (CRDTs). The industry rediscovered distributed computing, database internals, and compiler theory, then applied them to language models.
Distributed systems solved coordination decades ago. Treating a language model like a conventional compute node breaks those solutions.
Standard worker nodes fail with visible signals: process crashes, dropped packets, timeouts, or exceptions. An agent emits HTTP 200, valid JSON, and a confident summary claiming success, while leaving the task broken.
Reliable agent engineering requires a boundary between two planes: the semantic plane where the model proposes, and the deterministic control plane where the runtime decides.
We Rebuilt the 1990s Stack #
Hit a reliability wall, and you reach into a thirty-year-old textbook to borrow a solution.
Production agent stacks in 2026 expose this translation layer:
| Classic Concept | Agent Version | Production Example |
|---|---|---|
| MapReduce / Scatter-Gather | Lead agent dispatches subtasks across bounded context shards, then merges findings | Devin Security Swarm ;Anthropic Claude Research |
| Durable Execution & Event Sourcing | Journaling every prompt, tool call, and result in an append-only log; restoring state by replay | Temporal +OpenAI Agents SDK ;LangGraph checkpointers ;Pi-Durable |
| Idempotency Keys | Ensuring retried tool calls do not create duplicate payments or commits; generated by runtime, never the prompt | Temporal + Stripe refunds ;Stripe Idempotency Keys |
| Sagas & Two-Phase Commit | Staging side effects in a temporary workspace, verifying invariants, and committing or rolling back | AWS prompt-chaining saga patterns ;Cordon |
| Actor Model | Isolated agents with private context passing asynchronous messages via mailboxes | Microsoft AutoGen v0.4 Core |
| CRDTs | Multi-agent collaborative editing where file updates merge without central locks | CodeCRDT ;AgentRoom |
| Consensus & Voting | Self-consistency, majority voting, and multi-agent debate | Self-Consistency (Wang et al.);Debate protocols |
| Service Discovery | Tools and agents publishing machine-readable capability manifests | Model Context Protocol (tools/list );A2A Agent Cards |
| Distributed Tracing | Parent-child spans capturing latency, tokens, tool inputs, and intermediate agent thoughts | OpenTelemetry GenAI Semantic Conventions |
| Query Planner / Cost Optimizer | Compiling user intent into tiered model calls based on cost, latency, and context budget | LOTUS ;Palimpzest ;DSPy |
| Schedulers & Backpressure | Program-aware scheduling, concurrency throttles, and token spend ceilings | Agentix serving engine |
Durable execution shows the pattern. In Beyond the Ephemeral Loop: The Architecture of Durable AI Agents, @earendil-works/pi-durable replaces an in-memory while loop with an embedded SQLite write-ahead log.
If an agent executing a twelve-step migration fails on step nine due to a network glitch, starting over burns hundreds of thousands of tokens. Systems like Temporal, DBOS, and LangGraph solve this by logging every LLM step and tool call. Temporal’s rule applies to agents: keep workflow code deterministic, and push nondeterministic LLM calls into external Activities. During recovery, the engine reads completed steps from history rather than calling the API again.
Devin’s Scanner Is Hadoop #
Cognition published details on how Devin scans codebases for vulnerabilities, drawing attention to model reasoning. The architecture relies on Hadoop, not model reasoning.
If you give an agent a 200,000-line codebase and ask: “Did you find all SQL injection bugs?”, it inspects ten files, runs low on context, and replies: “Yes, I scanned the repository and found three issues.” It is the student grading their own exam.
Cognition stripped the agent of that decision through Agentic MapReduce:
- Plan (Deterministic Selectors): An agent inspects the repository and writes queries instead of scanning files itself. It writes Tree-sitter abstract syntax tree queries, regex patterns, or call-graph traversals.
- Shard (Plain Code Execution): A deterministic script runs those queries across every file in the codebase. It groups all matching locations into bounded, isolated batches.
- Map (Parallel Worker Sessions): The runtime spins up fresh, independent Devin sessions. Each worker receives one shard of matches. Context isolation prevents workers from sharing hallucinations or context bloat.
- Reduce and Verify (Synthesis and Reproduction): A reducer agent aggregates findings, drops duplicate reports, and connects cross-file paths. Verification agents then run the reproduction script inside an isolated sandbox to confirm the bug exists before notifying a human.
Completeness requires queue exhaustion. The scan ends when the files run out, not when the model claims completion.
The scanner matches the AST selector across 100% of the repository. The agent reasons over a single bounded slice.
Cognition reported 72% recall on pre-fix CVE benchmarks at about $90 per run in their Security Swarm evaluation. The cost is high, but the architectural split works: Hadoop separated the execution engine from mapper code. Agents require the same separation.
Anthropic saw the same dynamic with Claude Research. Dispatching work to 3 to 5 subagents cuts research time by up to 90% with a 90.2% performance gain, but token use rises 15-fold. Token volume alone explained 80% of the variance on BrowseComp.
Where the Analogy Breaks: Six Failures #
If agents map to distributed systems, why not deploy Raft, Paxos, and Kubernetes to manage them? Language models violate the assumptions behind classical distributed systems. In production, the analogy breaks in six places.
1. Failures Are Silent (The HTTP 200 Lie)
In a conventional cluster, failure is loud: segfaults, dropped sockets, or HTTP 500 errors.
An LLM worker does not crash with a stack trace. It responds with HTTP 200, valid JSON matching your schema, and a message saying: “I fixed the bug and verified all unit tests.”
The research proves how pervasive this is. In a 2026 study evaluating 9,876 trajectories on τ2-bench and 1,879 trajectories on AppWorld, false success accounted for 45% to 48% of all failures in single-controller environments. Among AppWorld runs where the agent judged its own work, false success made up 75.8% of failures.
Adding an independent dual-control verifier to inspect environmental state cut false success to 3%.
Check the database, run the compiler, or inspect the file on disk. Never ask an agent if it succeeded.
2. Replay Is a Fantasy (Non-Deterministic Reruns)
In Kafka, EventStore, or Temporal, deterministic replay is the bedrock of resilience. You feed the recorded event log back into your business logic, and you reproduce the exact same state machine.
Try doing that with an LLM.
Even on hosted APIs set to temperature=0, accuracy varied by up to 15 percentage points across identical repeated runs, with a 70-point gap between best and worst outcomes.
Calling an LLM a second time is a retry, never a replay. Reconstructing state requires storing exact token completions in an append-only event store.
3. The CRDT Catastrophe (Syntactic Convergence vs. Semantic Reality)
Distributed systems developers rely on Conflict-free Replicated Data Types (CRDTs). Nodes edit documents offline, exchange delta-states, and converge to identical files without locks.
Teams applied CRDTs to multi-agent coding. Research systems like AgentRoom and CodeCRDT put multiple agents into a shared, CRDT-backed workspace.
CodeCRDT reported 100% convergence with zero merge failures, yet produced semantic conflicts in 5% to 10% of runs:
- The CRDT engine operated without a flaw. Monotonic Lamport timestamps merged concurrent file edits with zero git conflict markers.
- Agent A added a required
scopeparameter toverify()inauth.ts. - At the same time, Agent B added a new route in
server.tsthat callsverify(req.user)with the old signature. - Both edits merged into the filesystem without warning. The text converged across all nodes.
- Running
pnpm buildthrew an immediate compiler error:TS2554: Expected 2 arguments, but got 1.
Distributed systems algorithms ensure clean syntax merges. Validating semantic compatibility requires a compiler. As explored in the OMP Subagent System Deep Dive, concurrent agents need hierarchical supervision, isolated working trees, and compiler oracles before merging changes.
4. The 60% Groupthink Trap (Majority Voting Fails)
To handle flaky nodes, distributed systems engineers reach for consensus algorithms: ask five nodes, take a majority vote, and mask failures.
That math relies on an unstated axiom: replica failures must be independent.
Language model replicas share failure modes. On a four-option multiple-choice leaderboard, two failing models agreed on the exact same wrong answer 60% of the time, according to an ICML 2025 study of more than 350 models.
Majority voting across five copies of a model amplifies shared errors instead of filtering them out. Replace redundant voters with compilers, type-checkers, and test suites.
5. Context Compaction Wiped My Rules
In an operating system, RAM holds data and CPU registers hold instructions. If memory fills up, the OS pages to disk and restores the exact bytes later.
An LLM context window bundles working memory, system instructions, negative constraints, tool schemas, and external data into a single token stream.
Compaction performs a lossy rewrite through another language model, unlike LRU cache eviction.
In Context Compaction Across Agent Harnesses: Why Summarization Is the Wrong Model, summarization drops critical negative constraints. Omitting “Never modify files in /migrations” leaves the agent running while its guardrail disappears. Ten turns later, it edits the forbidden folder.
6. Prompt Injection Is a Confused Deputy, Not a Byzantine Node
In Byzantine fault tolerance, an attacker controls a compromised node sending corrupt packets over the network. Honest nodes isolate the bad actor by taking a quorum.
Prompt injection targets the data channel. The attacker touches no infrastructure; they leave text inside an issue description, pull request, or documentation page.
An honest, authenticated agent reads that text, treats it as an instruction, and uses legitimate credentials to drop a database table or leak secrets.
Quorums fail because every honest node reading the same input receives the same injection. Defending the agent requires capability attenuation: separating data channels from control channels and revoking write permissions during untrusted reads.
What Real Production Systems Do #
Teams shipping reliable systems build deterministic controls around three domains:
1. Autonomous Vulnerability Research: Fuzzers as Ground Truth
Google Project Zero’s Big Sleep and DARPA’s AI Cyber Challenge (AIxCC) use language models to generate hypotheses, not verify bugs.
Big Sleep found a zero-day stack buffer underflow in SQLite before release by running inside an AddressSanitizer (ASan) harness with gdb and code navigation tools.
The agent drafted an input triggering the edge case. The runtime compiled the binary and ran the test in a sandbox. AddressSanitizer confirmed the vulnerability with a memory safety crash trace.
MicroVMs and Agent Sandboxing in Regulated Environments covers the isolation requirements. Security research succeeds when the environment acts as the proof oracle: fuzzers (AFL++, LibFuzzer) and memory sanitizers confirm claims while the model writes harness seeds.
2. Database Query Planners: Selinger Cost Models for Semantic Operators
In 1979, Pat Selinger published System R’s cost-based query optimizer. The optimizer evaluated physical access paths (index scans versus table scans) to minimize disk I/O and CPU cost.
Modern data systems apply that compiler theory to language models. Systems like LOTUS and Palimpzest (with its Abacus optimizer) treat model calls as physical operators in a relational plan.
Instead of writing custom prompts that loop over rows in a dataframe, a developer expresses declarative transformations: sem_filter, sem_join, and sem_top_k. The optimizer compiles that logical plan into a cost-minimal physical execution tree:
- Step 1 (Deterministic Index Scan): Run an exact regex filter or BM25 index over the dataset (cost: $0.00001, latency: 1ms). This discards 85% of irrelevant rows.
2. **Step 2 (Embedding Similarity):** Run vector similarity over the remaining candidates (cost: $0.0001).
3. **Step 3 (Small Model Cascade):** Run an 8B edge model to filter obvious matches (cost: $0.001).
- Step 4 (Frontier Model Resolution): Invoke a frontier reasoning model only on the ambiguous 2% of records.
LOTUS reports speedups of up to 1,000x on some operators, with statistical accuracy guarantees measured against a reference model. The same idea works outside data systems. In Codemode vs. MCP: Code as Orchestration, moving tool orchestration out of turn-by-turn ReAct loops and into code cut token overhead by 98% or more. Grep Is Not BM25 makes the case for cheap deterministic retrieval as the first stage.
3. Operating Systems for Agents: Schedulers and Paging
Agent infrastructure is moving toward operating system mechanics. At NSDI 2026, researchers from UC Berkeley and Google DeepMind introduced Agentix: An Efficient Serving Engine for LLM Agents as General Programs.
Traditional serving engines like vLLM optimize for single-turn token throughput. Agentix schedules the whole multi-turn agent program, the way an operating system schedules a process, and can preempt individual LLM calls to keep programs moving.
Other operating system ideas are close behind:
- Admission Control: Budget anticipated tokens across the task graph and reject excess work before launching runs that would stall mid-flight.
- Context Cache Paging: During 30-second tool delays, page key-value caches out of GPU memory to host RAM and restore them when the tool returns.
- Fencing Tokens and Zombie Culling: If an agent stalls during migration and the supervisor reassigns the shard, monotonic fencing tokens reject late writes from the zombie worker.
Teams are applying formal methods to coordination protocols. Formal Verification: The Proof Oracles Your AI Agent Doesn’t Know It Needs explains the scope: use tools like TLA+ to verify the harness state machine, proving that saga rollbacks complete, leases never deadlock, and retry loops terminate.
Reliability comes from the deterministic harness around the model. Harness Engineering for Reliable AI Agents outlines that architecture.
The Platform Rule: Let the Model Propose, Let the Runtime Decide #
If you build an agent platform in 2026, adhere to one architectural split: Keep the language model in the role of an untrusted advisor. Let it draft plans, interpret messy error messages, synthesize reports, and write code diffs.
Keep everything else in deterministic software. Your scheduling, permissions, token budgeting, deduplication, file sharding, and final commits must live in standard, testable code.
And measure cost per verified successful task, not cost per token. A cheap run that claims false success is the most expensive run you have.
The Bottom Line #
Agent architecture in 2026 is distributed systems applied to fallible, expensive workers.
Reliable production systems wrap probabilistic models inside deterministic runtimes: using code to shard inputs, compilers and sanitizers to verify claims, and event logs to ensure durability.
Use the model for semantic judgment. Use classic software engineering for everything else. Building production coding agents or wrestling with multi-agent orchestration? I’d love to hear what architecture patterns are working in your stack. Reach out on LinkedIn.