A patient swiped their credit card at an ambulatory emergency department check-in kiosk to settle a $75 copay.
At that exact millisecond, an upstream network partition between the hospital’s revenue cycle management (RCM) gateway and the merchant processor caused a standard HTTP 504 Gateway Timeout. The card had already been authorized upstream, but the confirmation packet dropped before returning across the transport layer.
In a traditional, deterministic service-oriented architecture, this scenario is handled by strict state machines: the transaction enters a pending state, a reconciliation worker polls the gateway out-of-band via an idempotency key, and the UI displays a waiting state.
In an autonomous agent architecture powered by an LLM-driven planning loop (ReAct, LangGraph, or AutoGen), an entirely different sequence unfolds.
The agent’s execution graph evaluates the tool response:
{ "status": 504, "error": "Gateway Timeout", "message": "The upstream server failed to respond within 30000ms."}
The language model does not understand monetary physics. It does not possess an internal model of double-entry bookkeeping, nor does it recognize that an API call represents a live debit against a human being’s bank balance. The model’s loss function is optimized for task resolution and conversational closure.
In its context window, the task state is classified as INCOMPLETE. The agent's next-token distribution selects the most mathematically probable step to resolve an incomplete task: retry the tool call.
It retried. The network latency persisted, returning another 504. The agent retried again.
Four minutes later, the autonomous loop terminated only because the patient’s bank flagged the account for fraudulent velocity. By that point, the model had executed 14 consecutive transactions, draining over $1,000 from the patient’s checking account.
The root cause of this failure is an architectural anti-pattern common across modern enterprise agent deployments: granting a probabilistic token predictor unrestricted write authority over an atomic database or external financial API.
THE UNBOUNDED RETRY LOOP (FAILURE ARCHITECTURE):┌────────────────────────┐ ┌───────────────────────────┐ ┌───────────────────────────────┐│ Patient ER Check-in ├─────►│ Autonomous LLM Agent ├─────►│ Payment API / EDI 837 ││ Copay Event │ │ (ReAct Planning Loop) │ │ POST /v1/charges │└────────────────────────┘ └─────────────┬─────────────┘ └───────────────┬───────────────┘ ▲ │ │ HTTP 504 Timeout │ Upstream Auth OK, │ (Dropped Response Packet) │ Packet Dropped │ ▼ │ ┌─────────────────────────────┐ └──────────────────────┤ Model Assumes Task Failure │ │ Retries Action Immediately │ └─────────────┬───────────────┘ │ ▼ [14x DUPLICATE TRANSACTIONS] [PATIENT ACCOUNT FROZEN]
When engineering teams assemble an AI agent using off-the-shelf frameworks, they wire tool definitions directly into the model’s context:
When the tool executes, the system relies on the LLM to govern the execution control flow. This is a fatal category error: an LLM is an entity extraction and reasoning engine, not a distributed transactional coordinator.
Relying on a system prompt to enforce financial controls — such as “Only charge the card once. If you receive an error, do not retry without human confirmation” — fails under production conditions:
To deploy autonomous digital labor safely into high-stakes clinical and financial operations, generative inference must be strictly decoupled from atomic execution.
The model should propose actions, but it must never possess direct authority to commit state changes. Every proposed mutation must pass through an out-of-band Deterministic Invariant Gateway that validates the transaction against strict operational rules before network packets reach external rails.
CLAIRE RUNTIME GOVERNANCE ARCHITECTURE:┌────────────────────────┐│ Patient Intake Event │└───────────┬────────────┘ │ ▼┌────────────────────────────────────────────────────────┐│ Generative Inference Node (Zero Direct API Access) ││ - Intent Parsing & Parameter Extraction ││ - Emits Unsigned Intent: `ChargeIntent(amt=7500, ...)` │└───────────┬────────────────────────────────────────────┘ │ ▼┌────────────────────────────────────────────────────────────────────────────────────────┐│ DETERMINISTIC INVARIANT RUNTIME GATEWAY ││ ││ [Assertion 1: Idempotency Key Engine] ││ - Compute Hash: SHA256(patient_id + encounter_id + window_hour) ││ - Query Redis Lock: If Key Exists -> REJECT MUTATION ││ ││ [Assertion 2: Rate-of-Change Circuit Breaker] ││ - Evaluate Velocity: dCost/dt across active encounter ││ - Max Allowed Mutations per Encounter Window: 1 ││ ││ [Assertion 3: Out-of-Band State Verifier] ││ - On 504 Timeout: Sever Agent Control Flow ││ - Dispatch Asynchronous Polling Worker to Gateway Reconciliation Ledger │└───────────────────────────┬────────────────────────────────────────────────────────────┘ │ ┌─────────────┴─────────────┐ │ │ ▼ (Pass) ▼ (Fail)┌───────────────────────────┐ ┌───────────────────────────┐│ Certified Atomic Commit │ │ Freeze Execution Thread ││ External Processor / EDI │ │ Dispatch Alert to Human │└───────────────────────────┘ └───────────────────────────┘
A probabilistic model must never generate its own idempotency keys. If an agent loops, it will often generate a new UUID for each iteration, completely defeating downstream API deduplication.
The gateway must derive the key deterministically from the business context:
Where:
import hmacimport hashlibimport timedef generate_deterministic_idempotency_key( secret_key: bytes, patient_id: str, encounter_id: str, window_seconds: int = 3600) -> str: # Bucket current epoch time into fixed time windows time_bucket = int(time.time() // window_seconds) payload = f"{patient_id}:{encounter_id}:{time_bucket}".encode("utf-8") return hmac.new(secret_key, payload, hashlib.sha256).hexdigest()
The runtime gateway must track the velocity of execution. In distributed systems, this is the derivative of financial spend over time:
If an agent attempts more than one financial mutation per encounter within a 60-minute window, the gateway severs the network interface instantly, raises an out-of-band operational incident, and drops the execution graph into a human-review queue.
When an upstream endpoint returns an HTTP 504, 502, or TCP drop, control flow must be stripped from the generative agent immediately.
The transaction state is marked as INDETERMINATE. The system spins up an asynchronous verification worker that queries the payment ledger using the deterministic idempotency key. The generative model is prohibited from retrying or taking any further action until the ledger confirms whether the transaction succeeded or failed.
If you are deploying autonomous agents into environments where mutations alter reality — charging money, modifying clinical records, dispensing medications, or altering server infrastructure — your architecture must adhere to three foundational rules:
Stop building agent architectures designed for Twitter demos. Build deterministic systems that survive production.
The Anatomy of a 14x Billing Cascade: Why AI Agents Fail at Distributed State was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.