From Java Monoliths to LangGraph Agents: What Transfers

A developer with a decade of enterprise Java experience found that principles from building distributed systems—timeouts, idempotency, circuit breakers, rate limiting, and audit trails—transfer directly to building LLM agent systems. The engineer's work on Atlas, a security platform, applies these patterns to agentic workflows, using LangGraph for state machines and Redis-backed budgets for cost control. The developer notes that while backend discipline transfers well, three areas—LLM-specific failure modes, semantic failure detection, and routing between models—had no direct equivalent and required new learning.

I spent ten years building enterprise Java systems — a payments monolith broken into Spring Cloud microservices, banking applications, and currently a Cisco security platform. More recently I moved into building LLM systems seriously. The surprise wasn't how much I had to learn. It was how much transferred. Agentic systems are distributed systems. The vocabulary is new; the failure modes are not. A Java service that calls a flaky dependency without timeouts is a junior-engineer mistake. An agent that calls an LLM without them is somehow normal. LLM calls are slow, occasionally hang, and fail in novel ways malformed JSON, refusals, half-finished tool calls . Everything Resilience4j taught me applies directly: bounded retries with backoff, circuit breakers that fail fast with an honest error instead of silently substituting something expensive, and — this one is new — semantic failure detection, because an LLM can return HTTP 200 with garbage inside. In Atlas, the gateway circuit-breaks to 503 + Retry-After when the model endpoint is down. That is deliberately boring. Boring is the point. Backend rule: any handler that can be retried must be idempotent. Agent rule: same, except the retry might be the planner deciding to call the tool again because the first response "didn't look right." If your agent's tools mutate state, you need idempotency keys and an audit trail more than a chat UI needs streaming. Atlas routes every write through an MCP tool server with audience-scoped tokens RFC 8707 and an append-only, hash-chained audit log, behind a human-in-the-loop checkpoint for governed actions. That design is not AI innovation — it's what any bank-grade action API looks like. The innovation is refusing to drop the standard just because an LLM is the caller. In backend work you plan capacity: connection pools, thread pools, rate limits. In LLM systems the scarce resource is spend. The same machinery applies — token-bucket rate limiting, a per-user daily budget guard in Atlas: Redis-backed, keyed per user per UTC day , alerting at a spend ceiling — plus one new lever: routing . Atlas routes simple queries to a small model and reserves larger models for multi-step reasoning, the way you'd route reads to a replica. Cost regression is gated in CI like a latency budget. LangGraph clicked for me precisely because it isn't magic: it's an explicit state graph with checkpoints — nodes, edges, persisted state, resumability. If you've designed a workflow engine or a saga with compensation steps, you already think this way. The planner–executor pattern with a durable human-approval checkpoint is a state machine an enterprise architect would recognize from a 2010 BPM diagram. That's a compliment. Honesty requires the other list. Three things had no backend equivalent and took real work: If you're a backend engineer eyeing AI engineering: your instincts about timeouts, idempotency, budgets, audit, and state are not legacy baggage — they're the exact discipline most LLM systems are missing. Learn retrieval, learn evals, keep the discipline.