{"slug": "do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect", "title": "Do agents survive a crash, and why does an LLM retry execute the same side effect twice? 34 runs measured", "summary": "A developer measured crash recovery and retry idempotency across LangGraph, Strands, and CrewAI in 34 runs, finding that LangGraph's durable checkpointer resumed a killed agent in 0.01-0.02s with zero LLM calls while Strands and CrewAI re-ran the full workflow. The core finding: content-hash idempotency keys silently fail when an LLM rewords tool arguments on retry, producing a different hash and executing the side effect twice, whereas position-based keys ({workflow}:{step}:{tool}) identify intent and dedupe correctly.", "body_md": "When you hand an agent work with external effects, two worries always come up.\n\nFirst: **when the process dies, what happens to the work in progress?** An agent paused at an approval gate crashes — can it resume, or does everything restart from scratch?\n\nSecond: **when an LLM retries, does it execute the same side effect twice?** Retrying a send or a data mutation carries a real double-execution risk.\n\nMy previous article measured HITL approval, audit trails, and structured output across three frameworks. It recorded a failure where Strands double-fired a `publish` call with the identical draft. This article measures what comes next: 34 more runs, same task, same model, same recorder proxy.\n\nRepo (all code, traces, and analysis scripts open):\n\n[https://github.com/sunnydachs/agent-framework-showdown](https://github.com/sunnydachs/agent-framework-showdown)\n\n(Read the previous article here. [here](https://dev.to/sunnydachs/what-happens-when-enterprise-requirements-hit-strands-langgraph-and-crewai-45-runs-measured-ocg).)\n\nSame task, same model, same recorder proxy: 3 frameworks x 3 cells x multiple runs.\n\n```\n[agent] → [approval gate] → [publish (destructive)]\n      ↑               ↑\n  crash it here    pauses waiting\n```\n\nThe three cells:\n\n`publish` under 3 key strategies (position key / content hash / no key) and count duplicate executions on retry\nThe three idempotency key strategies:\n\n```\nposition key:   {workflow}:{step}:{tool}\ncontent hash:   sha256(the raw arguments)\nno key:         nothing\n```\n\nSIGKILL the agent while it waits for approval, then resume in a new process:\n\n| Framework | Persistence | Resume time | State survived | LLM calls to resume | \n|---|---|---|---|---|\n| LangGraph (durable checkpointer) | checkpointer on disk | 0.01-0.02s | 3/3 | **zero** | \n| LangGraph (no checkpointer) | none | - | 0/3 | - | \n| Strands | none built-in | 4.9s avg | full re-run | 5.3 avg | \n| CrewAI | none for agents | 4.2s avg | full re-run | 2.0 avg | \n\nLangGraph's durable checkpointer persists the graph state to disk even while the process is dead. The new process restores to that position in 0.01s — with **zero LLM calls**. The difference between *resume* and *redo* is only whether the state lives outside the process.\n\nWithout a checkpointer, an identical-looking \"resume\" is a full re-run: Strands runs its average 5.3 LLM calls again and pays the full token cost a second time.\n\nLangGraph has a known issue here ([#8764](https://github.com/langchain-ai/langgraph/issues/8764)): if the process dies before the first checkpoint is persisted, recovery may find no checkpoint and no record that the run was ever accepted. On the version I tested, resuming the empty thread **succeeded without raising** — so the behavior is version-dependent. Don't rely on the error either way; keep an external acceptance ledger.\n\nAverage duplicate executions when an LLM retry re-calls `publish`:\n\n| Key strategy | Same-args retry | Reworded retry | \n|---|---|---|\n| Position key | 1 dup, all deduped | 3 dups, 33% deduped + rest rejected as caller bug | \n| Content hash | 1 dup, all deduped | **1 dup, 0% deduped — it slipped through** | \n| No key | 1.33 dups, 0% deduped | 1 dup, 0% deduped | \n\nThis is the core result. **The content-hash key silently fails the moment the model rewords the arguments on retry.**\n\nThe reason is simple. An LLM retry does not replay the saved HTTP request. It **reasons again** from a context that now includes the timeout error, and emits a new tool call. The arguments get reworded, the order changes, fields appear. With `sha256(args)` as the key, the retry produces a different hash, sails past the dedup check, and executes the side effect a second time.\n\nThe position key (`{workflow}:{step}:{tool}`) identifies the **intent** — where the call sits in the workflow — not the bytes. The same position with the same operation yields the same key no matter how the arguments change.\n\nOne more measurement: every duplicated publish carried a **different tool_call ID on the wire**. Nothing at the protocol layer detects \"this is the second time for this operation.\" Detection lives at the recording level only.\n\nNote: the position-key strategy rejects \"same position, different arguments\" calls as a caller bug (2.67 of the runs here). That is by design — it flags intent drift instead of letting the key be reused.\n\nCan an auditor reading the traces alone recover these four facts:\n\nBut this is only because **the recording is at the wire level**. The proxy keeps every attempt — first call, dedup, caller-bug rejection — as its own record, so the auditor can reconstruct everything.\n\nFramework-level trace surfaces show none of the double-firing. What matters in audit design is *where the evidence lives*.\n\nAcross the 34 runs, each key strategy fails differently:\n\nFor enterprise use, the silent failure is the scariest class. **The intuition \"a hash key makes retries safe\" breaks the moment the caller becomes non-deterministic** — that is the conclusion from these measurements.\n\nEverything is open. The README has the commands for all experiments (34 runs here + 72 from the earlier articles):\n\n*This is a personal OSS project — no warranty. Use at your own risk, and issues are welcome.*", "url": "https://wpnews.pro/news/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect", "canonical_source": "https://dev.to/sunnydachs/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect-twice-34-runs-52gd", "published_at": "2026-09-23 15:28:32+00:00", "updated_at": "2026-09-23 15:58:53.153791+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "developer-tools"], "entities": ["LangGraph", "Strands", "CrewAI", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect", "markdown": "https://wpnews.pro/news/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect.md", "text": "https://wpnews.pro/news/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect.txt", "jsonld": "https://wpnews.pro/news/do-agents-survive-a-crash-and-why-does-an-llm-retry-execute-the-same-side-effect.jsonld"}}