One approval, two payments: what an AI agent does after a crash

A developer maintaining the Call4Agents library measured crash-recovery behavior in LangGraph, Pydantic AI and the OpenAI Agents SDK, finding that in 30 runs per framework a single human approval produced two payments every time when the process died before state was saved. The tests also showed that editing a saved tool call after approval caused LangGraph and Pydantic AI to pay 9999 to an attacker-chosen recipient, while the OpenAI SDK refused it; an external guard reduced each case to one payment or none.

Written with help from AI Claude . I maintain Call4Agents, the library mentioned at the end. Every number here comes from scripts you can rerun, and the fixes in the "What to do" list work without my library. A person approves one payment. The tool pays. The process dies before the framework saves its progress. A new process resumes the way the docs say. The payment runs again. I measured this in LangGraph, Pydantic AI and the OpenAI Agents SDK. In all three, one approval gave 2 payments, in every run : 30 runs per framework between 21 October and 5 November, on the newest versions each day. A guard This isn't a framework bug. It's at-least-once execution, which is what any system gives you when it saves progress after doing the work. LangGraph's docs already warn about it. OpenAI added a docs section after I asked. Pydantic AI's triage reproduced it. The trouble is the gap between what "approved" sounds like and what it promises. People read "a human approved this" as "this happens once, exactly as approved". Neither part is guaranteed. os. exit , so no cleanup code runs. A "payment" is a line added to a local file. pay amount=10, to="alice" , which needs approval. A person approves, the tool pays, and the process dies right after. A new process resumes from the saved state. Count the payments. A second case: the saved call is changed to pay 9999, "mallory" after the pause and before it runs. That models anyone who can write to the state store. | Framework | Crash after the approved payment | Saved call edited after the yes | |---|---|---| | LangGraph | 2 payments | paid 9999 to mallory | | Pydantic AI | 2 payments | paid 9999 to mallory | | OpenAI Agents SDK | 2 payments | refused by the SDK itself | | Any of them, with a guard outside | 1 payment | nothing paid | Two details I don't want to hide: pip install call4agents call4agents incident This replays three failures, without and with the guard. My run on 6 November took 5.5 seconds: Incident 1. The double payment A person approves pay 10, alice . The bank charges. The process dies before saving. It resumes. without Call4Agents charges: 2 paid: 10 to alice, 10 to alice with Call4Agents charges: 1 paid: 10 to alice The same, but the process dies inside the call, so nobody knows if it finished: without Call4Agents charges: 2 paid: 10 to alice, 10 to alice with Call4Agents charges: 1 paid: 10 to alice stopped: in doubt, a person must check Incident 2. The changed call A person approves pay 10, alice . The saved call is changed to pay 9999, mallory before it runs. without Call4Agents charges: 1 paid: 9999 to mallory with Call4Agents charges: 0 paid: nothing refused: no yes for this exact call Incident 3. The ignored no A person says no to pay 10, alice . A bug in the agent loop runs the call anyway. without Call4Agents charges: 1 paid: 10 to alice with Call4Agents charges: 0 paid: nothing refused: no yes for this exact call The processes and crashes are real. The bank and the agent loop are small local stand-ins, not the frameworks. Look at the second line of Incident 1. When the process dies inside the call, nobody knows whether the payment went through. The guard doesn't guess and doesn't run it again: it stops as "in doubt". Then a provider check, an idempotency key or a person decides. I think that's the honest answer, because no library can know what happened in someone else's bank. Doing the work and saving that you did it are two separate steps, and no process can make them one. If the crash falls between them, the saved state still says "not done", so a resume does it again. Approval adds another trap. Approval state is usually a flag next to the call: this call id was approved . Nothing ties the "yes" to the exact arguments, to one use, or to a real pause. The library had this kind of bug too. A reader found that in every release from 0.1.0 to 0.5.0, a "yes" for 10 could let 20 run, with only a crash at the wrong moment. All my checks passed the whole time. It's fixed in 0.5.2, with a public advisory https://github.com/salahmakboul/call4agents/security/advisories/GHSA-gxg5-95g9-xw4w . After that I added a test that kills a real process before and after every write, 412 crashes in total. With the old bug put back in, that test failed by itself. It would have found my bug before the reader did. Fake models, one machine, no real payment provider, no production traffic. Three frameworks at a few versions, with the crash at one point. It's a small experiment you can repeat, not a production study. The full write-up, with the script and every run's output, is here: https://github.com/salahmakboul/call4agents/tree/main/docs/finding https://github.com/salahmakboul/call4agents/tree/main/docs/finding If you run an approval flow, try to break it the same way and tell me what happens, with or without my library.