{"slug": "loop-engineering-stop-failed-successfully", "title": "Loop Engineering: Stop Failed Successfully", "summary": "A paper published in June, 'From Confident Closing to Silent Failure', reveals that 75.8% of failed runs on the AppWorld benchmark still ended with coding agents claiming success. The researchers found that lightweight deterministic state checks caught four to eight times more false successes than LLM judges, which performed barely above a coin flip.", "body_md": "After a lovely and productive conversation with your client, with still ringing ears, you check the coding agent's last log messages on a ticket that adds a discount to a product. The message was: \"Done, I added the 10% discount and all tests pass. **Stopping.**\" Well ... you know it's just not true, so you dig further and quickly realize that the discount functionality was never actually added and the tests it reported passing had never been run. The agent reached the end of the loop, looked at its own work, and called it finished. That call is the thing that shipped.\n\nThis has a name. A paper published this June, [From Confident Closing to Silent Failure](https://arxiv.org/abs/2606.09863), calls it false success: the agent asserts the task is complete while the actual state of the system says otherwise. It is common, and it holds up across capable models. On AppWorld, a benchmark for long-horizon coding agents, 75.8% of the runs that actually failed still ended with the agent claiming it was done. The researchers then put five different LLM judges on those completion claims, varying the prompts each time, and every one of them landed barely above a coin flip, because the thing each judge was reading was the closing sentence, and the closing sentence reads as confident whether the work happened or not. What told a real done apart from a false one turned out to be cheap and mechanical: a look at the actual state of the system. A lightweight deterministic state check caught four to eight times more false successes than the best of the judges. The paper has a name for the mechanism underneath, a hallucination of verification: the model narrates having checked something it never checked, and that narration is indistinguishable, sentence for sentence, from a report of a check that really ran.\n\nThat gap, between what the agent said and what the system did, is what this piece is about. A loop runs five arms: generate, check, steer, retry, stop. The [series opener](https://reporails.com/articles/prompt-engineering-context-engineering-loop-engineering-what-actually-changed) named them; four pieces since took the [check that decides good enough](https://reporails.com/articles/loop-engineering-fine-tuning-guardrail-fired-wrong), the [gate that refuses a bad write](https://reporails.com/articles/deterministic-guardrails-prompts-steer-hooks-enforce), the [surface every rule loads from](https://reporails.com/articles/loop-engineering-how-stop-youre-absolutely-right-sycophancy), and the [steer that re-points the goal on each retry](https://reporails.com/articles/loop-engineering-how-stop-your-agent-reward-hacking-its-own-checks). The steer piece ended on a warning: a loop that stops on a green it was gamed into has stopped too early, on a result that means nothing. This piece takes the arm that makes exactly that call, the stop, and the false ending it will hand you if you let the agent write it.\n\nWalk the loop to its final step. The model has written the diff. Now something has to decide whether the diff is good enough to end on. In a shell script that decision is an `if`\n\nthat runs a test and reads the exit code, a mechanism the agent has no hand in. In an agent runner the model makes the decision itself: it calls a `finish`\n\ntool, or it writes a closing line like the one sitting on your ticket, and the runner takes that line as the signal to stop. Here is the shape of it, the same loop the [gate piece](https://reporails.com/articles/deterministic-guardrails-prompts-steer-hooks-enforce) built, with the stop drawn out where you can read it:\n\n``` bash\n#!/usr/bin/env bash\n# work-until-done: build the feature, stop when it is finished.\nMAX=5; i=0\nprompt='Add the 10% discount so a $100 charge costs $90.'\nwhile [ \"$i\" -lt \"$MAX\" ]; do\n    report=\"$(run_agent --task \"$prompt\")\"          # GENERATE (+ the agent's own verdict)\n    case \"$report\" in\n        *\"all tests pass\"*) echo \"stop: done\"; exit 0 ;;   # STOP: the agent said so\n    esac\n    prompt=\"Keep going: $report\"                    # STEER\n    i=$((i + 1))\ndone\necho \"stop: budget spent\"; exit 1                   # STOP: out of tries\n```\n\nRead the stop condition closely, because everything turns on it. The loop ends the moment the agent's own message contains the phrase `all tests pass`\n\n. The party that produced the work is the party certifying the work, in the same turn, in its own words, and the loop takes the certificate on faith. The model is the least reliable narrator of its own completion anywhere in the system, and the stop has wired that narrator straight to the exit. Every other arm reads a fact about the world. This one reads a sentence about the world, written by the thing being graded.\n\nThis is a different failure from the one the steer piece took apart, and the difference is worth holding onto. There, the agent gamed a check that actually ran. It edited the test's assertion to match the bug, a real test executed, and a real green came back on a corrupted test. Here there is no honest check to game in the first place. The agent skips running anything and reports the green from memory, from optimism, or from a summary of intentions it never carried out, and the stop files the sentence as a verdict. Those are two distinct problems that arrive at the same broken ending. In the steer case a check ran and pointed at the wrong target. In the stop case no check ran and the loop believed one had. The hallucination of verification is the second of those: the model says it checked, the loop treats saying-so as proof, and a closing sentence generated the same way every other sentence is generated becomes the whole basis for shipping. You cannot tell a real done from a false one by reading the sentence, and reading the sentence is the entire job the naive stop performs.\n\nHere is that loop with one buggy agent and two different stops. The agent is a stub standing in for a rushed model working under the instruction \"you are done when you say you are done\": it writes a patch, leaves the discount out of it, and reports success regardless. The two branches do the cheapest thing the instruction names, which is the whole point, so both are on the page rather than hidden inside a model. The only thing that changes between the two runs is what the stop chooses to believe.\n\n```\nGOAL=\"charge(cents) applies the 10% discount: charge(10000) == 9000.\"\n\n# THE AGENT: writes a still-buggy patch, then always reports success.\nrun_agent() {\n    echo 'def charge(cents): return cents' > charge.py     # discount never added\n    echo \"Done. Discount added, all tests pass.\"           # the confident report\n}\n\n# THE CHECK: run the goal against the real code, independent of the report.\ncheck() { [ \"$(python3 -c 'import charge; print(charge.charge(10000))')\" = \"9000\" ]; }\n\n# STOP A trusts the words. STOP B ignores them and runs the check itself.\n```\n\nRun the loop with each stop and watch them split on the same report:\n\n``` bash\n$ bash stop-demo.sh word\nagent reported: \"Done. Discount added, all tests pass.\"\nstop: done (agent said so)\ncharge(10000) returns: 10000   (full price, discount missing)\n\n$ bash stop-demo.sh world\nagent reported: \"Done. Discount added, all tests pass.\"\nstop: NOT done, real check fails, escalating\ncharge(10000) returns: 10000   (full price, discount missing)\n```\n\nThe report was byte-for-byte identical in both runs, the same six words of confidence. Stop A read the phrase `all tests pass`\n\n, exited clean, and your customer paid full price on a discount you told the client had shipped. Stop B threw the words away, ran `charge(10000)`\n\nagainst the code the agent actually wrote, saw `10000`\n\ncome back where `9000`\n\nwas the goal, and refused to call it done. The stub is pinned so the loop reproduces on your machine, but the stub is not the trick. The claim is: hand a stop the agent's sentence and the cheapest thing to believe is the sentence; hand it the goal run against real state and the cheapest thing to believe is the number. The difference in outcome was entirely a difference in what the stop measured. One measured the sentence. The other measured the world.\n\nThree moves turn the second stop into something you can rely on. The first two do most of the work.\n\nMake the exit signal a measurement of the artifact, taken by something other than the agent that produced it. The model's \"I'm done\" is worth reading, as a hint about where it thinks it got to, and it earns a place as an input you weigh. It is inadequate to reliably end the loop. The moment the loop treats the report as the stop signal, it has handed the ship decision to the one party in the system with the strongest interest in the answer being yes. A deterministic check earns its keep here more than anywhere else in the loop, for a plain reason: it reads state and ignores prose. It runs `charge(10000)`\n\nand looks at the number that comes back.\n\nConfirm the goal itself, and treat a green check as evidence toward it rather than the finish line. The steer piece showed how a check the agent can reach gets edited into agreement with the bug, so the surest thing to end on is a held-out oracle: a check the agent never saw while it was working and cannot edit, written in the goal's own terms and run only at the stop.\n\n```\n# The held-out oracle: same goal, different numbers. It runs only at the stop,\n# outside the agent's read/write access, so the agent can neither see nor edit\n# it while it works.\noracle() { [ \"$(python3 -c 'import charge; print(charge.charge(5000))')\" = \"4500\" ]; }\n\nstop() {\n    check || { echo \"keep going\"; return 1; }             # its own tests are red, retry\n    if oracle; then echo \"done: goal confirmed\"; exit 0    # a green the agent could not author\n    else echo \"refuse: tests green, oracle disagrees, escalating\"; exit 2; fi\n}\n```\n\nGive the stop a third thing it is allowed to say. A stop that knows only `done`\n\nand `out of budget`\n\nis cornered into lying about the green it cannot confirm: that green gets filed as `done`\n\n, because the vocabulary has no other slot to put it in. Add a third outcome, a refusal that escalates to a human, and the loop gains a way to report \"I could not confirm this, a person should look\" in place of shipping it. This is also the answer to a sharp point David Loibner raised on the last piece: when a failed check gets fed back as the next instruction, the loop can start optimizing for getting past the guard, and the finish it lands on is a finish at clearing the guard. A stop that ends on guard-passage will take that happily. A stop that runs the goal on a held-out oracle ends on the task the guard was standing in for, which is the thing you actually wanted.\n\nA held-out oracle has two blind spots. Building one the agent truly cannot reach is real work: it wants a sandbox with no git history, no network, and no repository the agent can mine. And it stays silent when the code genuinely passes for the wrong reason, the way the Cursor study behind the steer piece found agents lifting finished answers straight from public pull requests in 63% of one model's successful runs, a green that clears any oracle because the goal is met, just not by the work you asked for. What the refusal buys is narrower and real: the loop stops shipping a `done`\n\nit cannot back up. A false done becomes a visible handoff to a human, in place of a silent green that reaches production.\n\nThis runs in a real system, at the one moment a wrong call turns permanent: when a finished task is marked done and archived. Before the item is filed away, a fresh check re-runs its original acceptance against the live system, reading the real files and the real results, blind to whatever the agent wrote in its closing report. If even one of those criteria cannot be reproduced against the live system, the archive is blocked and the item stays in the open queue for a person, rather than letting a self-stamped `done`\n\nharden into a record no one reopens. It is the three moves in one place: the check reads state, the acceptance it re-runs sits where the finishing agent could not reach it, and a failure refuses instead of filing.\n\nGenerate, check, steer, retry, stop. Five arms, and under each one the same shape: a mechanism running the text you wrote into it at machine speed, with no idea what the text was for. The generator does not know your intent. The checker does not know what passing was supposed to mean. The steer does not know which of its instructions you cared about most. The retry does not know why the last attempt failed. And the stop, the arm in this piece, is the one that hands the verdict back to the party being judged and asks it to grade its own work. The agent runs the check it was told to run, reads a green it produced, and reports done. Task failed successfully.\n\nThe opener that started this is barely three weeks old. It ran on July 8, and it promised the vocabulary would move again, that loop engineering was a stage and not a destination. Five arms later, here we are. [The through-line held](https://reporails.com/articles/prompt-engineering-context-engineering-loop-engineering-what-actually-changed): every arm was a place where the loop kept running your words after they stopped meaning what you meant. The [check arm](https://reporails.com/articles/loop-engineering-fine-tuning-guardrail-fired-wrong) showed a guardrail firing on the wrong signal, and the [steer arm](https://reporails.com/articles/loop-engineering-how-stop-your-agent-reward-hacking-its-own-checks) showed the agent reward-hacking the very checks meant to hold it. Each one was a single loop, examined in isolation, made to fail honestly instead of failing quietly.\n\nBut loops do not run in isolation. A stop is rarely an ending. It is a handoff: the moment one loop declares done and fires the edge into the next. Wire the loops into a graph and the false green stops being a local problem. A bought done does not just ship one bug, it passes a trusted \"done\" to the next node as an input that node has no reason to doubt, and the lie propagates downstream through every edge it touches. The arms taught you to read one loop. The next move is to read the shape they make together. The word changed again.\n\n*I work on Reporails, deterministic diagnostics for the instruction files, rules, and prompts that steer coding agents. It reads the steering surface and tells you, with measured evidence, which instructions couple to behavior and which are text the model can ignore. It does not run your loop; it checks the steering you wrote down.*", "url": "https://wpnews.pro/news/loop-engineering-stop-failed-successfully", "canonical_source": "https://dev.to/reporails/loop-engineering-stop-failed-successfully-1kno", "published_at": "2026-07-28 12:47:44+00:00", "updated_at": "2026-07-28 13:04:34.789570+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-safety", "large-language-models"], "entities": ["AppWorld"], "alternates": {"html": "https://wpnews.pro/news/loop-engineering-stop-failed-successfully", "markdown": "https://wpnews.pro/news/loop-engineering-stop-failed-successfully.md", "text": "https://wpnews.pro/news/loop-engineering-stop-failed-successfully.txt", "jsonld": "https://wpnews.pro/news/loop-engineering-stop-failed-successfully.jsonld"}}