{"slug": "designing-ai-agents-that-can-self-correct", "title": "Designing AI Agents That Can Self-Correct", "summary": "A new tutorial from Anthropic demonstrates that AI agents can reliably self-correct only when grounded in external verification, citing a 2024 paper showing large language models cannot self-correct reasoning without outside input. The guide builds a code-generation agent using LangGraph and pytest that runs real tests, retries with bounds, and escalates to humans when needed, noting that reflection adds roughly 5% on hard benchmarks like MATH but costs 40% more compute on easy tasks.", "body_md": "In this article, you will learn how to design AI agents that can reliably self-correct by grounding their feedback loops in external verification rather than the model’s own judgment.\n\nTopics we will cover include:\n\n- Why self-correction in language models only works when the agent has an external signal to check against, and when it isn’t worth the cost.\n- How to build a code-generation agent with a real test-based verifier, a bounded retry loop, and a structured escalation path.\n- How to add a consistency-based confidence gate that generates an independent second solution to confirm correctness before shipping.\n\n## Introduction\n\nIn 2024, a team of researchers published a paper with a blunt title: “[Large Language Models Cannot Self-Correct Reasoning Yet](https://arxiv.org/abs/2310.01798).” Their finding was uncomfortable for anyone building agents at the time. When you ask a model to check its own reasoning with no outside input, it doesn’t reliably catch its mistakes. Sometimes it does the opposite: it talks itself into believing a wrong answer is right, and the “corrected” version comes out worse than the first draft, a pattern later work has confirmed and built on.\n\nThat finding sits at the center of everything in this article. Self-correction in AI agents is real; it isn’t a trick or a marketing term, but it only works under a specific condition: the agent needs something outside its own opinion to check against. Give it that, and the loop catches real mistakes. Skip it, and you’ve built an elaborate way for the model to agree with itself.\n\nThis tutorial builds one complete example so that the condition stays concrete rather than abstract: a code-generation agent that writes a Python function, actually runs the function’s tests, fixes what fails, and knows when to stop trying and hand the problem to a person instead.\n\n**Prerequisites**:\n\n- Python 3.10 or newer\n- An Anthropic API key\n-\n\n```\npip install langgraph langchain-anthropic pytest python-dotenv\n\n1\n\npip install langgraph langchain-anthropic pytest python-dotenv\n```\n\n## Why Asking a Model to Check Its Own Work Usually Fails\n\nPicture asking a student to grade their own exam with no answer key. They’ll fix the mistakes they notice, but the mistakes they don’t notice are exactly the ones they’ll approve again on a second look. That’s the coherence trap: a language model’s critique of its own output is generated by the same weights, trained on the same patterns, that produced the output in the first place. It’s not an independent check. It’s the same judgment asked twice, and the two answers tend to agree, whether or not either is correct.\n\nThis doesn’t mean reflection is worthless; it means reflection only works when it’s grounded in something the generator didn’t produce. The original [Reflexion paper](https://openreview.net/pdf?id=FDG2G7JDWO) out of Stanford showed agents with verbal self-reflection reaching **91% pass@1** on [HumanEval](https://deepeval.com/docs/benchmarks-human-eval), up from an 80% baseline, and a 20-point absolute gain on HotpotQA question answering over a standard ReAct agent. [Madaan et al.’s Self-Refine paper](https://arxiv.org/abs/2303.17651) found a similar 20% average improvement across seven different tasks. Those are real gains, and what they have in common is that the tasks gave the model something to check against: code has tests that either pass or fail, and multi-step retrieval has documents that either answer the question or don’t.\n\nWhere reflection stops paying its way is simpler tasks with nothing external to check. The [2025 CorrectBench study](https://www.claudio-novaglio.com/en/blog/ai-automation/ai-agent-self-correction-feedback-loop) found self-correction adds roughly 5% on hard reasoning benchmarks like MATH, but on easy tasks, plain chain-of-thought reasoning does just as well using **40%** less compute. Reflection isn’t free. It costs tokens, latency, and money every time the loop runs, so the question worth asking before you build one isn’t “would reflection help,” it’s “do I have something external for the critic to check against, and is the task hard enough to justify the extra calls?”\n\nThat’s the rule the rest of this article follows: *ground the critic in something the generator didn’t write*. For code, that’s running the tests. For research, that’s a retrieved source. For a form-filling agent, that’s schema validation. Whatever your project is, find that external signal before you write a single line of correction logic, because without it, you’re building a more expensive version of the same mistake.\n\n## The Building Blocks, Before You Write Any Code\n\nFive pieces show up in almost every production self-correction system, and it’s worth knowing what each one is actually for before wiring them together.\n\n**Reflection loops** are the generate-critique-revise cycle itself. The loop only works if it’s bounded. An unbounded reflection loop isn’t a safety feature; it’s a liability, and a widely shared 2026 postmortem described a document-processing agent that entered a retry loop overnight and[ran up a $437 bill](https://dev.to/magicrails/i-let-my-ai-agent-run-overnight-it-cost-437-dd7)in eight hours before anyone noticed. Every loop in this article carries a hard cap.**Verifiers** check the generator’s output. The important distinction is between a verifier and a calibration model: a verifier scores output quality in a way that’s independent of which model produced it, while a calibration model estimates how confident the specific generating model should be in its own output, which is a subtly different and weaker signal, as a[2025 paper on fine-grained confidence estimation](https://arxiv.org/abs/2508.12040)lays out. In production, the strongest and cheapest verifiers are usually the simplest: run the code, check the schema, query the database. Save trained process reward models, which score intermediate reasoning steps rather than only the final answer, for cases where you genuinely can’t execute or check the output directly.**Confidence scoring** sounds like it should solve the “how sure is the agent” question cheaply, but current research is direct about its limits. A[2026 ACL paper on uncertainty quantification](https://aclanthology.org/2026.acl-long.737/)tested three common approaches (log-probability, self-consistency sampling, and verbalized confidence) on agent tasks and found all three scored close to a random guess for predicting failure, with AUROC values around**0.55 to 0.6 against a 0.5 baseline**. Verbalized confidence, the cheapest option since it just means asking the model how sure it is, is also the least reliable once an agent’s context gets long and noisy. The more dependable version of confidence scoring in practice is consistency-based: generate a solution twice, independently, and check whether they agree. Disagreement is a real signal. Two independent attempts agreeing with each other are meaningfully stronger evidence than one attempt saying “I’m 95% sure.”**Retry policies** govern what happens after a failure. The standard pattern is exponential backoff with jitter — wait a bit longer after each failure with some randomness added so a fleet of agents doesn’t all retry at the same moment — paired with a circuit breaker so a sustained outage trips the whole call site instead of hammering a struggling service for an hour. The detail that catches teams off guard is that this needs to be enforced outside the model’s own reasoning. An agent that decides on its own to “try a different approach” after a timeout is still retrying, just invisibly, and infrastructure-level rate limits can’t see a retry that’s happening inside the model’s chain of thought rather than as a distinct API call.**Recovery architecture** is what happens once the retry budget is spent. A circuit breaker and a kill switch solve different problems: a kill switch is a person noticing something wrong and stopping it manually, while a circuit breaker is an automatic rule that trips before a person needs to notice anything. The end state of a good recovery path is not “crash,” it’s a clean escalation with the full failure trajectory logged somewhere a person can actually read it, which is the same idea behind dead-letter queues in traditional fault-tolerant systems, applied to agent failures instead of message queues.\n\nWith the vocabulary and the failure modes in place, here’s the build.\n\n## Build the Generator and the Grounded Verifier\n\n**The project**: an agent that receives a short function spec, writes the implementation, and checks it against a real test file rather than its own judgment of whether the code looks correct.\n\nStart with the project folder:\n\n```\nmkdir self-correcting-agent && cd self-correcting-agent\npython3 -m venv venv\nsource venv/bin/activate\npip install langgraph langchain-anthropic pytest python-dotenv\n\n1234\n\nmkdir self-correcting-agent && cd self-correcting-agentpython3 -m venv venvsource venv/bin/activatepip install langgraph langchain-anthropic pytest python-dotenv\n```\n\nCreate a **.env** file with your key:\n\n```\n# .env\nANTHROPIC_API_KEY=your-anthropic-key-here\n\n12\n\n# .envANTHROPIC_API_KEY=your-anthropic-key-here\n```\n\nNow the generator, which asks Claude to write a function based on a spec, and includes the previous failure as feedback if this isn’t the first attempt:\n\n``` python\n# agent.py\nimport os\nfrom dotenv import load_dotenv\nfrom langchain_anthropic import ChatAnthropic\n\nload_dotenv()\n\nmodel = ChatAnthropic(model=\"claude-sonnet-4-6\", temperature=0.2, max_tokens=500)\n\ndef generate_code(spec: str, feedback: str | None) -> str:\n    \"\"\"Asks the model to write a function matching the spec. If feedback\n    from a failed test run is provided, it's included so the model isn't\n    guessing blind on retries.\"\"\"\n    prompt = f\"Write a single Python function for this spec:\\n{spec}\\n\"\n    prompt += \"Return only the function code, no explanation, no markdown fences.\"\n    if feedback:\n        prompt += f\"\\n\\nThe previous attempt failed these tests:\\n{feedback}\\nFix it.\"\n\n    response = model.invoke(prompt)\n    # Strip markdown fences in case the model adds them despite instructions\n    code = response.content.strip()\n    if code.startswith(\"```\"):\n        code = code.split(\"```\")[1]\n        if code.startswith(\"python\"):\n            code = code[len(\"python\"):]\n    return code.strip()\n\n1234567891011121314151617181920212223242526\n\n# agent.pyimport osfrom dotenv import load_dotenvfrom langchain_anthropic import ChatAnthropic load_dotenv() model = ChatAnthropic(model=\"claude-sonnet-4-6\", temperature=0.2, max_tokens=500) def generate_code(spec: str, feedback: str | None) -> str:    \"\"\"Asks the model to write a function matching the spec. If feedback    from a failed test run is provided, it's included so the model isn't    guessing blind on retries.\"\"\"    prompt = f\"Write a single Python function for this spec:\\n{spec}\\n\"    prompt += \"Return only the function code, no explanation, no markdown fences.\"    if feedback:        prompt += f\"\\n\\nThe previous attempt failed these tests:\\n{feedback}\\nFix it.\"     response = model.invoke(prompt)    # Strip markdown fences in case the model adds them despite instructions    code = response.content.strip()    if code.startswith(\"**What this does**: the function builds a single prompt that includes the spec and, critically, the actual test failure output from the last attempt when there’s been one. That feedback is what separates this from a blind retry; the model isn’t generating a fresh guess each time, it’s responding to specific evidence of what broke. The markdown-stripping at the end handles a common annoyance: models often wrap code in fences even when told not to, and leaving those in would break the file we’re about to write to disk.\n\nNext, the verifier — the part doing the actual grounding:\n\n``` python\n# verifier.py\nimport subprocess\nimport tempfile\nfrom pathlib import Path\n\ndef run_tests(code: str, test_code: str) -> tuple[bool, str]:\n    \"\"\"Writes the generated code and a test file to a temporary directory\n    and actually runs pytest against them. This is the external check the\n    generator can't talk its way around — the tests either pass or they don't.\"\"\"\n    with tempfile.TemporaryDirectory() as tmp:\n        tmp_path = Path(tmp)\n        (tmp_path / \"solution.py\").write_text(code)\n        (tmp_path / \"test_solution.py\").write_text(test_code)\n\n        result = subprocess.run(\n            [\"python3\", \"-m\", \"pytest\", \"test_solution.py\", \"-q\"],\n            cwd=tmp_path,\n            capture_output=True,\n            text=True,\n            timeout=15,\n        )\n        passed = result.returncode == 0\n        output = result.stdout + result.stderr\n        return passed, output\n\n123456789101112131415161718192021222324\n\n# verifier.pyimport subprocessimport tempfilefrom pathlib import Path def run_tests(code: str, test_code: str) -> tuple[bool, str]:    \"\"\"Writes the generated code and a test file to a temporary directory    and actually runs pytest against them. This is the external check the    generator can't talk its way around — the tests either pass or they don't.\"\"\"    with tempfile.TemporaryDirectory() as tmp:        tmp_path = Path(tmp)        (tmp_path / \"solution.py\").write_text(code)        (tmp_path / \"test_solution.py\").write_text(test_code)         result = subprocess.run(            [\"python3\", \"-m\", \"pytest\", \"test_solution.py\", \"-q\"],            cwd=tmp_path,            capture_output=True,            text=True,            timeout=15,        )        passed = result.returncode == 0        output = result.stdout + result.stderr        return passed, output\n```\n\n**What this does**: this function has no opinion of its own about whether the code is good. It writes the model’s output to a real file, runs pytest against it as a genuine subprocess, and reports back exactly what **pytest** reports: pass, fail, and the specific assertion errors if it failed. There’s no LLM call anywhere in this function. That absence is the entire point. This is the grounded signal that the first section argued you need before reflection is worth building at all.\n\n## Add the Correction Loop with a Bounded Retry Budget\n\nWith a generator and a real verifier, the next step is wiring them into a loop that retries on failure, feeds the test output back as feedback, and stops after a fixed number of attempts. This is where [LangGraph](https://www.langchain.com/langgraph) earns its place: the state machine model makes the cycle, and its exit conditions, explicit instead of buried in nested if-statements.\n\n``` python\n# graph.py\nfrom typing import TypedDict, Optional\nfrom langgraph.graph import StateGraph, END\nfrom agent import generate_code\nfrom verifier import run_tests\n\nclass AgentState(TypedDict):\n    spec: str\n    test_code: str\n    code: str\n    feedback: Optional[str]\n    attempts: int\n    max_attempts: int\n    status: str\n\ndef generate_node(state: AgentState) -> AgentState:\n    code = generate_code(state[\"spec\"], state.get(\"feedback\"))\n    return {**state, \"code\": code}\n\ndef verify_node(state: AgentState) -> AgentState:\n    passed, output = run_tests(state[\"code\"], state[\"test_code\"])\n    attempts = state[\"attempts\"] + 1\n    if passed:\n        return {**state, \"attempts\": attempts, \"status\": \"verified\", \"feedback\": None}\n    return {**state, \"attempts\": attempts, \"status\": \"failed\", \"feedback\": output[-800:]}\n\ndef escalate_node(state: AgentState) -> AgentState:\n    # In production this is where you'd log the full trajectory to a\n    # database or ticket queue instead of just changing the status\n    return {**state, \"status\": \"escalated\"}\n\ndef router(state: AgentState) -> str:\n    \"\"\"This is the correction budget in code. Failure alone doesn't loop\n    forever — it loops until attempts hits the cap, then stops for good.\"\"\"\n    if state[\"status\"] == \"verified\":\n        return \"end\"\n    if state[\"status\"] == \"failed\" and state[\"attempts\"] < state[\"max_attempts\"]:\n        return \"retry\"\n    return \"escalate\"\n\nbuilder = StateGraph(AgentState)\nbuilder.add_node(\"generate\", generate_node)\nbuilder.add_node(\"verify\", verify_node)\nbuilder.add_node(\"escalate\", escalate_node)\nbuilder.set_entry_point(\"generate\")\nbuilder.add_edge(\"generate\", \"verify\")\nbuilder.add_conditional_edges(\"verify\", router, {\n    \"retry\": \"generate\",\n    \"escalate\": \"escalate\",\n    \"end\": END,\n})\nbuilder.add_edge(\"escalate\", END)\n\ngraph = builder.compile()\n\n123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354\n\n# graph.pyfrom typing import TypedDict, Optionalfrom langgraph.graph import StateGraph, ENDfrom agent import generate_codefrom verifier import run_tests class AgentState(TypedDict):    spec: str    test_code: str    code: str    feedback: Optional[str]    attempts: int    max_attempts: int    status: str def generate_node(state: AgentState) -> AgentState:    code = generate_code(state[\"spec\"], state.get(\"feedback\"))    return {**state, \"code\": code} def verify_node(state: AgentState) -> AgentState:    passed, output = run_tests(state[\"code\"], state[\"test_code\"])    attempts = state[\"attempts\"] + 1    if passed:        return {**state, \"attempts\": attempts, \"status\": \"verified\", \"feedback\": None}    return {** state, \"attempts\": attempts, \"status\": \"failed\", \"feedback\": output[-800:]} def escalate_node(state: AgentState) -> AgentState:    # In production this is where you'd log the full trajectory to a    # database or ticket queue instead of just changing the status    return {**state, \"status\": \"escalated\"} def router(state: AgentState) -> str:    \"\"\"This is the correction budget in code. Failure alone doesn't loop    forever — it loops until attempts hits the cap, then stops for good.\"\"\"    if state[\"status\"] == \"verified\":        return \"end\"    if state[\"status\"] == \"failed\" and state[\"attempts\"] < state[\"max_attempts\"]:        return \"retry\"    return \"escalate\" builder = StateGraph(AgentState)builder.add_node(\"generate\", generate_node)builder.add_node(\"verify\", verify_node)builder.add_node(\"escalate\", escalate_node)builder.set_entry_point(\"generate\")builder.add_edge(\"generate\", \"verify\")builder.add_conditional_edges(\"verify\", router, {    \"retry\": \"generate\",    \"escalate\": \"escalate\",    \"end\": END,})builder.add_edge(\"escalate\", END) graph = builder.compile()\n```\n\n**What this does**: **AgentState** is the shared memory the whole loop reads and writes, tracking not just the code but the attempt count and status, which is what makes the cap enforceable. **verify_node** is where the real test output becomes feedback for the next generation attempt, if there is one. The **router** function is the single most important piece of this file: it’s a plain Python function, not a prompt, deciding whether to loop, stop, or hand off, which means the retry cap can never be argued out of by the model’s own reasoning, the way an infrastructure-level timeout can be. That distinction is exactly what the circuit breaker research cited earlier points to as the real fix — not a bigger kill switch, but a rule that lives outside the agent’s own decision-making.\n\nTo run it, add a small entry point:\n\n``` python\n# run.py\nfrom graph import graph\n\nspec = \"write is_palindrome(s), a function that returns True if a \" \\\n       \"string reads the same forwards and backwards, ignoring case and spaces\"\n\ntest_code = \"\"\"\nfrom solution import is_palindrome\n\ndef test_simple_true():\n    assert is_palindrome(\"level\") is True\n\ndef test_simple_false():\n    assert is_palindrome(\"hello\") is False\n\ndef test_ignores_case_and_spaces():\n    assert is_palindrome(\"Nurses Run\") is True\n\"\"\"\n\nresult = graph.invoke({\n    \"spec\": spec,\n    \"test_code\": test_code,\n    \"code\": \"\",\n    \"feedback\": None,\n    \"attempts\": 0,\n    \"max_attempts\": 3,\n    \"status\": \"pending\",\n})\n\nprint(\"Status:\", result[\"status\"])\nprint(\"Attempts used:\", result[\"attempts\"])\nprint(\"\\nFinal code:\\n\", result[\"code\"])\n\n1234567891011121314151617181920212223242526272829303132\n\n# run.pyfrom graph import graph spec = \"write is_palindrome(s), a function that returns True if a \" \\       \"string reads the same forwards and backwards, ignoring case and spaces\" test_code = \"\"\"from solution import is_palindrome def test_simple_true():    assert is_palindrome(\"level\") is True def test_simple_false():    assert is_palindrome(\"hello\") is False def test_ignores_case_and_spaces():    assert is_palindrome(\"Nurses Run\") is True\"\"\" result = graph.invoke({    \"spec\": spec,    \"test_code\": test_code,    \"code\": \"\",    \"feedback\": None,    \"attempts\": 0,    \"max_attempts\": 3,    \"status\": \"pending\",}) print(\"Status:\", result[\"status\"])print(\"Attempts used:\", result[\"attempts\"])print(\"\\nFinal code:\\n\", result[\"code\"])\n```\n\n**How to run it**: with your **.env** file in place and the virtual environment active, run **python run.py**. On a spec like this, don’t be surprised if the first attempt fails; a first-pass implementation commonly ignores case or spaces, exactly like the naive **s == s[::-1]** version does, and it’s genuinely useful to watch the loop catch that, feed the pytest failure back in, and produce a corrected version on the second pass.\n\n## Add a Confidence Gate Before Anything Ships\n\nPassing the tests you wrote isn’t the same as being correct. A solution can pass three test cases and still be fragile on inputs nobody thought to check. Since the second section covered why self-reported confidence scores are only barely better than guessing, the gate we’re adding here uses the more reliable signal instead: generate a second, independent solution to the same spec, and check whether it agrees with the first one on cases beyond the original tests.\n\n``` python\n# confidence_gate.py\nfrom agent import generate_code\nfrom verifier import run_tests\n\nEDGE_CASES = \"\"\"\nfrom solution import is_palindrome\n\ndef test_empty_string():\n    assert is_palindrome(\"\") is True\n\ndef test_single_character():\n    assert is_palindrome(\"a\") is True\n\ndef test_mixed_case_and_punctuation_spacing():\n    assert is_palindrome(\"A Santa At NASA\") is True\n\"\"\"\n\ndef confidence_check(spec: str, primary_code: str, main_test_code: str) -> dict:\n    \"\"\"Generates an independent second solution and checks whether both\n    solutions agree on the original tests plus a held-out set of edge\n    cases the correction loop never saw. Agreement between two independent\n    attempts is a stronger signal than either model asking itself how\n    confident it feels.\"\"\"\n    second_code = generate_code(spec, feedback=None)\n\n    second_on_main, _ = run_tests(second_code, main_test_code)\n    primary_on_edges, _ = run_tests(primary_code, EDGE_CASES)\n    second_on_edges, _ = run_tests(second_code, EDGE_CASES)\n\n    agree = second_on_main and primary_on_edges and second_on_edges\n    return {\n        \"confirmed\": agree,\n        \"second_code\": second_code,\n        \"primary_passed_edges\": primary_on_edges,\n        \"second_passed_edges\": second_on_edges,\n    }\n\n123456789101112131415161718192021222324252627282930313233343536\n\n# confidence_gate.pyfrom agent import generate_codefrom verifier import run_tests EDGE_CASES = \"\"\"from solution import is_palindrome def test_empty_string():    assert is_palindrome(\"\") is True def test_single_character():    assert is_palindrome(\"a\") is True def test_mixed_case_and_punctuation_spacing():    assert is_palindrome(\"A Santa At NASA\") is True\"\"\" def confidence_check(spec: str, primary_code: str, main_test_code: str) -> dict:    \"\"\"Generates an independent second solution and checks whether both    solutions agree on the original tests plus a held-out set of edge    cases the correction loop never saw. Agreement between two independent    attempts is a stronger signal than either model asking itself how    confident it feels.\"\"\"    second_code = generate_code(spec, feedback=None)     second_on_main, _ = run_tests(second_code, main_test_code)    primary_on_edges, _ = run_tests(primary_code, EDGE_CASES)    second_on_edges, _ = run_tests(second_code, EDGE_CASES)     agree = second_on_main and primary_on_edges and second_on_edges    return {        \"confirmed\": agree,        \"second_code\": second_code,        \"primary_passed_edges\": primary_on_edges,        \"second_passed_edges\": second_on_edges,    }\n```\n\n**What this does**: the held-out edge cases (empty strings, single characters, punctuation) were never shown to the correction loop, so passing them isn’t something either solution could have been specifically patched for. The second solution also has to clear the original test file on its own, written independently, with no memory of the first attempt’s mistakes.\n\nIf an independently generated second attempt and the original both clear all of that, the agreement itself is the confidence signal — not a number the model reports about its own certainty. When this pattern is tested, the second differently-written solution and the corrected first one typically agree on every case, which is the outcome that lets you ship without a human in the loop. When they disagree, that’s not a minor discrepancy to shrug off; it’s exactly the kind of signal that should route to a person, since it means the tests you wrote weren’t strict enough to fully pin down the correct behavior in the first place.\n\nWire this into the graph as one more node after verification passes, routing to escalation on disagreement instead of a silent pass:\n\n``` python\n# in graph.py, add:\nfrom confidence_gate import confidence_check\n\ndef confidence_node(state: AgentState) -> AgentState:\n    result = confidence_check(state[\"spec\"], state[\"code\"], state[\"test_code\"])\n    if result[\"confirmed\"]:\n        return {**state, \"status\": \"confirmed\"}\n    return {**state, \"status\": \"escalate_disagreement\"}\n\n12345678\n\n# in graph.py, add:from confidence_gate import confidence_check def confidence_node(state: AgentState) -> AgentState:    result = confidence_check(state[\"spec\"], state[\"code\"], state[\"test_code\"])    if result[\"confirmed\"]:        return {**state, \"status\": \"confirmed\"}    return {** state, \"status\": \"escalate_disagreement\"}\n```\n\nUpdate the router so “verified” leads to “confidence_node” instead of straight to **END**, and add a conditional edge out of it that sends “confirmed” to **END** and anything else to “escalate”. The shape of the graph stays the same — generate, verify, gate, escalate — it just gets one more grounded check before calling anything done.\n\n## What Happens When the Agent Can’t Fix Itself\n\nA retry budget only works if hitting it actually does something useful instead of just quietly failing. The **escalate_node** in the graph above is deliberately bare-bones as written; in a real deployment, it needs to do three things: stop the loop for good (which the router already guarantees), record exactly what was tried, and put the failure somewhere a person will actually see it.\n\n``` python\n# recovery.py\nimport json\nfrom datetime import datetime, timezone\n\ndef log_escalation(state: dict, log_path: str = \"escalations.jsonl\") -> None:\n    \"\"\"Appends the full failure trajectory to a log file. In production,\n    swap this for a write to a database or a ticket in your team's queue —\n    the point is that nothing gets silently dropped.\"\"\"\n    record = {\n        \"timestamp\": datetime.now(timezone.utc).isoformat(),\n        \"spec\": state[\"spec\"],\n        \"final_code\": state[\"code\"],\n        \"attempts\": state[\"attempts\"],\n        \"last_feedback\": state.get(\"feedback\"),\n        \"status\": state[\"status\"],\n    }\n    with open(log_path, \"a\") as f:\n        f.write(json.dumps(record) + \"\\n\")\n\n123456789101112131415161718\n\n# recovery.pyimport jsonfrom datetime import datetime, timezone def log_escalation(state: dict, log_path: str = \"escalations.jsonl\") -> None:    \"\"\"Appends the full failure trajectory to a log file. In production,    swap this for a write to a database or a ticket in your team's queue —    the point is that nothing gets silently dropped.\"\"\"    record = {        \"timestamp\": datetime.now(timezone.utc).isoformat(),        \"spec\": state[\"spec\"],        \"final_code\": state[\"code\"],        \"attempts\": state[\"attempts\"],        \"last_feedback\": state.get(\"feedback\"),        \"status\": state[\"status\"],    }    with open(log_path, \"a\") as f:        f.write(json.dumps(record) + \"\\n\")\n```\n\n**What this does**: this is the same idea behind a dead-letter queue in ordinary distributed systems, applied to an agent’s failure instead of a message that couldn’t be processed. Nothing here tries to fix the problem again. It records exactly what spec was given, what the last attempt looked like, and why it failed, so a person picking this up later isn’t starting from zero. Call **log_escalation(result)** right after **graph.invoke(…)** whenever **result[“status”]** isn’t “confirmed”, and you have a clean, auditable trail instead of a print statement that scrolled off a terminal three deploys ago.\n\nThis is also the point worth remembering from the very first section. The circuit breaker here isn’t a consolation prize for a system that failed to be fully autonomous. It’s the thing that makes the autonomy trustworthy in the first place, because a system that knows exactly when to stop and ask for help is a more reliable system than one that always claims to have the answer.\n\n## Wrapping Up\n\nEverything in this build comes back to one idea: a self-correcting agent is only as good as what it’s allowed to check itself against. The **generator** writes code, but it never gets to decide on its own whether that code is right; **pytest** decides that. The confidence gate doesn’t ask the model how sure it feels; it checks whether two independent attempts land on the same answer. And when neither of those checks clears, the system doesn’t retry forever, hoping the next attempt is better; it stops on a fixed budget and hands the problem to a person with the full history attached.\n\nIf you take this further, the natural next step is process reward models, which score intermediate reasoning steps instead of only the final pass or fail — useful once your tasks get complex enough that a single end-to-end test can’t catch everything going wrong along the way. But for the large majority of agents worth building, the pattern in this article — ground the critic, cap the loop, log the failure — is the durable version of self-correction. It’s the one that survives contact with a real production system instead of just a clean demo.", "url": "https://wpnews.pro/news/designing-ai-agents-that-can-self-correct", "canonical_source": "https://machinelearningmastery.com/designing-ai-agents-that-can-self-correct/", "published_at": "2026-08-06 11:13:29+00:00", "updated_at": "2026-08-09 09:00:07.231041+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-research", "ai-tools", "developer-tools"], "entities": ["Anthropic", "LangGraph", "pytest", "Stanford", "HumanEval", "CorrectBench", "MATH", "Reflexion"], "alternates": {"html": "https://wpnews.pro/news/designing-ai-agents-that-can-self-correct", "markdown": "https://wpnews.pro/news/designing-ai-agents-that-can-self-correct.md", "text": "https://wpnews.pro/news/designing-ai-agents-that-can-self-correct.txt", "jsonld": "https://wpnews.pro/news/designing-ai-agents-that-can-self-correct.jsonld"}}