{"slug": "my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes", "title": "\"My Agent Refused 96 Times\": Building Self-Editing Agents with Hard Failure Modes", "summary": "A senior ML engineer's benchmark of a production support agent revealed that the system refused to answer a valid question 96 times in a test set, but engineering review showed the model correctly identified insufficient or contradictory context in those cases, choosing refusal over hallucination. The article advocates for designing agents with hard failure modes, treating 'I don't know' as a first-class return type, and implementing verification gates to prevent confident hallucinations in critical systems.", "body_md": "*Originally published on tamiz.pro.*\n\nIn the early days of shipping LLM-based agents, we optimized for output volume. If the model could not find the answer, it often generated a plausible one anyway. This is the \"yes-man\" problem. In critical systems—financial auditing, code generation, or compliance checks—this creates a dangerous class of errors: *confident hallucinations*.\n\nRecently, a senior ML engineer shared a benchmark result from a production support agent: the system refused to answer a valid question 96 times in a test set. While the product team initially flagged this as a failure rate, the engineering review revealed the opposite. In 96% of those cases, the model correctly identified that the retrieved context was insufficient or contradictory. It chose the hard failure mode: refusal.\n\nThis is a pivotal shift in how we design autonomous systems. We are moving from probabilistic output to deterministic verification. This article explores how to architect agents with hard failure modes, treating \"I don't know\" as a first-class return type rather than an exception to be suppressed.\n\nLarge Language Models are trained on corpora where the goal is often to be helpful and coherent. Consequently, they exhibit a strong bias toward generating a response, even when the semantic signal is absent. This is known as *sycophancy* in evaluation contexts.\n\nWhen you build a RAG (Retrieval-Augmented Generation) agent, the default pipeline is:\n\nThe flaw lies in step 3. Without a gatekeeper, if the retrieved chunks contain irrelevant data (due to embedding similarity thresholds being too loose), the model will attempt to bridge the gap with internal parametric knowledge, leading to hallucination.\n\nConsider the following naive implementation, which represents the \"volume-first\" architecture:\n\n``` php\n# DANGEROUS: No verification gate\nasync def get_answer(question: str) -> str:\n    context = await retrieve_context(question)\n    prompt = f\"Context:\\n{context}\\n\\nQuestion: {question}\"\n    return await llm.complete(prompt)\n```\n\nIn a high-stakes environment, this function might return `\"The SQL query should use JOIN on table B\"`\n\neven if table B was never mentioned in the context. The agent has failed, but the API client sees a successful 200 OK with text. The error is silent.\n\nA **hard failure mode** is a deterministic exit path where the agent explicitly signals that it cannot fulfill the request based on verifiable criteria, rather than probabilistic guessing.\n\nThis concept borrows heavily from systems programming (e.g., Rust’s `Result<T, E>`\n\nor Go’s error handling). The agent must validate its own output before returning it to the user. If validation fails, the agent does not return a \"best effort\" string; it returns a structured `Failure`\n\nobject.\n\nThe term \"self-editing\" implies a secondary LLM call or a deterministic check that reviews the primary output. It is a form of *self-correction* or *refusal*.\n\nThere are two main types of self-editing gates:\n\nTo build an agent that \"refuses 96 times,\" we need a multi-step pipeline where the output of the generator is the input to a verifier. This is often called a **critic-loop** or **verification-based generation**.\n\nBefore the LLM even generates text, define the strict shape of the expected answer. If your agent extracts data, it must output JSON. Use a library like Pydantic or Zod to enforce this.\n\nIf the LLM outputs malformed JSON, the parser fails. This is not a hallucination; this is a syntax error. The agent should immediately fall back to a refusal state or retry with a stricter prompt, rather than attempting to parse a broken string.\n\nThis is the core mechanism. We must introduce a verification step that checks if the generated answer is *entailed* by the context.\n\nWe can implement this using a **Logit Bias** approach or a **Second-Pass LLM**. The second-pass approach is more robust for complex reasoning.\n\nInstead of trusting the first completion, we ask the LLM to evaluate its own work. The prompt structure changes from:\n\n`Q: [Question] A: [Answer]`\n\nTo:\n\n`Context: [Context] Q: [Question] A: [Draft Answer] Verify if A is fully supported by Context. Output TRUE or FALSE.`\n\nIf the verifier outputs `FALSE`\n\n, the agent enters the failure mode. It does not return the draft. It returns a standard refusal message.\n\nHere is a Python implementation of a self-editing agent using a verification loop:\n\n``` python\nfrom pydantic import BaseModel\nimport openai\nimport json\n\nclass VerificationResult(BaseModel):\n    is_supported: bool\n    reasoning: str\n\nclass AgentResponse(BaseModel):\n    answer: str | None\n    status: str # \"success\" or \"refused\"\n    reason: str | None\n\ndef verify_answer(context: str, question: str, draft: str) -> VerificationResult:\n    prompt = f\"\"\"\n    You are a rigorous fact-checker. \n\n    Context:\n    {context}\n\n    Question: {question}\n    Draft Answer: {draft}\n\n    Task: Determine if the Draft Answer is strictly supported by the Context.\n    - If the answer is in the context, output TRUE.\n    - If the answer is a hallucination or not mentioned, output FALSE.\n    - Do not use outside knowledge.\n\n    Return JSON: {{\"is_supported\": true/false, \"reasoning\": \"...\"}}\n    \"\"\"\n    response = openai.chat.completions.create(\n        model=\"gpt-4o-mini\", # Cost-effective verifier\n        messages=[{\"role\": \"user\", \"content\": prompt}],\n        response_format={\"type\": \"json_object\"}\n    )\n    return VerificationResult(**json.loads(response.choices[0].message.content))\n\nasync def self_editing_agent(question: str, context: str) -> AgentResponse:\n    # 1. Generate draft\n    draft_prompt = f\"Context:\\n{context}\\n\\nAnswer: {question}\"\n    draft_resp = openai.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": draft_prompt}]\n    )\n    draft_answer = draft_resp.choices[0].message.content\n\n    # 2. Verify\n    verdict = verify_answer(context, question, draft_answer)\n\n    # 3. Decide\n    if verdict.is_supported:\n        return AgentResponse(answer=draft_answer, status=\"success\", reason=None)\n    else:\n        # HARD FAILURE MODE: Refuse\n        return AgentResponse(\n            answer=None, \n            status=\"refused\", \n            reason=f\"Insufficient context to answer confidently: {verdict.reason}\"\n        )\n```\n\nWhen the agent refuses, the response must be explicit. In the example above, the status is `refused`\n\n. The frontend or downstream service should treat this differently from a success response.\n\nIf you have 96 refusals in a test suite, analyze them. Are they *true positives* (the question was indeed unanswerable)? If so, your system is working correctly. If they are *false positives* (the answer was in the context but the verifier missed it), you need to tune the verifier prompt or lower the temperature of the generator.\n\nA hard failure mode does not always mean immediate termination. In many systems, a \"refusal\" triggers a fallback strategy. This is where **self-editing** becomes powerful.\n\nIf the verifier rejects the answer, the agent can attempt to *re-generate* with modified constraints. For example:\n\nThis adds latency but drastically improves precision. For every 100 requests, you might accept 85, retry and accept 10, and refuse 5. The 96 refusals mentioned in the industry anecdotes often come from systems that skip the retry and go straight to refusal, which is actually safer.\n\nTraditional accuracy metrics are misleading here. You want to optimize for **Precision** (when the agent speaks, it is right) over **Recall** (the agent answers everything).\n\nThe \"96 refusals\" story is compelling because it highlights that the cost of a false positive in an AI agent is often orders of magnitude higher than the cost of a false negative (refusal). In a medical diagnosis bot, a false positive (hallucinating a treatment) is catastrophic. A false negative (refusing to answer) is merely inconvenient.\n\nBeyond the basic verifier loop, there are advanced patterns used in production-grade agents (like those at companies building autonomous coding assistants).\n\nIf your agent uses tools (APIs, databases), the *result* of the tool call is the ground truth. You can verify that the LLM's summary of the tool result matches the actual tool output.\n\nFor example, if the agent calls `get_balance(user_id)`\n\nand receives `0.00`\n\n, the LLM must not write \"The user has $100 in their account.\" A simple string comparison or checksum can verify this. This is a deterministic hard failure mode.\n\nEncourage the model to output its reasoning steps (CoT) before the final answer. The verifier can check the *reasoning* chain for logical gaps. If the reasoning is flawed but the answer is correct (lucky guess), the verifier should still reject it. This enforces *explainable* accuracy.\n\nDuring the self-editing loop, you can dynamically adjust the temperature. If the first generation is rejected, retry with `temperature=0`\n\nto minimize variance and force the model to stick closer to the context distribution.\n\nIf your verifier is just another LLM call, it can also hallucinate. It might falsely reject a correct answer. To mitigate this:\n\n`gpt-4o-mini`\n\nor `claude-3-haiku`\n\n).Self-editing adds at least one extra LLM round-trip. This doubles latency and cost.\n\nThe story of the agent refusing 96 times is a triumph of engineering integrity over product vanity. In the short term, a high refusal rate looks bad on a dashboard. In the long term, it builds trust. Users learn that when the agent speaks, they can trust it implicitly.\n\nBuilding self-editing agents with hard failure modes requires a shift in mindset: from generating content to *verifying correctness*. By implementing structured verification loops, enforcing schema strictness, and treating refusal as a valid state, we can build AI systems that are not just intelligent, but reliable.\n\nAs we push toward more autonomous agents, the ability to say \"no\" is the defining characteristic of robustness. The code samples and patterns provided here serve as a foundation for transitioning your agent from a probabilistic chatbot to a deterministic worker.\n\n**Q: How do I measure if my refusal rate is too high?**\n\nA: Establish a \"Golden Set\" of questions with known answerable/unanswerable labels. Calculate your **Refusal Accuracy**: (True Refusals / Total Refusals). If this is >95%, your refusal rate is healthy, regardless of the absolute number. If it is <80%, your verifier is too aggressive.\n\n**Q: Can I use this pattern with non-LLM components?**\n\nA: Yes. Hard failure modes apply to any stochastic system. If you have a heuristic classifier that sometimes confuses two classes, you can add a confidence threshold. If confidence < T, refuse. This is the same logic, just deterministic instead of LLM-based.\n\n**Q: What is the best model for the verifier?**\n\nA: The verifier does not need to be creative; it needs to be precise. Smaller models like `gpt-4o-mini`\n\n, `claude-3-haiku`\n\n, or even quantized local models like `llama-3.2-3b`\n\noften perform surprisingly well at verification tasks because they follow instructions closely without adding flair. Benchmark a few candidates on your specific data.", "url": "https://wpnews.pro/news/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes", "canonical_source": "https://dev.to/tamizuddin/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes-jii", "published_at": "2026-08-29 06:00:50+00:00", "updated_at": "2026-08-29 06:18:52.750033+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-safety", "developer-tools"], "entities": ["tamiz.pro"], "alternates": {"html": "https://wpnews.pro/news/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes", "markdown": "https://wpnews.pro/news/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes.md", "text": "https://wpnews.pro/news/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes.txt", "jsonld": "https://wpnews.pro/news/my-agent-refused-96-times-building-self-editing-agents-with-hard-failure-modes.jsonld"}}