{"slug": "your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token", "title": "Your Agent Returned 200 and Did Nothing: Detecting Silent Failures with Token Counts", "summary": "An engineer has identified a common but overlooked failure mode in LLM agents: silent failures where an agent returns HTTP 200 with tokens spent but produces no usable output. The engineer proposes using token counts to detect these cases, offering a heuristic and code snippet to flag empty or undershooting responses. The approach aims to improve observability for agent pipelines, where standard monitoring misses such quiet breakdowns.", "body_md": "Your agent returned HTTP 200, zero errors in the logs, and then did nothing. The failure wasn't loud — it was silent.\n\nThis happens more than you'd think. An agent invokes a model, gets back a response, processes it, returns success. But somewhere in that flow the actual output — the thing that was supposed to happen — vanished. No exception. No timeout. Just an empty response where a decision or action should have been.\n\nStandard monitoring catches loud failures: connection timeouts, rate limits, exceptions. It misses the silent ones. Here's why, and how to spot them.\n\nWhen an LLM agent runs, three things happen:\n\nMost observability tools watch #1. Some logging catches parts of #2. Almost nobody watches #3 in a way that actually matters.\n\nThe mechanical truth: HTTP 200 and a nonzero token count do not guarantee real output.\n\nAn agent asks Claude (or GPT, or any model) to do something that brushes up against a boundary — political, violent, ambiguous. The refusal policy triggers. It comes back:\n\n```\nHTTP 200\ninput_tokens: 450\noutput_tokens: 127\nresponse: \"\" // or a short refusal like \"[No response]\"\n```\n\nThe call succeeded. Tokens were spent. Nothing usable came out. If your downstream logic checks `if response: do_something()`\n\n, you've got a silent failure.\n\nLess common, but real: the model returns a completion that's all whitespace or filler with no actual content. Output tokens go up. Meaning doesn't.\n\n```\nHTTP 200\ninput_tokens: 300\noutput_tokens: 89\nresponse: \" \\n\\n \" // whitespace, no content\n```\n\nTokens spent, call succeeded, nothing happened.\n\nAn agent calls a tool — database lookup, API call, calculation. The tool errors or returns nothing. The model, correctly, generates a response explaining it couldn't do the thing. Except your downstream code expected a value, not an apology.\n\n```\nHTTP 200\ninput_tokens: 520\noutput_tokens: 212\nresponse: \"I attempted to fetch the user but the database returned no match.\"\nexpected: { user_id: 12345, status: \"active\" }\n```\n\nThe agent succeeded at speaking. It failed at acting.\n\nOutput tokens measure how much the model actually generated, independent of quality. A genuinely empty response will have an output token count near zero. A response that merely looks empty — but has some content, even noise — will burn tokens.\n\nThat's why token count matters more than HTTP status:\n\nRough heuristic: if output tokens are suspiciously low for the task, or nonzero but the response is empty/whitespace, you're looking at a silent failure.\n\n``` python\ndef detect_silent_failure(agent_run):\n    http_ok = agent_run.status_code == 200\n    tokens_spent = agent_run.output_tokens > 0\n    has_content = len(agent_run.response.strip()) > min_threshold  # e.g. 10 chars\n\n    if http_ok and tokens_spent and not has_content:\n        return \"SILENT_FAILURE\"\n    if http_ok and tokens_spent < expected_tokens - threshold:\n        return \"UNDERSHOOTING\"\n    return \"OK\"\n```\n\nThe key is comparing actual output against token count. If a model spent 150 output tokens and your response is empty, something broke between generation and delivery.\n\nA crash tells you something broke — you debug it. A \"success\" that produced nothing is worse: it quietly corrupts pipelines, skips records, or leaves a user waiting on a response that's never coming. HTTP status and token count both look fine, so plain logging won't catch it. You need something that checks whether the response actually matches what was supposed to happen.\n\nIf you're running agents and not instrumenting for this, you're flying blind. Three things help:\n\nThe green light doesn't mean it's working. The token count tells you if anyone's actually home.", "url": "https://wpnews.pro/news/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token", "canonical_source": "https://dev.to/opsveritas/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token-counts-1nc2", "published_at": "2026-08-03 13:02:00+00:00", "updated_at": "2026-08-03 13:15:56.134024+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["Claude", "GPT"], "alternates": {"html": "https://wpnews.pro/news/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token", "markdown": "https://wpnews.pro/news/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token.md", "text": "https://wpnews.pro/news/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token.txt", "jsonld": "https://wpnews.pro/news/your-agent-returned-200-and-did-nothing-detecting-silent-failures-with-token.jsonld"}}