Your Agent Returned 200 and Did Nothing: Detecting Silent Failures with Token Counts

An engineer has identified a common but overlooked failure mode in LLM agents: silent failures where an agent returns HTTP 200 with tokens spent but produces no usable output. The engineer proposes using token counts to detect these cases, offering a heuristic and code snippet to flag empty or undershooting responses. The approach aims to improve observability for agent pipelines, where standard monitoring misses such quiet breakdowns.

Your agent returned HTTP 200, zero errors in the logs, and then did nothing. The failure wasn't loud — it was silent. This happens more than you'd think. An agent invokes a model, gets back a response, processes it, returns success. But somewhere in that flow the actual output — the thing that was supposed to happen — vanished. No exception. No timeout. Just an empty response where a decision or action should have been. Standard monitoring catches loud failures: connection timeouts, rate limits, exceptions. It misses the silent ones. Here's why, and how to spot them. When an LLM agent runs, three things happen: Most observability tools watch 1. Some logging catches parts of 2. Almost nobody watches 3 in a way that actually matters. The mechanical truth: HTTP 200 and a nonzero token count do not guarantee real output. An agent asks Claude or GPT, or any model to do something that brushes up against a boundary — political, violent, ambiguous. The refusal policy triggers. It comes back: HTTP 200 input tokens: 450 output tokens: 127 response: "" // or a short refusal like " No response " The call succeeded. Tokens were spent. Nothing usable came out. If your downstream logic checks if response: do something , you've got a silent failure. Less common, but real: the model returns a completion that's all whitespace or filler with no actual content. Output tokens go up. Meaning doesn't. HTTP 200 input tokens: 300 output tokens: 89 response: " \n\n " // whitespace, no content Tokens spent, call succeeded, nothing happened. An agent calls a tool — database lookup, API call, calculation. The tool errors or returns nothing. The model, correctly, generates a response explaining it couldn't do the thing. Except your downstream code expected a value, not an apology. HTTP 200 input tokens: 520 output tokens: 212 response: "I attempted to fetch the user but the database returned no match." expected: { user id: 12345, status: "active" } The agent succeeded at speaking. It failed at acting. Output tokens measure how much the model actually generated, independent of quality. A genuinely empty response will have an output token count near zero. A response that merely looks empty — but has some content, even noise — will burn tokens. That's why token count matters more than HTTP status: Rough heuristic: if output tokens are suspiciously low for the task, or nonzero but the response is empty/whitespace, you're looking at a silent failure. python def detect silent failure agent run : http ok = agent run.status code == 200 tokens spent = agent run.output tokens 0 has content = len agent run.response.strip min threshold e.g. 10 chars if http ok and tokens spent and not has content: return "SILENT FAILURE" if http ok and tokens spent < expected tokens - threshold: return "UNDERSHOOTING" return "OK" The key is comparing actual output against token count. If a model spent 150 output tokens and your response is empty, something broke between generation and delivery. A crash tells you something broke — you debug it. A "success" that produced nothing is worse: it quietly corrupts pipelines, skips records, or leaves a user waiting on a response that's never coming. HTTP status and token count both look fine, so plain logging won't catch it. You need something that checks whether the response actually matches what was supposed to happen. If you're running agents and not instrumenting for this, you're flying blind. Three things help: The green light doesn't mean it's working. The token count tells you if anyone's actually home.