# Agentic RAG fails before the reasoning starts > Source: > Published: 2026-08-07 17:00:00+00:00 [ Agent Reliability Agent Drift ](/blog/ai-agent-reliability-is-a-context-problem/) ### AI agent reliability is a context problem Key takeaway Agentic RAG failure modes split into two independent bugs that final-answer accuracy blends into one number. A 2026 study of 12,000 paired agent trajectories across HotpotQA, 2WikiMultiHopQA, and MuSiQue found agents routinely retrieve relevant evidence and then answer without reading it: on MuSiQue, 617 of 1,000 trajectories finalized with zero read calls. A deterministic rule that rejects finalization until the agent has read something recovered 14.9 to 19.9 accuracy points on those trajectories, with no change to the model, the retriever, or the reasoning budget. Delivering the identical text without requiring the read action recovered only a minority of the gain and went negative on one dataset, so what fixes the failure is the retrieval protocol rather than the amount of context. On the MuSiQue multi-hop benchmark, 617 out of 1,000 agent trajectories ended with the agent writing a final answer without opening a single retrieved document. The searches ran. The snippets came back. The agent read none of them and answered anyway. That number comes from [Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG](https://arxiv.org/abs/2608.02011), an August 2026 study by Daeyoung Roh and Donghee Han that logged 12,000 paired agent trajectories across three multi-hop question answering datasets. Its argument is that the [agentic retrieval](/glossary/agentic-retrieval/) community has been debugging the wrong layer. When an agent gets a multi-hop question wrong, the instinct is to blame the retriever or the model’s reasoning. The paper’s measurements say a large share of those failures happen earlier than either, in the agent’s control flow, before evidence-conditioned reasoning is ever tested. The fix they propose is close to embarrassing in its simplicity, and it works. The failure is procedural rather than cognitive: an agent issues a search, receives snippets that contain the answer, and emits a final answer without calling read on any of them. The paper calls these **discipline failures** and defines three observable subtypes, all measured deterministically off saved traces rather than inferred from model behavior. | Subtype | Definition | |---|---| | No-read final | The agent emits a final answer with `read_count = 0` | | Snippet-only final | The answer entity appears only in search snippets, never in a read passage | | Low-evidence final | Named entities from the question are covered in read chunks at under 80% | The authors are careful about what they are claiming. In their words, these labels “are not claims about latent model cognition; they are deterministic measurements over saved agent traces.” Nothing here depends on interpreting what the model was thinking. Either a read call happened before the final answer or it did not. The setup is deliberately plain. A two-tool interface, `search` and `read` , over a hybrid retriever combining BM25 with Qwen3-Embedding-0.6B, fused by reciprocal rank and returning the top five results to the agent. Up to ten loop steps per question. That is close to the minimum viable [agentic retrieval](/glossary/agentic-retrieval/) loop, which is what makes the failure rate notable: there is no elaborate scaffold here to blame. Discipline failures and reasoning failures co-occur in only 11.2 to 13.1% of cases, which establishes them as two distinct defects rather than two views of the same one. The paper’s second failure category is **post-gold-read failure**: the agent did open at least one chunk containing supporting evidence, and still produced a wrong answer. That is a genuine reasoning error, and it is the thing most [RAG](/glossary/rag/) evaluation is implicitly designed to measure. The overlap between the two categories stays far below 60% across every entity extractor the authors tried. If the two indicators were really tracking one underlying problem, they would rise and fall together. They do not. This is the finding with the most immediate practical weight, because it indicts the standard metric. Final-answer accuracy sums both failure types into one number, and the two want opposite interventions. A post-gold-read failure argues for a stronger model or better prompting. A discipline failure argues for neither, because the model was never the constraint. Improving the retriever does nothing for an agent that will not read what the retriever returns, which is the same shape of problem as [agents that never call the MCP servers you installed for them](/blog/why-agents-skip-installed-mcp-servers/): the capability is present and unused. Skipped-reading rates vary from 4.5% to 57.0% depending on the model and the reasoning effort, with the worst cases on the hardest dataset. The paper reports discipline-failure rates across three controller configurations and three datasets: | Model | HotpotQA | 2WikiMultiHopQA | MuSiQue | |---|---|---|---| | gpt-4o-mini | 7.6% | 12.8% | not reported | | gpt-5-mini, minimal reasoning | 13.3% | 22.1% | 57.0% | | gpt-5-mini, medium reasoning | 4.5% | 5.7% | 21.7% | Two patterns fall out. First, the rate climbs with question difficulty. MuSiQue is the hardest of the three, requiring the longest evidence chains, and it is where agents skip reading most. That is the wrong direction: the questions that most need careful evidence gathering are the ones getting the least. Second, medium reasoning effort roughly halves the rate compared to minimal. That looks like a straightforward argument for turning reasoning up, and it is not, for reasons the paper gets to later. A deterministic environment rule that rejects finalization until the agent has read something recovers 14.9 to 19.9 accuracy points on the trajectories that would otherwise have skipped reading. The authors call it **Read-Gate**, and its entire implementation is a guard in the environment, not a change to the agent. When the agent emits a final action with `read_count = 0` , the environment rejects the action and returns a corrective observation: “You attempted to answer before reading any evidence. You MUST call read(evidence_id) on a promising chunk before giving the final answer.” The loop then continues within the same ten-step budget. The retriever, model weights, decoding temperature, reasoning budget, and judge are all untouched. On the subset of questions where the baseline agent read nothing: | Dataset | Trajectories | Zero-read accuracy | Forced-read accuracy | Gain | |---|---|---|---|---| | HotpotQA | 215 | 58.1% | 73.0% | +14.9 pp | | 2WikiMultiHopQA | 261 | 42.1% | 62.1% | +19.9 pp | | MuSiQue | 617 | 22.5% | 37.4% | +14.9 pp | Across the full 1,000-question sets, where most trajectories were already reading and therefore unaffected, the gains dilute to +3.2, +5.3, and +9.4 points. That last figure is worth sitting with. On MuSiQue, one environment-level guard, with no model change and no retrieval change, moved end-to-end accuracy from 34.2% to 43.6%. Injecting identical chunk text directly into context, instead of requiring the agent to read it, recovers only a minority of the gain and goes negative on one dataset. This is the paper’s control condition, and it is the result that turns an evaluation finding into a context engineering one. | Dataset | Baseline | Read-Gate | Context injection | |---|---|---|---| | HotpotQA | 79.6% | +3.2 pp | −0.1 pp | | 2WikiMultiHopQA | 64.4% | +5.3 pp | −7.4 pp | | MuSiQue | 34.2% | +9.4 pp | +3.9 pp | The agent receives the same chunk content in both arms. The only difference is whether that content arrived because the agent asked for it, mid-loop, or because the harness pushed it in. Pushing it in was worse than doing nothing at all on 2Wiki, and roughly break-even on HotpotQA. The straightforward reading is that the benefit is not the tokens. It is what the read step does to the agent’s trajectory: it forces one more turn in the loop, with the evidence arriving as a response to the agent’s own action, at the point in the sequence where the agent was about to commit. That cuts against the reflex to solve grounding problems by putting more of the corpus into the window, which is the same conclusion arrived at from the opposite direction by [the long context versus RAG data](/blog/long-context-vs-rag-what-the-data-shows/) and by [the pruning regime map](/blog/context-pruning-helps-agents-until-it-doesnt/). Volume of available context is not the operative variable. Increasing a model’s internal reasoning budget increases the rate at which it answers without reading anything, across every dataset tested. The authors ran Gemini 2.5 Flash with the thinking budget off and at 1024 tokens, paired on 1,000 questions per dataset: | Dataset | Zero-read rate change | Net correct answers | McNemar p | |---|---|---|---| | HotpotQA | +5.7 pp | −44 | 0.0025 | | 2WikiMultiHopQA | +24.8 pp | −67 | 0.0001 | | MuSiQue | +42.6 pp | −73 | 3.1x10⁻⁵ | All three are statistically significant, and all three go the wrong way on net accuracy. On MuSiQue, granting the model more room to think made it 42.6 points more likely to skip the evidence entirely. The authors do not propose a mechanism. They report the effect and note only that it is largest on MuSiQue and smallest on HotpotQA, “suggesting that the effect may be more pronounced when answers require longer evidence chains,” and that hidden deliberation and evidence inspection “can vary independently.” The tempting explanation, that deliberation raises the model’s confidence that it already knows the answer, is not something this study tests. What it does establish is that the two dials move separately, which sits alongside the earlier observation that gpt-5-mini at medium reasoning had lower discipline-failure rates than at minimal. The two are not in conflict so much as a warning that reasoning effort is not a monotonic dial for grounding, and that its effect on retrieval discipline has to be measured per model rather than assumed. The practical lesson is that retrieval grounding is a protocol property, not a payload property, and it needs to be enforced by the environment rather than requested in a prompt. Read-Gate works because it is a hard constraint the agent cannot talk its way past. The context-injection arm failed because it tried to achieve the same end by changing what the agent was given rather than what it was required to do. That reframes a chunk of [context engineering](/glossary/context-engineering/) work. Much of the discipline is about what goes into the window and in what shape. This paper is about the sequence of actions that puts it there, and it finds that sequence carries accuracy the payload does not. The failure is also interface-shaped. It exists because `search` returns previews and `read` returns bodies, which is why “snippet-only final” is one of the three subtypes: there is a gap between what the agent has seen and what it could see, and it can answer from the wrong side of it. Read-Gate closes that gap from outside, with a harness rule the agent does not know is there until it trips. The other place to close it is inside the result. If a search result declares what sits behind it, how many adjacent chunks exist in the same source, which related entries and of what kind, then “do I have enough?” becomes a question the agent can answer from what it is already holding, instead of one a harness has to answer on its behalf. [Wire](/why-wire/retrieval-benchmarks/) containers return [those hints on every wire_search match](/blog/one-job-per-tool-mcp-agent-retrieval/). Neither route guarantees the agent goes further. The difference is whether going further requires it to guess first. Concretely, three things follow for anything running an agentic retrieval loop in production: Read-Gate produces no benefit, and can subtract several points, on agents that were already reading. At medium reasoning effort the same constraint measured +0.0 on HotpotQA, −7.0 on 2WikiMultiHopQA, and −4.0 on MuSiQue. The authors predicted this in advance: the intervention only pays where residual discipline error is high, and where it is low the enforcement turn is pure overhead against a fixed step budget. There are other limits worth stating. The controllers are drawn from two model families, with Gemini 2.5 Flash and Qwen2.5-Instruct at 3B, 7B, and 14B used as cross-family checks rather than as primary conditions. Correctness is scored by an LLM judge, gpt-5-mini at temperature 0.0, though it was cross-validated against Gemini 2.5 Pro on a stratified sample of 450 with a kappa of 0.924, which is a stronger validation than most papers in this area bother with. And all three datasets are multi-hop question answering over Wikipedia-derived corpora, so the transfer to open-web or private-corpus agents is an assumption, not a result. The headline number is 617 of 1,000, and the reason it matters is not its size but its category. Those trajectories did not fail because the index was bad or the model was weak. They failed because the agent skipped a step, and every metric in common use scored that identically to a reasoning error. If you run an agentic retrieval loop, the cheapest useful thing you can do this week is instrument read counts and see what your own rate is. If it is high, an environment guard is worth trying before you touch the retriever or the model, because it is the intervention with the shortest path from cause to effect. If it is low, leave it alone, because the same guard costs points on agents that were already disciplined. And treat reasoning effort as something to measure against grounding rather than something to turn up, given that more thinking budget made skipping worse on every dataset tested. The broader point is that giving a model control over retrieval hands it a procedure it can get wrong, in a way that looks exactly like getting the answer wrong. Those need separate names and separate measurements before they can have separate fixes. Sources: [Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG (arXiv:2608.02011)](https://arxiv.org/abs/2608.02011) · [HotpotQA (arXiv:1809.09600)](https://arxiv.org/abs/1809.09600) · [Constructing A Multi-hop QA Dataset, 2WikiMultiHopQA (arXiv:2011.01060)](https://arxiv.org/abs/2011.01060) · [MuSiQue: Multihop Questions via Single-hop Question Composition (arXiv:2108.00573)](https://arxiv.org/abs/2108.00573) Wire transforms your documents into structured, AI-optimized context containers. Upload files, get MCP tools instantly.