{"slug": "the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag", "title": "The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering", "summary": "A study posted to arXiv (2610.26951v1) introduces evidence-based generation (EBG), a method requiring models to return answers with line-level citations, plus a joint Ans+Evid metric that scores correctness and grounding together. Testing ten proprietary frontier and three open-weight LLMs on reformulated benchmarks — MultiHop-RAG, QASPER-MP, a synthetic two-hop haystack, and an evidence-augmented NoLiMa spanning 10K to 1M characters — the authors found retrieval, not reading, is the bottleneck: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposed grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening Ans-versus-Ans+Evid gap when context contradicts parametric knowledge; data and code are at github.com/debobanerjee/LC-EBG.", "body_md": "arXiv:2610.26951v1 Announce Type: new \nAbstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks- MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K-1M characters. Across ten proprietary frontier and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.", "url": "https://wpnews.pro/news/the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag", "canonical_source": "https://arxiv.org/abs/2610.26951", "published_at": "2026-10-21 04:00:00+00:00", "updated_at": "2026-10-21 04:17:58.214441+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "natural-language-processing"], "entities": ["arXiv", "MultiHop-RAG", "QASPER-MP", "NoLiMa", "evidence-based generation", "Ans+Evid"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag", "markdown": "https://wpnews.pro/news/the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag.md", "text": "https://wpnews.pro/news/the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag.txt", "jsonld": "https://wpnews.pro/news/the-bottleneck-is-retrieval-not-reading-diagnosing-evidence-based-generation-rag.jsonld"}}