{"slug": "agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses", "title": "Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses", "summary": "A new arXiv study (2608.26130v1) finds that LLM coding agents never request a second page of tool responses, and that improving the precision-at-1 rate (p1) of first-chunk selection does not systematically improve downstream accuracy. Across 500 SWE-bench Verified tasks and 4,800 LLM calls, a parameter-free keyword scorer raised p1 from 24.2% to 35.0% (+10.8 p.p., p=3.9×10^-8), but per-model accuracy deltas stayed under 3 percentage points and were not consistently signed. Adding four file-metadata signals to the keyword scorer hurt p1 by 4.8 p.p. (p=0.001).", "body_md": "arXiv:2608.26130v1 Announce Type: new\nAbstract: Coding agents built on large language models (LLMs), such as Claude Code, Cursor, OpenAI Codex, GitHub Copilot, and Aider, receive tool responses that routinely exceed the agent's per-turn token budget. The standard remedy, pagination, is available in every protocol that produced these responses; yet across the corpus of session logs from a public Model Context Protocol middleware we observed no agent-initiated requests for a second chunk. The first chunk is what the agent reads, so we ask how often the gold item (the one the agent needs) is placed first in it: the precision-at-1 rate $p_1$.\nIn a controlled offline benchmark we treat first-chunk selection as a 0/1 knapsack and compare six value functions on 500 SWE-bench Verified tasks, then test whether $p_1$ matters with a single-turn file-localisation probe on five language models (4,800 LLM calls; not an end-to-end resolve-rate test). Two pre-registered hypotheses did not hold and are our main findings. The central one is negative: raising $p_1$ does not systematically raise downstream accuracy. Per-model deltas stay under three percentage points (p.p.), are not consistently signed, and no model is significant; the agent recovers the gold from anywhere in the chunk, so what reaches its answer is first-chunk inclusion, not the gold's rank within it. The second: adding four file-metadata signals to a keyword scorer hurts $p_1$ by 4.8 p.p. (paired significance test, $p = 0.001$).\nA parameter-free keyword scorer does raise $p_1$, from a 24.2% baseline to 35.0% (+10.8 p.p., far beyond chance; $p = 3.9 \\times 10^{-8}$), and to 35.8% with a fallback to the tool's native ordering when no keyword matches. But by our central finding this is a rank-1 gain, and rank-1 is the part that does not reach the agent's answer: downstream accuracy does not move.", "url": "https://wpnews.pro/news/agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses", "canonical_source": "https://arxiv.org/abs/2608.26130", "published_at": "2026-08-28 04:00:00+00:00", "updated_at": "2026-08-28 04:20:21.879527+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["arXiv", "SWE-bench Verified", "Claude Code", "Cursor", "OpenAI Codex", "GitHub Copilot", "Aider", "Model Context Protocol"], "alternates": {"html": "https://wpnews.pro/news/agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses", "markdown": "https://wpnews.pro/news/agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses.md", "text": "https://wpnews.pro/news/agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses.txt", "jsonld": "https://wpnews.pro/news/agents-don-t-paginate-first-chunk-selection-for-llm-tool-responses.jsonld"}}