AI Agent Tool Pagination: Stop One Response From Swallowing the Context Window

A developer has published a pattern for paginated AI agent tool results that keeps a single response from consuming the context window, built around a typed result contract with a durable, tenant-scoped result store and a compact model-visible page summary. The contract exposes coverage.complete, hasMore, an opaque nextCursor, applied filters and server-generated summaries so agents can see when results are incomplete and choose a narrower follow-up retrieval instead of looping through every page. The writeup identifies three recurring production failures: first-page certainty, context flooding and blind looping.

An agent asks a search tool for invoices, tickets, or documents. The tool returns 20 tidy JSON objects. The model writes a confident answer. The answer can still be wrong. Those 20 objects may be page one of 2,000. Or they may be the only records the model can afford to read before its context becomes mostly old tool output. In both cases, a perfectly valid API response creates a bad decision. The fix is not telling the model to “remember pagination.” Give it a result contract that makes incompleteness visible, bounds how much data enters context, and lets the agent choose the next useful retrieval step. This guide shows a practical pattern for paginated tool results that works with ordinary HTTP APIs, internal tools, and MCP-backed workflows. Conventional APIs can assume a program will loop until nextCursor disappears. An agent cannot safely make that assumption. It has a task, a limited context window, and a tool response whose meaning is partly described in natural language. Three failure modes repeat in production: | Failure | What the agent sees | What goes wrong | |---|---|---| | First-page certainty | A plausible list of records | It answers as if there are no more matches. | | Context flooding | Hundreds of rows, fields, and snippets | Important instructions and evidence are pushed out of the working context. | | Blind looping | A continuation cursor | It fetches every page even when one aggregation or a better filter would answer the task. | Pagination is therefore a reasoning and interface-design problem, not just an API detail. The right question is not “How can the agent load everything?” It is “What is the smallest next slice of evidence that can change the answer?” That change in framing also improves user experience. A chat interface can show progress such as “I found 1,840 matches; I’m comparing the top accounts by open-ticket count” rather than silently issuing ten calls. The user can then refine the task before the system spends time and tokens browsing records that will not affect the conclusion. Keep the complete response in a durable, access-controlled result store. Put only a compact page summary in the model-visible response. Give the model a reference it can use to fetch a narrow follow-up slice. This architecture has four pieces: The result store is not long-term agent memory. Treat it as short-lived evidence for one task: tenant-scoped, expiry-bound, auditable, and deleted or redacted according to your data policy. This separation also makes retries safer. A transport retry can retrieve the same page by reference without asking the model to reconstruct a query or blindly repeat a broad search. Keep retrieval state deterministic; let model reasoning operate on a compact, explicitly scoped view of that state. Offset pagination is easy for a UI but fragile for changing data. Cursor pagination is usually safer because the server can preserve ordering and make the cursor opaque. The agent should never manufacture or inspect a cursor. More important than the cursor is the surrounding contract. A useful response says what the page proves and what it does not prove: type ToolPage<T = { resultRef: string; // opaque, tenant-scoped, short lived items: T ; // hard-capped, e.g. 10–25 records returnedCount: number; knownTotal?: number; // omit if expensive or misleading hasMore: boolean; nextCursor?: string; // opaque; valid only with resultRef sort: { field: string; direction: "asc" | "desc" }; appliedFilters: Record<string, string | number | boolean ; coverage: { complete: boolean; reason?: "more results" | "time budget" | "permission limited"; }; summary: string; // factual, generated by the server suggestedNext: Array< | "get next page" | "search within results" | "get aggregate" | "get record" ; }; coverage.complete matters. items.length < limit is not sufficient evidence that the task has complete data: permissions, timeouts, server caps, or a moving query can also shorten a page. The summary should be factual rather than interpretive. Good: “Showing 20 newest open tickets, sorted by updated time; more matching tickets exist.” Bad: “These are the most important tickets.” Let the model decide importance, but make the available evidence explicit. Paging every result is rarely the best plan. A search tool should offer alternatives that map to common user questions. For example, a request such as “Which customers had the most unresolved billing issues?” can usually be answered by an aggregate, then a narrow drill-down: js // First: a cheap grouped view, not 500 ticket rows. const groups = await tickets.aggregate { resultRef, groupBy: "customerId", metrics: "count" , filters: { status: "open", category: "billing" }, limit: 10, } ; // Then: inspect only the evidence behind the leading group. const details = await tickets.searchWithin { resultRef, customerId: groups.items 0 .customerId, fields: "id", "subject", "updatedAt", "priority" , limit: 10, } ; Design these moves deliberately: get aggregate answers counts, totals, ranges, and grouped comparisons. search within results finds a concept, record, or field inside a stored set. get record returns one complete record after the agent has selected an ID. get next page is for ordered browsing when order itself matters. This is also a useful dividing line for MCP. Protocol-level pagination can enumerate tools, prompts, and resources. Your business tool still needs its own domain-level result contract for searches, CRM records, logs, or catalog items. Do not rely on a prompt rule such as “continue until complete.” Give the runtime a small, deterministic continuation policy. It should consider the task, evidence quality, page count, and resource budget. function chooseNextStep input: { taskNeedsExhaustiveList: boolean; page: ToolPage<unknown ; pagesRead: number; maxPages: number; canAnswerFromCurrentEvidence: boolean; } { if input.canAnswerFromCurrentEvidence return "answer with scope"; if input.page.hasMore return "answer with scope"; if input.pagesRead = input.maxPages return "ask or narrow"; if input.taskNeedsExhaustiveList return "aggregate or search"; return "get next page"; } The final answer should carry the same scope honestly. If the tool only inspected the 50 newest records, say so. If it used an aggregate across every matching record and sampled five examples, say that too. Scope language is not a disclaimer added at the end; it is part of correct reasoning. Here is a simplified server implementation. Notice that the server owns the cursor and the result reference. It also limits fields before any data reaches the agent. js const PAGE SIZE = 20; export async function searchTickets input: { tenantId: string; query: string; cursor?: string; resultRef?: string; } { const state = input.resultRef ? await resultStore.load input.resultRef, input.tenantId : await resultStore.create { tenantId: input.tenantId, query: input.query } ; const page = await ticketRepo.search { tenantId: input.tenantId, query: state.query, after: input.cursor, limit: PAGE SIZE + 1, select: "id", "subject", "status", "priority", "updatedAt" , } ; const hasMore = page.rows.length PAGE SIZE; const items = page.rows.slice 0, PAGE SIZE ; const nextCursor = hasMore ? encodeCursor items.at -1 .updatedAt, items.at -1 .id : undefined; return { resultRef: state.ref, items, returnedCount: items.length, hasMore, nextCursor, sort: { field: "updatedAt", direction: "desc" as const }, appliedFilters: { query: state.query }, coverage: { complete: hasMore, reason: hasMore ? "more results" : undefined }, summary: Showing ${items.length} newest matches${hasMore ? "; more matches exist" : "; complete result set"}. , suggestedNext: hasMore ? "get next page", "search within results", "get aggregate" : "get record" , }; } In a real system, sign or encrypt cursors, bind resultRef to the actor and tenant, and expire both quickly. Never let a cursor from one query, user, or tenant open another result set. Every tool that can return a collection needs explicit limits. Pick them before you tune prompts: updatedAt, id so a record does not vanish or duplicate between pages. These limits make cost and correctness testable. They also prevent a user request such as “show me all activity” from turning into an unbounded retrieval loop. Unit tests for nextCursor are necessary but incomplete. Add workflow tests where the agent has to choose an information-gathering action. | Test case | Expected behavior | |---|---| | First page has more results | The answer says its scope or retrieves more evidence; it never claims a complete list. | | Aggregate answers the question | The agent uses the aggregate instead of walking every page. | | Cursor crosses tenant boundary | The tool rejects it without revealing whether records exist. | | Page limit is reached | The runtime narrows the query or asks a focused question. | | Underlying records change | The response preserves snapshot semantics or marks the result as potentially stale. | | Large text field appears | The allowlist omits it until the agent requests a specific record. | Log page count, result bytes, continuation reason, aggregate use, and final-answer scope. Those signals tell you whether pagination is reducing tool cost or merely hiding a retrieval problem. “Return everything and let the model summarize it.” This is expensive and makes the model less reliable as the output grows. Put raw records in the result store instead. “Always follow nextCursor .” A cursor is an option, not an instruction. An aggregate or a narrower filter is often the next best action. “Put pagination instructions in the tool description.” Keep the description, but enforce caps and state binding in code. A model can miss a sentence; the runtime should not. “Use page length to infer completeness.” A partial page can be caused by permissions or server limits. Return explicit coverage metadata. Only when the task needs an exhaustive, ordered list and the budget allows it. For rankings, counts, trends, or a single record, an aggregate or targeted search is normally better. Choose a small, measured cap—often 10 to 25 compact records—then test with your real schemas. The byte size and field selection matter more than the row count. Usually. Opaque cursors better support stable ordering and discourage the agent from inventing page numbers. They still need tenant binding, expiry, and explicit completeness metadata. MCP has pagination conventions for protocol list operations. A custom tool that searches business data should define its own response schema with a cursor, bounded items, and clear continuation semantics. Store the full result server-side, return a short factual summary plus a scoped result reference, and expose follow-up actions such as search, aggregate, and record lookup. State the evidence scope plainly: for example, “I reviewed the 50 most recently updated matching tickets” or “I used an aggregate over all matches and inspected five examples.” Pagination is successful when the agent can explain both what it found and how complete that evidence is. Build that truth into the tool response, and the context window stops being a dumping ground for records it never needed.