Context Length Exceeded: Reduce the Request and Verify the Result

A developer at BetterToken explains how to fix 'Context length exceeded' errors by accounting for the total context budget rather than randomly deleting prompt parts. The approach involves measuring each request component, removing mechanical duplicates and irrelevant history, and verifying that required facts and answer completeness remain intact after resending.

Fix Context length exceeded by accounting for the total budget, not by randomly deleting half the prompt. Identify the model and measure each request component first. Then remove mechanical duplicates, irrelevant history, and heavy tool results. After resending, verify both that the error is gone and that required facts and answer completeness remain intact. Context is the working memory of one request, not just the latest user prompt. In simplified form: system instructions + conversation history + current message + images and documents + tool definitions + tool results + output / thinking budget = total context usage Anthropic's context-window documentation https://platform.claude.com/docs/en/build-with-claude/context-windows explicitly includes the system prompt, all messages, images, documents, tool definitions, tool results, and generated response. Prompt caching changes the cost of reused tokens, but it does not remove them from the window. Do not hard-code a “universal limit”: the context window and overflow behavior depend on the selected model and API. Check the current model card on the day you configure it. | Component | What to measure | Safe reduction | |---|---|---| | System instructions | Repeated rules and long examples | Merge duplicates and retain mandatory constraints | | History | Tokens by message and old branches | Remove irrelevant branches or replace them with a verifiable summary | | Files and RAG fragments | Each document's size, duplicates, and low-relevance chunks | Lower top k , deduplicate, and pass only the required sections | | Tool definitions | Unused tools and long descriptions | Pass only the tools needed for the current step | | Tool results | Complete JSON, logs, HTML, base64, and repeated responses | Keep required fields, links, and identifiers; store large data outside the prompt | | Output budget | max tokens and thinking budget | Reserve a realistic amount or split a large result into stages | Claude has a separate Token Counting API https://platform.claude.com/docs/en/build-with-claude/token-counting that accounts for messages and tools before a request is sent. For another provider, use its own counter if available. A local tokenizer is useful for early warnings, but its estimate is not a guaranteed server-side count for a different model. For a request sent through BetterToken, the Dashboard lets you compare input, output, and cache tokens after a test. The current API documentation https://docs.bettertoken.ai/api-reference/introduction?utm source=devto&utm medium=syndication&utm campaign=context-length-exceeded-kak-ispravit-oshibku&utm content=context-length-exceeded-reduce-request-verify helps verify the contract, but the Dashboard does not show the full prompt or replace preflight token counting; it is a post-request budget check. Look for repeated system rules, the same file in several messages, duplicate RAG chunks, repeated schemas, and complete logs pasted more than once. This is the safest stage because it reduces size without changing the task. Separate long-lived facts from the temporary flow of the conversation. Preserve goals, accepted decisions, mandatory constraints, and open questions. Remove old reasoning, rejected alternatives, and tool results that have already been processed, or compact them into a structured summary. A bad summary says “we discussed the integration.” A useful one records the chosen endpoint, schema version, accepted constraints, confirmed facts, and next step. Pass relevant sections instead of an entire document. In a tool result, keep the fields required by the next step instead of the full HTTP response or log. Do not discard sources or mandatory data merely to make the request fit; split the work into several verifiable stages instead. Input and output share the total budget. If the request nearly fills the window, the model may not have enough room for a complete answer. Reduce optional input, set a realistic output budget, or split the result into parts. Remove mechanical duplicates before critical facts. A larger-context model may be the right choice for a document that cannot be split safely. Moving to a larger window without removing duplicates only postpones the next failure and may reduce information density. components = count by section request estimated input = sum components reserved output = requested output budget if estimated input + reserved output approaches current model window: remove exact duplicates drop irrelevant history compact tool results and retrieved chunks count again send only after required facts and constraints remain present approaches is intentionally not replaced by a fixed percentage. The necessary headroom depends on counter accuracy, the model, thinking, and the specific API's behavior. Compare the new request against this checklist: context length exceeded or prompt is too long .If the error disappears but the model forgets a key constraint, the fix failed. Restore the mandatory block and free space by removing less relevant history or a heavy tool result. If the answer is cut off, inspect the output budget separately; it is another part of the same total window. The working sequence is: identify the model → count components → remove exact duplicates → extract irrelevant history → compact files and tool results → leave room for the answer → resend → verify quality. This addresses the cause without turning context into an arbitrarily truncated set of facts. Originally published on the BetterToken blog. BetterToken provides pay-as-you-go access to AI model APIs through OpenAI-compatible and Anthropic-compatible endpoints — useful if you are wiring Claude Code, Codex, or your own tooling to a custom base URL. See the docs https://docs.bettertoken.ai/?utm source=devto&utm medium=syndication&utm campaign=context-length-exceeded-kak-ispravit-oshibku&utm content=context-length-exceeded-reduce-request-verify to get started.