# A Bigger Context Window Is Not More Memory: Five Token Myths and a Budget Gate > Source: > Published: 2026-10-27 00:23:25+00:00 Your counter says 12k tokens. The API returns a 400. Which one is lying? Probably neither. They are counting different things. One of them is counting your string. The other is counting what the model actually receives. I keep seeing the same five assumptions in code review. Each one feels safe. Each one fails quietly, or fails loudly at 2 a.m. Here is the FAQ, the evidence you can collect yourself, and a small gate you can run in CI. This is a rule of thumb for English prose. Vendors describe it that way. It is not a unit conversion. The ratio moves with content. Code, JSON, CJK text, emoji, and long identifiers all drift. Whitespace and punctuation get their own pieces. So does anything the tokenizer has never seen. **Corrected model:** characters are a hint. Tokens are what the model bills and truncates. You do not need to trust my ratio. Measure yours. Grab 20 real prompts from your logs. Count them with the tokenizer your provider actually ships. Then compare against `len(text) / 4`. Different tokenizers disagree with each other too. `tiktoken` encodings are one family (see [the tiktoken repo](https://github.com/openai/tiktoken)). Other model families use other vocabularies. A prompt that fits under one tokenizer can overflow under another. So: which tokenizer does your budget check use? It is at least three numbers wearing a trench coat. Input plus output usually share one window. Reserve the answer before you count the prompt. Most overflow bugs I read are really "nobody subtracted `max_tokens`". Tool schemas are the sneaky part. A generous function signature block can cost more than your whole user message. It is sent on every turn. **Corrected model:** budget = window − reserved output − template overhead. Then subtract tool schema cost, every turn. Long-context evaluations keep reporting uneven recall by position. The well-known result is ["Lost in the Middle"](https://arxiv.org/abs/2307.03172). Details placed mid-prompt get retrieved less reliably than details at the edges. That is not a reason to panic. It is a reason to stop stuffing. **Corrected model:** context is capacity, not attention. A 200k window does not mean 200k of useful signal. Placement and redundancy matter more than volume. Practical translation: put the constraint you cannot afford to lose near the top. Put the task near the bottom. Do not bury the schema between two chatty tool results. Old messages carry two things. Conversation, and contract. When you slice `messages[2:]`, you may delete the system constraints. You may also orphan a tool call. Many APIs reject an assistant `tool_calls` block that has no matching `tool` result, or the reverse. That is the 400 you cannot reproduce locally. Your unit test never had the tool pair in the middle. **Corrected model:** truncation is a structural edit, not a string edit. Validate pairing after every cut. Sometimes something upstream trims. A proxy, a gateway, a retry wrapper, an SDK default. The request succeeds. The answer misses the paragraph you cared about. You cannot detect that from the response alone. You can detect it from the request you sent. **Corrected model:** log the estimate at send time. If the answer looks blind, diff the budget against the prompt you thought you sent. Here is the artifact. It estimates prompt cost, reserves answer space, and checks tool-call pairing. It exits non-zero when the prompt does not fit. Treat the built-in counter as a conservative estimate. Swap in a real tokenizer when you have one. ``` bash #!/usr/bin/env python3 """context_gate.py - fail fast when a prompt no longer fits the budget.""" import json import re import sys from dataclasses import dataclass def estimate_tokens(text: str) -> int: # Deliberately conservative placeholder. Never exact. Swap in a real tokenizer. words = len(re.findall(r"\w+|[^\w\s]", text)) chars = (len(text) + 3) // 4 return max(words, chars) @dataclass class Budget: context: int # total window, from the provider docs reserve: int # space kept for the answer overhead: int = 8 # role markers / separators per message def count_prompt(messages, budget: Budget, counter=estimate_tokens) -> int: total = 0 for m in messages: total += counter(str(m.get("content") or "")) + budget.overhead for call in m.get("tool_calls") or []: total += counter(json.dumps(call.get("function", {}))) + budget.overhead return total def unpaired_tool_calls(messages): pending, problems = [], [] for i, m in enumerate(messages): for call in m.get("tool_calls") or []: pending.append(call.get("id")) if m.get("role") == "tool": cid = m.get("tool_call_id") if cid in pending: pending.remove(cid) else: problems.append(f"message {i}: result for unknown call {cid}") problems += [f"call {c}: no result" for c in pending] return problems def check(messages, budget: Budget, counter=estimate_tokens): used = count_prompt(messages, budget, counter) allowed = budget.context - budget.reserve return { "prompt_tokens_est": used, "prompt_budget": allowed, "answer_reserve": budget.reserve, "fits": used <= allowed, "pairing_problems": unpaired_tool_calls(messages), } if __name__ == "__main__": path, context, reserve = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]) with open(path) as f: payload = json.load(f) report = check(payload["messages"], Budget(context=context, reserve=reserve)) print(json.dumps(report, indent=2)) sys.exit(0 if report["fits"] and not report["pairing_problems"] else 1) ``` Run it against a captured request: ``` python3 context_gate.py prompt.json 32768 1024 ``` A failing run looks like this: ``` { "prompt_tokens_est": 31895, "prompt_budget": 31744, "answer_reserve": 1024, "fits": false, "pairing_problems": [] } ``` The `32768` is a flag you supply. Read the real number from your provider's docs. Do not copy mine. `prompt_tokens_est` next to every response you store. Step 5 is the one people skip. Then the incident review has no numbers. I run this preflight on MonkeyCode's free server option so the check travels with the job instead of living on my laptop. I also point it at free model access to compare estimates across more than one tokenizer family. That comparison is the whole point — one counter proves nothing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. When the gate says no, do not just slice. Pick the cheapest effective edit. | Symptom | First cut | Why | |---|---|---| | Tool schema dominates | Prune optional params, shorten descriptions | Paid every turn, not once | | Long history, short task | Summarize turns older than N | Keeps the contract intact | | One huge document | Move to retrieval, pass top-k | Placement beats volume | | Duplicated instructions | Keep one canonical system block | Redundancy is not safety | | Answer keeps getting cut | Raise reserve before trimming prompt | Reserve is the real ceiling | Context is a budget with a shape. Tokens vary by tokenizer. Output competes with input. Placement beats stuffing. Truncation is structural. And anything upstream can trim silently. So the next time a 400 lands on your desk, what will you show the reviewer: a guess about characters, or a gate that printed the number?