Tool Search Defers Tools to Save Tokens

Claude Code's tool search system defers MCP tool definitions once they exceed a token threshold, replacing them with a single ToolSearch tool the model calls to load definitions on demand, according to the article. In one example, a setup with GitHub, Slack and Jira MCP servers totaling 147 tools drops each API call from roughly 90,000 tokens of tool definitions to about 15,000, at a cost of one extra turn and roughly 200 tokens per discovery, saving about 1.5 million tokens over a 20-turn conversation. The design fails closed: unknown models, proxy gateways or missing token counts fall back to loading all tools rather than silently hiding them.

How Tool Search Defers Tools to Save Tokens How Tool Search Defers Tools to Save Tokens Claude Code can use dozens of built-in tools and an unlimited number of MCP tools. Every tool the model might call needs a definition — a name, description, and JSON schema — sent with each API request. A single MCP tool definition might cost 200–800 tokens. Connect three MCP servers with 50 tools each, and you’re burning 60,000 tokens on tool definitions alone. Every turn. Before the model reads a single message. That’s not sustainable. A 200K context window that loses 30% to tool definitions before the conversation starts is a bad experience. The model has less room to think, compaction triggers sooner, and cost per turn climbs. The naive solution is obvious: don’t send tools the model doesn’t need. But which tools does the model need? You don’t know until it tries to use one. And if the tool definition isn’t there when the model tries to call it, the call fails. Claude Code solves this with a system called tool search . When MCP tool definitions exceed a token threshold, most tools are deferred — their definitions are withheld from the API request. In their place, the model gets a single ToolSearch tool it can invoke to discover and load tools on demand. The API receives a tool reference content block in the search result, expands it to the full definition, and the model can call the tool on its next turn. Consider the concrete flow. A user has configured MCP servers for GitHub, Slack, and Jira — 147 tools total. Without tool search, every API call sends 147 tool definitions: ~90,000 tokens. With tool search, the API call sends ~25 built-in tool definitions plus ToolSearch itself: ~15,000 tokens. The model’s prompt tells it “147 deferred tools are available — use ToolSearch to load them.” When the model needs to create a GitHub issue, it calls ToolSearch { query: "github create issue" } . The system returns a tool reference for mcp github create issue . On the next turn, that tool’s full schema is available, and the model calls it normally. Total overhead for this discovery: one extra turn, ~200 tokens. Savings over a 20-turn conversation: ~1.5 million tokens. This article traces the entire pipeline: the deferral decision, the threshold calculation, the search algorithm, the discovery loop across turns, and the snapshot mechanism that preserves discovered tools across context compaction. Every layer is designed around the same principle: fail closed, fail toward asking . If anything is uncertain — an unknown model, a proxy gateway, a missing token count — the system falls back to loading all tools, never to silently hiding them. The Deferral Decision Not every tool can be deferred. The model needs certain tools on turn one, before it has a chance to search for anything. The deferral decision is a priority-ordered checklist: function isDeferredTool tool : Explicit opt-out: MCP tools can declare they must always load if tool.alwaysLoad is true: return false MCP tools are deferred by default workflow-specific, often numerous if tool.isMcp is true: return true ToolSearch itself is never deferred — it's the bootstrap if tool.name is "ToolSearch": return false Core communication tools are never deferred Agent, Brief — model needs these immediately if tool is a critical communication channel: return false Everything else: defer only if explicitly marked return tool.shouldDefer is true The alwaysLoad opt-out is the escape hatch. An MCP server can set meta 'anthropic/alwaysLoad' on a tool to force it into every API request regardless of deferral mode. This handles tools like a primary database query tool that the model will need on nearly every turn. Notice the ordering. alwaysLoad is checked before the MCP check. This means an MCP tool can opt out of deferral even though MCP tools are deferred by default. And ToolSearch is checked after the MCP check, which means if someone wraps ToolSearch in an MCP server don’t , it still won’t be deferred. The checklist is a priority chain where each rule can only override the ones below it. The shouldDefer flag at the bottom is for built-in tools that want to participate in deferral without being MCP tools. Currently this isn’t widely used, but it exists as an extension point — a built-in tool could mark itself as deferrable if it’s rarely needed and expensive to describe. Three Modes The deferral system operates in one of three modes, controlled by an environment variable: function getToolSearchMode : Kill switch: if all beta features are disabled, never defer if DISABLE EXPERIMENTAL BETAS: return "standard" value = env.ENABLE TOOL SEARCH Explicit "always defer" mode if value is truthy or value is "auto:0": return "tst" Threshold-based: only defer when tools exceed a token budget if value is "auto" or value is "auto:N" where 1 <= N <= 99: return "tst-auto" Explicit disable if value is falsy: return "standard" Default: always defer MCP and shouldDefer tools return "tst" The default mode is tst — always defer. This is the right default because any user with MCP tools has already accepted the latency of an extra search turn in exchange for a larger effective context window. The tst-auto mode provides a middle ground: defer only when the token cost actually justifies it. The Threshold Calculation In tst-auto mode, the system measures how many tokens the deferred tools would consume and compares against a budget: threshold = floor contextWindow percentage / 100 Default percentage: 10% For a 200K context model: threshold = 20,000 tokens The token count comes from the API’s countTokens endpoint when available. The system serializes each deferred tool into its API schema name + description + JSON schema , sends them to the counting endpoint, and caches the result keyed by the tool name set. The cache invalidates when MCP servers connect or disconnect, changing the tool pool. There’s a subtlety in the counting. The API adds a fixed preamble ~500 tokens whenever tools are present in a request. When counting tools individually, each count includes this overhead, so counting N tools individually would report N × 500 tokens of phantom overhead. The system subtracts this constant from the total: rawCount = countTokensViaAPI deferredToolSchemas adjustedCount = max 0, rawCount - 500 When the token counting API is unavailable — perhaps the provider doesn’t support it, or the network request fails — the system falls back to a character-based heuristic. It sums the character lengths of each tool’s name, description, and serialized input schema, then converts using a ratio of 2.5 characters per token: charThreshold = floor tokenThreshold 2.5 totalChars = sum tool.name.length + tool.description.length + tool.schema.length for each deferred tool enabled = totalChars = charThreshold This heuristic is intentionally conservative. Tool definitions are schema-heavy lots of short keys and structural characters , which tokenize at a higher density than natural language. A 2.5 chars/token ratio slightly overestimates the token count, biasing toward enabling deferral — the safe direction. The Search Mechanism When tool search is enabled, the model sees a ToolSearch tool in its tool list. The tool accepts a query string and returns up to 5 results configurable . There are two query modes. Direct Selection The model can request specific tools by name: ToolSearch { query: "select:mcp github create issue" } ToolSearch { query: "select:Read,Edit,Grep" } comma-separated Direct selection is a lookup, not a search. For each requested name, the system checks the deferred tool pool first, then falls back to the full tool set. Finding a tool in the full set that isn’t deferred is a no-op — the tool is already loaded — but returning it prevents the model from retrying in a loop. Why does the fallback to the full tool set matter? After context compaction or in subagent conversations, the model sometimes tries to “select” a tool it previously used, not realizing the tool is already loaded because its earlier search result was summarized away . Without the full-set fallback, the select would fail, the model would get “no matching deferred tools found,” and it would waste a turn figuring out the tool is already available. The fallback makes this a silent success. Keyword Search When the model doesn’t know the exact tool name, it searches by keyword: ToolSearch { query: "slack send message" } ToolSearch { query: "+github pull request" } + requires term The search algorithm scores each deferred tool against the query terms: function scoreToolForQuery tool, terms : parts = parseToolName tool.name "mcp slack send message" - parts: "slack", "send", "message" "NotebookEdit" - parts: "notebook", "edit" score = 0 for term in terms: Exact part match highest signal if term in parts: score += 12 if tool.isMcp else 10 Substring match within a part elif any term in part for part in parts : score += 6 if tool.isMcp else 5 Full name fallback elif term in fullName and score is 0: score += 3 searchHint match curated capability phrase if wordBoundaryMatch term, tool.searchHint : score += 4 Description match lowest signal, most noise if wordBoundaryMatch term, tool.description : score += 2 return score MCP tools get slightly higher weight on exact matches 12 vs 10 and substring matches 6 vs 5 . This is deliberate: when tool search is active, most deferred tools are MCP tools. Boosting their scores ensures they rank above built-in tools that happen to share terminology. The searchHint field is a curated string that tools can provide to improve discoverability. It’s weighted above description matches 4 vs 2 because it’s intentional signal — a tool author explicitly saying “this tool handles X” — rather than incidental keyword overlap in a long description. Description matching uses word-boundary regex \bterm\b to avoid false positives. Without boundaries, a search for “read” would match every tool whose description contains “already”, “thread”, or “spreadsheet”. There’s also a required-term mechanism. Prefixing a term with + makes it mandatory: only tools matching ALL required terms in their name, description, or search hint are scored. This lets the model narrow results when a server has many tools: +slack send finds tools with “slack” in the name AND ranks them by “send” relevance. A Concrete Scoring Example Suppose the deferred pool contains these tools: mcp slack send message MCP mcp slack list channels MCP mcp github create issue MCP mcp email send email MCP The model searches: ToolSearch { query: "slack send" } . Here’s the scoring: mcp slack send message: parts = "slack", "send", "message" "slack": exact part match, MCP → +12 "send": exact part match, MCP → +12 Total: 24 mcp slack list channels: parts = "slack", "list", "channels" "slack": exact part match, MCP → +12 "send": no match in parts, no match in name → +0 Total: 12 mcp email send email: parts = "email", "send", "email" "slack": no match → +0 "send": exact part match, MCP → +12 Total: 12 mcp github create issue: parts = "github", "create", "issue" "slack": no match → +0 "send": no match → +0 Total: 0 Result: "mcp slack send message", "mcp slack list channels", "mcp email send email" . The Slack send tool wins, the other Slack tool ties with the email send tool, and the GitHub tool is excluded. Note how multi-term queries naturally boost tools that match on multiple dimensions — a tool matching both “slack” AND “send” scores 24, while one matching only “slack” scores 12. The regex patterns are pre-compiled once per search to avoid creating them inside the hot loop N tools × M terms × 2 checks . Each unique term gets one compiled regex, and all tools share them. The MCP Prefix Fast Path When the query starts with mcp , the system checks for prefix matches before falling through to keyword search: if query starts with "mcp ": matches = tools where name starts with query if matches found: return first maxResults matches This handles the common pattern where the model knows the server name but not the specific action. Searching mcp github returns all GitHub MCP tools without keyword scoring. What Search Returns The search doesn’t return tool definitions. It returns tool reference content blocks: Tool result sent back to the API: { type: "tool result", tool use id: "...", content: { type: "tool reference", tool name: "mcp github create issue" }, { type: "tool reference", tool name: "mcp github list issues" } } This is a beta API feature. The API server receives the tool reference block and expands it into the full tool definition in the model’s context. The client never sends the definition itself — the API resolves the reference from the deferred schemas that were sent with defer loading: true . This is the key insight of the architecture. The client marks deferred tools with defer loading: true in their schema, telling the API “here’s the definition, but don’t show it to the model unless referenced.” The tool reference block is the trigger that expands a deferred definition. The model sees the full schema in its context only after a successful search. Why not just return the full tool definition in the search result? Two reasons. First, the API handles the injection into the model’s tool context — the client doesn’t need to construct a new API request with the tool added. Second, tool reference is a structured content block that the API validates against the known deferred schemas. The client can’t fabricate a tool definition in a tool result and have it treated as a callable tool. The API is the authority on which tools exist. The Two-Layer Gate For tool search to actually engage, two checks must pass: Optimistic check fast, stateless : Can tool search possibly be enabled? This runs early — during tool pool assembly — to decide whether ToolSearch itself should be included in the tool list. It checks mode and proxy gateway, but NOT model or threshold. This is called “optimistic” because it says “yes” even if the definitive check might say “no” later. Definitive check async, contextual : Should tool search be used for this specific API request? This runs at request time with the full context: model name, tool list, token counts. It checks model support, ToolSearch availability, and for tst-auto the threshold. The two-layer design avoids a chicken-and-egg problem. You can’t check the definitive gate until you’ve assembled the tool pool. But the tool pool includes ToolSearch. If ToolSearch isn’t in the pool, the definitive check will say “ToolSearch unavailable, disable.” So the optimistic check decides whether to include ToolSearch, and the definitive check decides whether to use it. The Discovery Loop Tool search creates a multi-turn protocol. On turn 1, the model sees only non-deferred tools plus ToolSearch. It calls ToolSearch. On turn 2, the discovered tools are available. But how does the system know which tools to include on turn 2? Scanning Message History Before each API request, the system scans the conversation history for tool reference blocks: function extractDiscoveredToolNames messages : discovered = empty set for message in messages: Compact boundaries carry a snapshot explained later if message is compact boundary: for name in message.metadata.preCompactDiscoveredTools: discovered.add name continue tool reference blocks only appear in user messages tool result is a user-role message in the API if message is not user: continue for block in message.content: if block is tool result with array content: for item in block.content: if item.type is "tool reference": discovered.add item.tool name return discovered The extracted set determines which deferred tools to include in the next request: function filterToolsForRequest tools, deferredToolNames, discoveredToolNames : return tools where: Always include non-deferred tools tool.name not in deferredToolNames Always include ToolSearch itself OR tool.name is "ToolSearch" Include deferred tools that have been discovered OR tool.name in discoveredToolNames This creates an accumulating set. Once a tool is discovered via search, it stays available for the rest of the conversation. The model never needs to re-search for a tool it’s already found. There’s an important detail in what gets sent to toolToAPISchema . The filtering controls which tools appear in the API’s tool array. But the ToolSearch prompt — which lists available deferred tools for the model to see — is generated from the full tool list, not the filtered one. This separation ensures the model can always search the complete pool, even though only discovered tools have their schemas sent. A Complete Round-Trip Let’s trace a single discovery end-to-end: Turn 1 : User says “Create a GitHub issue for this bug.” - System computes deferred set: 147 MCP tools. - System scans history: no tool reference blocks yet. - Filtered tools: 25 built-in + ToolSearch. 147 deferred sent with defer loading: true . - Model sees 26 tools. It knows it needs GitHub. It calls ToolSearch. Turn 1 response : Model generates tool use for ToolSearch with query "select:mcp github create issue" . Turn 1 result : System looks up the name, finds it in deferred pool. Returns: { type: "tool result", content: { type: "tool reference", tool name: "mcp github create issue" } } Turn 2 : System prepares next API request. - Scans history: finds tool reference for mcp github create issue . - Filtered tools: 25 built-in + ToolSearch + mcp github create issue . - Model sees 27 tools. mcp github create issue has full schema. Model calls it. The discovery cost was one turn and ~200 tokens. The savings persist for the rest of the conversation — mcp github create issue stays in the filtered set. The defer loading Flag Each tool that passes through the filter gets serialized into an API schema. Deferred tools — those in deferredToolNames — get an extra field: schema = { name: tool.name, description: tool.description, input schema: tool.jsonSchema, defer loading: true <-- tells API: definition present but hidden } The defer loading flag is the contract between client and API. It says: “I’m sending you this tool’s definition so you can expand tool reference blocks that name it, but don’t include it in the model’s visible tool list unless a reference triggers it.” Without this flag, sending a tool definition means the model sees it. With it, the definition is staged but invisible until referenced. This is what makes the system work — the API has all the information it needs to expand references without the client re-sending definitions on demand. Announcing Available Tools The model needs to know what tools it can search for. There are two mechanisms for this, gated by a feature flag: Pre-message approach original : Before the conversation messages, the system prepends a user message listing all deferred tool names: <available-deferred-tools mcp github create issue mcp github list issues mcp slack send message ... </available-deferred-tools Delta attachment approach newer : Instead of prepending the full list every turn, the system computes a diff against what’s already been announced: function getDeferredToolsDelta tools, messages : Scan prior attachment messages for previous announcements announced = empty set for message in messages: if message is attachment of type "deferred tools delta": for name in message.addedNames: announced.add name for name in message.removedNames: announced.delete name deferred = tools where isDeferredTool tool deferredNames = names of deferred tools poolNames = names of all tools added = deferred tools not yet announced removed = announced tools no longer in the pool AND no longer in base tools Note: a tool that was deferred but is now loaded undeferred is NOT reported as removed — it's still available, just loaded differently if no changes: return null return { addedNames, removedNames } The delta approach has a critical advantage: it doesn’t bust the prompt cache. The pre-message approach changes the first message whenever the tool pool changes MCP server connects late, tools added/removed , which invalidates the cached prefix. Deltas are appended as attachment messages, leaving the prefix stable. Surviving Compaction Context compaction summarizes old messages to free space. But compaction destroys tool reference blocks — the summary is plain text, not structured content. If the system can’t find tool references after compaction, it thinks no tools have been discovered, and every deferred tool disappears from subsequent requests. The Snapshot Mechanism Before compaction runs, the system takes a snapshot of all discovered tools and stores it on the compact boundary marker: function compact messages : Snapshot BEFORE summarizing discoveredTools = extractDiscoveredToolNames messages summary = summarize messages boundaryMarker = createBoundaryMessage summary if discoveredTools is not empty: boundaryMarker.metadata.preCompactDiscoveredTools = sorted discoveredTools return boundaryMarker, ...remainingMessages This snapshot appears in three compaction paths: full compaction, partial compaction which keeps recent messages intact , and session-memory compaction. All three perform the same snapshot. After compaction, when extractDiscoveredToolNames scans the messages, it encounters the compact boundary marker first and reads the snapshot: Post-compaction message array: compact boundary { metadata.preCompactDiscoveredTools: "mcp github create issue", ... }, ... remaining messages with tool reference blocks ... The scan merges the snapshot with any new references in remaining messages. The union is the full discovered set — nothing is lost. Why This Works The snapshot is idempotent. Multiple compactions each snapshot the accumulated set. If compaction A captures tools {X, Y} and the model later discovers Z, compaction B captures {X, Y, Z}. The set only grows. Partial compaction scans all messages, not just the ones being summarized. This is deliberate — it’s simpler than tracking which tools were referenced in which half, and set union is idempotent, so double-counting is harmless. Edge Cases and Fail-Closed Design Model Support Not every model supports tool reference content blocks. The system uses a negative list: models are assumed to support tool search unless they match a pattern in the unsupported list. UNSUPPORTED MODEL PATTERNS = "haiku" function modelSupportsToolReference model : normalized = lowercase model for pattern in UNSUPPORTED MODEL PATTERNS: if pattern in normalized: return false return true new models work by default This is a deliberate design choice. A positive list allowlist would require code changes for every new model. The negative list means new models inherit tool search support automatically. Only models known to lack the capability are excluded. The unsupported pattern list can be updated remotely via feature flags, without shipping a new release. This handles the case where a new model launches without tool reference support — the team adds it to the list, and all running instances pick it up. Proxy Gateway Detection: A Two-Act Failure This is a case where a real-world failure, a fix, and a failure of the fix shaped the final design. Act 1 : Users routing API calls through third-party proxy gateways LiteLLM, corporate firewalls started getting API 400 errors: "Messages content type tool reference not supported." The proxy only accepted standard content types — text , image , tool use , tool result — and rejected the beta tool reference blocks. Tool search worked fine with direct Anthropic API calls but broke through any intermediary. Act 2 : The fix was aggressive: detect non-Anthropic base URLs and disable tool search entirely. This stopped the 400 errors but created a new problem — users with compatible proxies LiteLLM passthrough mode, Cloudflare AI Gateway lost deferred tool loading. All their MCP tools loaded into the main context window every turn. For users with many MCP tools, this was a significant regression in context efficiency. The final design balances both failures: function isToolSearchEnabledOptimistic : if mode is "standard": return false Proxy detection: first-party provider but non-Anthropic URL Only triggers when ENABLE TOOL SEARCH is unset default behavior if ENABLE TOOL SEARCH is not set AND provider is "firstParty" AND baseURL is not a known Anthropic host: return false proxy would reject tool reference blocks return true The key insight is the ENABLE TOOL SEARCH is not set condition. When the environment variable is unset, the system assumes unknown proxies can’t handle beta features. But setting any non-empty value — true , auto , auto:10 — tells the system “I know what I’m doing, my proxy supports this.” The user takes explicit responsibility for their proxy’s capabilities. There’s also a global kill switch: DISABLE EXPERIMENTAL BETAS forces standard mode regardless of other settings. When this is set, the system strips beta-specific fields from tool schemas before sending them to the API, ensuring no defer loading or tool reference reaches the wire. This was itself motivated by a separate failure: the kill switch originally didn’t remove all beta headers, breaking LiteLLM-to-Bedrock proxies that rejected unknown beta flags. Pending MCP Servers MCP servers connect asynchronously. When a user starts Claude Code, some servers may still be initializing. If tool search is enabled but no deferred tools exist yet because no servers have connected , the system normally disables tool search for that request — there’s nothing to search. But if MCP servers are pending, it keeps ToolSearch available: if useToolSearch AND no deferred tools AND no pending MCP servers: useToolSearch = false nothing to search, save a tool slot if useToolSearch AND no deferred tools AND pending MCP servers: keep ToolSearch — tools will appear when servers connect When the model calls ToolSearch and no tools match, the result includes the names of pending servers: { matches: , total deferred tools: 0, pending mcp servers: "github", "slack" } This tells the model “your search found nothing, but these servers are still connecting — try again shortly.” Cache Invalidation Tool descriptions are memoized to avoid recomputing them on every search. But the deferred tool set can change mid-conversation MCP server connects, tools added/removed . The cache key is the sorted, comma-joined list of deferred tool names. When the set changes, the cache clears: function maybeInvalidateCache deferredTools : currentKey = sorted tool.name for tool in deferredTools .join "," if currentKey = cachedKey: clearDescriptionCache cachedKey = currentKey The token count is also memoized with the same key scheme. This means connecting a new MCP server triggers one fresh token count and one fresh description computation, then subsequent searches reuse the cache. Tool Search Disabled Mid-Conversation If the model switches from a supported model Sonnet to an unsupported one Haiku mid-conversation, the message history may contain tool reference blocks that the new model can’t process. The system handles this by stripping tool-search artifacts: if not useToolSearch: for message in apiMessages: if message is user: stripToolReferenceBlocks message if message is assistant: stripCallerField message tool use caller metadata This ensures the API never receives tool reference blocks when the current model doesn’t support them, even if a previous model generated them. There’s an additional stripping path for a subtler failure: MCP server disconnection. If a server disconnects mid-conversation, previously valid tool reference blocks now point to tools that don’t exist in the current pool. The API rejects these with “Tool reference not found in available tools.” The normalization pipeline strips tool reference blocks for tools that aren’t in the current available set, even when tool search is otherwise enabled. The Turn Boundary Problem When the API server receives a tool result containing tool reference blocks, it expands them into a <functions block — the same format used for tool definitions at the start of the prompt. This expansion happens server-side, and it creates an unexpected problem in the wire format. The expanded <functions block appears inline in the conversation. If the same user message that contains the tool result also has text siblings auto-memory reminders, skill instructions, etc. , those text blocks render as a second Human: turn segment immediately after the </functions closing tag. This creates an anomalous pattern in the conversation structure: two consecutive human turns with a functions block in between. The model learns this pattern. After seeing it several times in a conversation, it starts completing the pattern: when it encounters a bare tool result at the tail of the conversation no text siblings , it emits the stop sequence instead of generating a meaningful response. The conversation just… stops. An A/B experiment with five arms confirmed the dose-response: more tool reference messages with text siblings → higher stop-sequence rate. Two mitigations work in concert: Turn boundary injection : When a user message contains tool reference blocks and no text siblings, the system injects a minimal text block "Tool loaded." as a sibling. This creates a clean Human: Tool loaded. turn boundary that prevents the model from seeing a bare functions block at the tail. Sibling relocation : When a user message contains tool reference blocks AND has text siblings from auto-memory, attachments, etc. , the system moves those text blocks to the next user message that has tool result content but NO tool reference . This eliminates the anomalous two-human-turns pattern. If no valid target exists the tool reference message is near the end of the conversation , the siblings stay — that’s safe because a tail ending in a human turn gets a proper assistant cue. Schema-Not-Sent Recovery Sometimes the model tries to call a deferred tool without first discovering it via ToolSearch. This happens when the model hallucinates having seen the tool’s schema perhaps from its training data or when a prior discovery was lost to compaction. The call fails at input validation — the model sends parameters that don’t match any known schema, because the schema was never sent. The raw validation error “expected object, received string” doesn’t tell the model what went wrong. So the system checks: is this a deferred tool that wasn’t in the discovered set? If yes, it appends a hint: "This tool's schema was not sent to the API — it was not in the discovered-tool set. Use ToolSearch to load it first: ToolSearch { query: 'select:<tool name ' } " This turns a confusing Zod error into an actionable instruction. The model reads the hint, calls ToolSearch, gets the schema, and retries — one extra turn instead of a conversation-ending failure. Invisible by Design ToolSearch calls never appear in the user’s terminal output. The tool’s renderToolUseMessage returns null and its userFacingName returns an empty string. In the message collapse system — which groups consecutive reads and searches into compact “Read 5 files” summaries — ToolSearch is classified as “absorbed silently”: it joins a collapse group without incrementing any counter. The user sees “Read 3 files, searched 2 files” but the ToolSearch call that loaded the tool definitions is invisible. This is deliberate. ToolSearch is infrastructure, not user-facing functionality. Showing “Searched for tools” in the output would be confusing — the user asked to create a GitHub issue, not to search for tools. The tool discovery is an implementation detail of how the model accesses MCP tools, and the UI hides it accordingly. The Complete Pipeline Here’s the full sequence for a single API request: 1. Mode check : Determine if tool search is tst , tst-auto , or standard . 2. Model check : Verify the model supports tool reference blocks. If not, disable. 3. Availability check : Confirm ToolSearch is in the tool pool not disallowed . 4. Threshold check tst-auto only : Count deferred tool tokens via API or character heuristic fallback . Compare to floor contextWindow × 10% . 5. Build deferred set : Mark each tool as deferred or not via the priority checklist. 6. Scan history : Extract discovered tool names from tool reference blocks and compact boundary snapshots. 7. Filter tools : Include non-deferred tools, ToolSearch, and discovered deferred tools. Exclude undiscovered deferred tools. 8. Serialize schemas : Add defer loading: true to deferred tools. Add beta header. 9. Announce pool : Prepend deferred tool list or compute delta attachment. 10. Send request : API receives full definitions with defer loading , shows only non-deferred and discovered tools to the model. 11. Model searches : Calls ToolSearch with a query. Gets tool reference blocks back. 12. Next turn : Step 6 finds the new references. Step 7 includes the discovered tools. The model can now call them. 13. Compaction : Before summarizing, snapshot discovered tools to boundary marker. After compaction, step 6 reads the snapshot. Each step fails toward loading more tools, not fewer. Unknown model? Load everything. Token count unavailable? Use conservative heuristic. Proxy detected? Load everything unless explicitly opted in. The worst case is wasting tokens on tool definitions. The best case is saving 90% of tool definition tokens while maintaining full functionality through on-demand discovery. The system turns an O N per-turn cost into O 1 for idle tools and O k for the k tools actually used in a conversation. For a user with 200 MCP tools who typically uses 5–10 per session, that’s a 95% reduction in tool definition tokens — context space reclaimed for actual work. Design Trade-offs Every engineering decision in this system reflects a trade-off. Here are the ones worth understanding: Deferral granularity : Why defer by tool, not by MCP server? Server-level deferral would mean discovering one tool loads all tools from that server. This is simpler but wasteful — a GitHub server might have 40 tools, and you only need 3. Tool-level deferral uses more search turns but saves more tokens. The scoring system mitigates the extra turns: a single keyword search for “github” returns the most relevant tools, not all 40. Negative vs. positive model list : The unsupported model list "haiku" means every new model gets tool search by default. The alternative — a positive list of supported models — would mean every new model launch requires a code update. The negative list risks sending tool reference blocks to a model that can’t handle them, but the API would return a clear error, and the feature flag system can add models to the unsupported list within minutes. Token counting precision : The character-per-token heuristic 2.5 is intentionally imprecise. Why not always use the API’s token counter? Because the counter requires a network round-trip that might fail or add latency. The heuristic runs instantly. And the cost of over-counting deferring when unnecessary is one extra search turn. The cost of under-counting not deferring when needed is 60,000 wasted tokens per turn. The asymmetry favors the conservative heuristic. Cache key design : Both the description cache and token count cache use the sorted tool name list as key, not a hash. This means cache comparison is O N in the number of deferred tools, but N is typically <200 and the comparison runs once per API request. A hash would be O 1 but risks collisions, and debugging cache issues with hashed keys is harder than with readable name lists. Snapshot vs. protection : Why snapshot discovered tools instead of protecting tool reference messages from compaction? The snip compaction strategy does protect these messages, but full compaction summarizes everything. Protecting individual messages from full compaction would fragment the summary and reduce its quality. The snapshot approach lets compaction work normally and reconstructs the discovery state from metadata. Get new posts delivered to your inbox Subscribe on Substack → https://oldeucryptoboi.substack.com/subscribe