Google's Gemini API Drops Schema Overhead for Agents With Hundreds of Tools Google launched Tool Retrieval in preview in the Gemini Interactions API, letting agents scale to hundreds of tools by loading function parameter schemas only on demand instead of placing thousands of schema tokens in every request. The feature is enabled by adding a `tool_search` tool and setting `defer_loading: true` per function, works with Gemini 3.6 and later including all Gemini 4 models, preserves prefix caching, and carries no separate Tool Retrieval fee beyond standard model input and output rates. Google said deferred schemas keep discovery lightweight while reducing input tokens and prompt growth, and MCP servers can be deferred wholesale. Google's Gemini API Drops Schema Overhead for Agents With Hundreds of Tools Gemini's new Tool Retrieval loads function schemas on demand, letting agents scale to hundreds of tools without blowing up context windows. - Google launched Tool Retrieval https://blog.google/innovation-and-ai/technology/developers-tools/gemini-api-tool-retrieval/?utm source=alphasignal in preview in the Gemini Interactions API. - Agents can scale to hundreds of tools by loading parameter schemas only on demand. - Enabled by adding a tool search tool and setting defer loading: true per function. - Works with Gemini 3.6 and later, including all Gemini 4 models. - Preserves prefix caching since deferred schemas stay out of the prompt start. - No extra charge, and MCP servers https://aistudio.google.com/docs/tool-retrieval?utm source=alphasignal can be deferred wholesale. Google has added Tool Retrieval to the Gemini API https://blog.google/innovation-and-ai/technology/developers-tools/gemini-api-tool-retrieval/?utm source=alphasignal , allowing agents to discover hundreds of functions while loading parameter schemas only when needed. The approach reduces input tokens, limits prompt growth, and gives the model a smaller set of schemas to evaluate before calling a function. | Detail | Availability | |---|---| | API | Interactions API preview | | Models | Gemini 3.6 and later, including all Gemini 4 models | | Retrieval modes | Server-side, client-side, and deferred MCP loading | | Billing | No separate Tool Retrieval fee; standard model input and output rates apply | Function schemas create a scaling tax When an application exposes a function to a model, it usually sends a schema describing the function’s name, arguments, types, and required fields. A catalog containing dozens or hundreds of functions can place thousands of schema tokens in every request, even when the model uses only one function. Large catalogs increase input costs and latency while giving the model more candidates to evaluate. Similar function names and overlapping descriptions can also reduce selection accuracy as the catalog expands. Deferred schemas keep discovery lightweight Tool Retrieval exposes the name and description of each deferred function while keeping its parameter schema out of the initial context. After Gemini selects a function, the API loads its schema by name and emits the function call. Server-side retrieval completes that sequence within one request. This design separates two tasks that previously shared the same prompt space: discovering the appropriate function and generating valid arguments for it. Gemini can scan the full catalog, then inspect the detailed schema for the selected function. Two settings activate server-side search Server-side retrieval requires a tool search entry and defer loading: true on each function whose schema should load on demand. Google runs the lookup, so the application does not need to build or host an embedding index. tools = {"type": "tool search"}, { "type": "function", "name": "issue refund", "description": "Refund an order to the original payment method.", "defer loading": True, "parameters": { "type": "object", "properties": { "order id": {"type": "string"} }, "required": "order id" } } Frequently called functions can remain fully declared, while infrequently used integrations can be deferred. The response’s interaction.steps sequence records the tool search call , tool search result , and resulting function call , giving developers a trace for diagnosing weak descriptions, incorrect matches, and argument-generation failures. Deferred schemas preserve cached prefixes Prefix caching reuses model computation when consecutive requests begin with the same token sequence. Deferred schemas appear later in the interaction, leaving the initial prompt stable across turns and preserving more opportunities for cache reuse. Application-managed retrieval can change the schemas placed near the beginning of each request, which alters the cached prefix. Tool Retrieval keeps the catalog’s compact names and descriptions stable while adding selected schemas after the search step. MCP servers support the same pattern. Setting defer loading: true on an mcp server tool makes the server’s functions available on demand, allowing an agent to connect to a large MCP catalog without declaring every function separately. Custom registries can run the search Applications with an existing embedding index, tenant-specific catalog, or permission layer can set "execution": "client" on tool search . Gemini then requests a search, the application queries its registry, and the application returns the matching tools. Client-side execution adds one application-managed round trip for each search. It also allows tools to appear during an active conversation, such as when a user connects a new integration or enables a plugin. Per-user retrieval should return only tools the user may access. Applications still need execution-time authorization, argument validation, and audit logging for every function call. Token savings favor large, sparse catalogs Tool Retrieval changes token usage in three places: - tool search adds a small, fixed block of input tokens. - Each loaded schema counts as input for the remainder of the conversation. - Deferred schemas that remain unused add no schema tokens. Net savings appear when the omitted schemas outweigh the search overhead and the schemas eventually loaded. A small agent with four short functions is unlikely to benefit. An agent with five frequently used functions and fifty occasional integrations is a stronger candidate. The Tool Retrieval docs https://aistudio.google.com/docs/tool-retrieval?utm source=alphasignal recommend keeping common functions loaded and deferring the rest. This hybrid configuration avoids search overhead for routine calls while reducing the cost of the broader catalog. Benchmark the full retrieval path Tool Retrieval brings semantic search over function descriptions into Gemini’s server-side request flow. Teams using custom vector indexes can remove that component when Google’s retrieval meets their ranking, authorization, and observability requirements; client-side execution remains available for specialized registries. A representative evaluation should compare full-schema requests with deferred retrieval across several measures: - Input tokens and total model cost - Median and tail latency - Tool-selection accuracy - Function argument validity - Prefix-cache hit rate - Client-side search frequency, when applicable Because the feature remains in preview, production evaluations should also include model-version pinning where available, regression tests for tool selection, and monitoring for changes to response steps or retrieval behavior.