cd /news/ai-agents/google-s-gemini-api-drops-schema-ove… · home › topics › ai-agents › article
[ARTICLE · art-172809] src=alphasignal.ai ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Google's Gemini API Drops Schema Overhead for Agents With Hundreds of Tools

Google launched Tool Retrieval in preview in the Gemini Interactions API, letting agents scale to hundreds of tools by loading function parameter schemas only on demand instead of placing thousands of schema tokens in every request. The feature is enabled by adding a `tool_search` tool and setting `defer_loading: true` per function, works with Gemini 3.6 and later including all Gemini 4 models, preserves prefix caching, and carries no separate Tool Retrieval fee beyond standard model input and output rates. Google said deferred schemas keep discovery lightweight while reducing input tokens and prompt growth, and MCP servers can be deferred wholesale.

key takeaway

Adding a `tool_search` tool and `defer_loading: true` to each function schema lets Gemini agents discover hundreds of tools while loading parameter schemas only after selection, cutting input tokens and preserving prefix caching at no extra charge.

read5 min views3 publishedNov 19, 2026
Google's Gemini API Drops Schema Overhead for Agents With Hundreds of Tools
Image: Alphasignal (auto-discovered)

Gemini's new Tool Retrieval loads function schemas on demand, letting agents scale to hundreds of tools without blowing up context windows.

  • Google launched Tool Retrieval in preview in the Gemini Interactions API.
  • Agents can scale to hundreds of tools by parameter schemas only on demand.
  • Enabled by adding a tool_search tool and settingdefer_: true per function.
  • Works with Gemini 3.6 and later, including all Gemini 4 models.
  • Preserves prefix caching since deferred schemas stay out of the prompt start.
  • No extra charge, and MCP servers can be deferred wholesale.

Google has added Tool Retrieval to the Gemini API, allowing agents to discover hundreds of functions while parameter schemas only when needed. The approach reduces input tokens, limits prompt growth, and gives the model a smaller set of schemas to evaluate before calling a function.

Detail Availability
API Interactions API preview
Models Gemini 3.6 and later, including all Gemini 4 models
Retrieval modes Server-side, client-side, and deferred MCP
Billing No separate Tool Retrieval fee; standard model input and output rates apply

Function schemas create a scaling tax #

When an application exposes a function to a model, it usually sends a schema describing the function’s name, arguments, types, and required fields. A catalog containing dozens or hundreds of functions can place thousands of schema tokens in every request, even when the model uses only one function.

Large catalogs increase input costs and latency while giving the model more candidates to evaluate. Similar function names and overlapping descriptions can also reduce selection accuracy as the catalog expands.

Deferred schemas keep discovery lightweight #

Tool Retrieval exposes the name and description of each deferred function while keeping its parameter schema out of the initial context. After Gemini selects a function, the API loads its schema by name and emits the function call. Server-side retrieval completes that sequence within one request.

This design separates two tasks that previously shared the same prompt space: discovering the appropriate function and generating valid arguments for it. Gemini can scan the full catalog, then inspect the detailed schema for the selected function.

Server-side retrieval requires a tool_search entry and defer_: true on each function whose schema should load on demand. Google runs the lookup, so the application does not need to build or host an embedding index.

tools = [
    {"type": "tool_search"},
    {
        "type": "function",
        "name": "issue_refund",
        "description": "Refund an order to the original payment method.",
        "defer_": True,
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"}
            },
            "required": ["order_id"]
        }
    }
]

Frequently called functions can remain fully declared, while infrequently used integrations can be deferred. The response’s interaction.steps sequence records the tool_search_call, tool_search_result, and resulting function_call, giving developers a trace for diagnosing weak descriptions, incorrect matches, and argument-generation failures.

Deferred schemas preserve cached prefixes #

Prefix caching reuses model computation when consecutive requests begin with the same token sequence. Deferred schemas appear later in the interaction, leaving the initial prompt stable across turns and preserving more opportunities for cache reuse.

Application-managed retrieval can change the schemas placed near the beginning of each request, which alters the cached prefix. Tool Retrieval keeps the catalog’s compact names and descriptions stable while adding selected schemas after the search step.

MCP servers support the same pattern. Setting defer_: true on an mcp_server tool makes the server’s functions available on demand, allowing an agent to connect to a large MCP catalog without declaring every function separately.

Applications with an existing embedding index, tenant-specific catalog, or permission layer can set "execution": "client" on tool_search. Gemini then requests a search, the application queries its registry, and the application returns the matching tools.

Client-side execution adds one application-managed round trip for each search. It also allows tools to appear during an active conversation, such as when a user connects a new integration or enables a plugin.

Per-user retrieval should return only tools the user may access. Applications still need execution-time authorization, argument validation, and audit logging for every function call.

Token savings favor large, sparse catalogs #

Tool Retrieval changes token usage in three places:

  • tool_search adds a small, fixed block of input tokens.
  • Each loaded schema counts as input for the remainder of the conversation.
  • Deferred schemas that remain unused add no schema tokens.

Net savings appear when the omitted schemas outweigh the search overhead and the schemas eventually loaded. A small agent with four short functions is unlikely to benefit. An agent with five frequently used functions and fifty occasional integrations is a stronger candidate.

The Tool Retrieval docs recommend keeping common functions loaded and deferring the rest. This hybrid configuration avoids search overhead for routine calls while reducing the cost of the broader catalog.

Benchmark the full retrieval path #

Tool Retrieval brings semantic search over function descriptions into Gemini’s server-side request flow. Teams using custom vector indexes can remove that component when Google’s retrieval meets their ranking, authorization, and observability requirements; client-side execution remains available for specialized registries.

A representative evaluation should compare full-schema requests with deferred retrieval across several measures:

  • Input tokens and total model cost
  • Median and tail latency
  • Tool-selection accuracy
  • Function argument validity
  • Prefix-cache hit rate
  • Client-side search frequency, when applicable

Because the feature remains in preview, production evaluations should also include model-version pinning where available, regression tests for tool selection, and monitoring for changes to response steps or retrieval behavior.

── more in #ai-agents 4 stories · sorted by recency
── more on @google 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/google-s-gemini-api-…] indexed:0 read:5min 2026-11-19 · —