Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions

A controlled study of 367,174 Model Context Protocol (MCP) tools found that parameter-schema changes produce the largest and most consistent shifts in LLM tool selection, according to an arXiv paper (arXiv:2610.14334v1). Across five LLMs and five tasks, each comparison altered exactly one metadata factor — server name, tool name, description, or parameter schema — while holding the others fixed, with position-adjusted effects estimated via conditional logistic regression. Models favored separate top-level parameters over grouped or composite forms, and when only one schema satisfied an explicit request requirement, all five models selected the compatible tool despite position preferences.

arXiv:2610.14334v1 Announce Type: new Abstract: In Model Context Protocol MCP systems, LLMs select tools from metadata rather than from their implementations. Names and descriptions summarize what a tool does, while the parameter schema specifies the fields a tool accepts and how they are organized. Our analysis of metadata for 367,174 MCP tools reveals substantial schema variation among functionally similar tools. Many of these tools share names and similar descriptions but expose different parameter schemas. This raises a key question: can parameter-schema differences influence tool selection before any call is made? We study this question through controlled metadata interventions across five LLMs and five tasks. Each comparison changes exactly one metadata factor server name, tool name, description, or parameter schema while keeping the others fixed. We balance tool positions and use conditional logistic regression to estimate position-adjusted effects. We find that: 1 parameter-schema changes produce the largest and most consistent selection shifts among the tested interventions. Models tend to favor separate top-level parameters over grouped or composite forms. 2 Descriptions that explain parameters also tend to increase selection, while name effects are weaker and less consistent. 3 Targeted comparisons show that schema effects depend on the specific parameter names and on how the request represents the same information. When only one schema satisfies an explicit request requirement, all five models select the compatible tool despite position preferences. These results show that parameter schemas influence not only how tools are called, but also whether they are selected. For MCP developers and evaluators, schema design and tool ordering can therefore affect which implementation is invoked even when functionality is unchanged.