Every AI agent is a model plus its tools. The model gets most of the attention: which one to pick, which benchmark it tops. The tools usually get written in an afternoon, by wrapping whatever API already exists. Yet the tools decide what an agent can see, what it can change, and how much of its context is left for thinking. Two agents on the same model can behave completely differently because of the tools they’re given.
A tool is any capability you hand to a model: a function in your code, a call to an external API, a database query, a shell command, even another agent. What every tool shares is its caller. That caller is a model. It reads every word of your tool’s description, every token you return fills its context, it can’t stop to ask what you meant, and it can be talked into following instructions hidden in the data a tool returns. This post is about designing for that caller: what OpenAI, Anthropic, Google and the MCP spec recommend, and what their guides leave out.
Key takeaways
- Design tools around workflows, not endpoints. A few distinct tools beat a complete API wrapper, and overlap between tools hurts more than their number.
- Treat names, descriptions and schemas as a contract the model reads literally. Treat responses and errors as part of the prompt.
- Past about 20 tools, stop every definition up front. Toolsets, tool search and code execution all help, and each has a real cost.
- Make a specialist a tool before you make it a separate agent. Multiple agents help with parallel work, and they cost a lot of tokens.
- The model decides what to call. Your infrastructure decides what’s allowed, what’s safe to retry and what can be undone.
Design for the caller #
The vendors’ guidance keeps coming back to the same test. OpenAI calls it the intern test: could a new person use the function correctly with only what the model sees? Anthropic says to describe a tool the way you’d describe it to a new hire, making implicit context explicit. Google’s version is shorter: “Be extremely clear and specific in your descriptions.”
Most of the rules below follow from that test.
1. Design the surface: workflows, not endpoints #
A common mistake is wrapping every API endpoint as a tool. Anthropic puts it plainly: “More tools don’t always lead to better outcomes.” Its examples are a search_contacts tool instead of list_contacts, and a schedule_event tool that finds availability and books the meeting in one step, instead of making the model chain the calls itself.
OpenAI makes the same point in two rules: “Combine functions that are always called in sequence,” and “Don’t make the model fill arguments you already know.” If your code already knows the user ID, don’t ask the model for it.
How many tools is too many?
The vendors give different numbers, and the differences are instructive:
The last row is the important one. Overlap hurts more than count. Two tools that both sound like they search customers will cause more wrong calls than twenty tools with clearly separate jobs. Before splitting work into a new agent, OpenAI recommends improving tool names, parameters and descriptions first.
Tool definitions also take up context before the user says a word. GitHub’s MCP server used to enable 101 tools by default, using 64.6k tokens. In October 2025, GitHub cut the default to 52 tools and 30.3k tokens, citing “tool selection degradation: LLMs struggle to choose the right tool from too many options.” Anthropic reports a five-server setup with 58 tools taking about 55K tokens up front, and says it has seen definitions reach 134K tokens before optimization.
2. Write the contract: names, descriptions and schemas #
The model reads your tool definition literally. Anything a developer would carry in their head has to be written into the definition.
- Names. Namespace tools that come from different systems, such as
asana_searchandjira_search. Anthropic notes that prefix versus suffix namingaffects results, and that the effect varies by model , so test both. - Parameters. Use
user_id, notuser. Use enums wherever the valid values are limited. OpenAI’s example of a bad design istoggle_light(on: bool, off: bool), whichallows invalid states . - Descriptions. Say what the tool does, when to use it, when not to, what each parameter means and what comes back.
- Examples. For tools with nested or optional parameters, include a correct example call. In Anthropic’s internal testing, adding Tool Use Examples raised accuracy on complex parameter handlingfrom 72% to 90% .
Here’s what that looks like for one tool:
{
"name": "crm_search_customers",
"description": "Find customers by name, email or company. Use this before any action on a customer. Do not use it to list all customers. Returns up to `limit` matches, most recently active first.",
"inputSchema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Name, email address or company name. Partial matches allowed."
},
"status": {
"type": "string",
"enum": ["active", "trial", "churned"],
"description": "Optional. Filter by account status."
},
"response_format": {
"type": "string",
"enum": ["concise", "detailed"],
"description": "concise: name, plan and customer_ref only. detailed: adds recent orders and notes."
},
"limit": { "type": "integer", "minimum": 1, "maximum": 20 }
},
"required": ["query"],
"additionalProperties": false
}
}
3. Design the response: return signal, not dumps #
Whatever a tool returns goes straight into the model’s context, and an agent’s context fills up fast.
- Return meaning, not identifiers. Anthropic found that resolving UUIDs to meaningful names“significantly improves Claude’s precision in retrieval tasks.”
- Let the model choose the level of detail. In Anthropic’s example, a “concise”
response_formatreturned 72 tokens, against 206 for “detailed.” - Paginate, filter and truncate by default. Anthropic’s Claude Code limits tool responses to 25,000 tokens by default. When you truncate, say so in the response and tell the model how to narrow its request.
- Use structured output where it helps. MCP lets a tool declare an
outputSchemaand returnstructuredContent, which clients can validate.
Errors are part of the prompt
A model can recover from an error only if the error tells it how. The MCP spec separates two kinds of errors. Protocol errors, such as calling a tool that doesn’t exist, are mainly for the client. Tool execution errors come back with isError: true and “contain actionable feedback that language models can use to self-correct,” so clients should pass them to the model.
Compare:
The first gives the model nothing to fix. The second tells it exactly what to change.
4. Scaling past 20 tools: three patterns #
Connect a few MCP servers and you’ll pass 20 tools quickly. There are three ways to stop every definition up front, and each moves the cost somewhere else.
Toolsets. Group tools and enable only the groups a task needs. GitHub’s default toolsets follow this pattern, and Google recommends choosing tools dynamically based on the conversation when you have many. It’s the simplest option, and it works when you can predict what a task needs.
Tool search. Give the model a tool for finding other tools. Anthropic’s Tool Search Tool cut context use by about 85% in its example, and raised Claude Opus 4’s accuracy on its MCP evaluations from 49% to 74%. Anthropic suggests it once your definitions pass about 10K tokens or you have more than 10 tools. OpenAI now supports deferring tools through tool search as well, on gpt-5.4 and later.
Code execution. Present tools as a typed code API and let the model write the code that connects them. Cloudflare’s Code Mode argues that “LLMs are better at writing code to call MCP, than at calling MCP directly.” Its reasoning is that models have seen far more real TypeScript than examples of tool calls. Anthropic describes the same idea in code execution with MCP, where a Google Drive-to-Salesforce workflow dropped from about 150,000 tokens to 2,000, because the intermediate data never passed through the model. Anthropic’s Programmatic Tool Calling cut average token use by 37% on complex research tasks.
The cost is real. Anthropic says running agent-generated code requires “a secure execution environment with appropriate sandboxing,” plus resource limits and monitoring. Cloudflare’s sandboxes have no internet access, and the code reaches tools only through bindings that hold the credentials. If you can’t operate a sandbox like that, use toolsets or tool search.
5. When a tool should be an agent #
Sometimes a tool’s job grows big enough to need its own prompt, tools and context. That’s when teams reach for multiple agents. There are two common shapes, and they behave differently.
OpenAI’s agent guide recommends getting the most out of a single agent first. When you do split, the Agents SDK docs distinguish agents as tools, where a manager keeps control, from handoffs, where another agent takes over the conversation. If one agent should own the conversation and combine the results, use agents as tools. Use handoffs when the specialist should talk to the user directly.
Anthropic’s research system shows both the benefit and the bill. Its multi-agent setup outperformed a single Claude Opus 4 agent by 90.2% on an internal research evaluation, but multi-agent systems used about 15 times as many tokens as a chat. Anthropic says the approach fits poorly when agents need the same context or depend heavily on each other, which includes most coding tasks. Those numbers come from Anthropic’s system, not yours, but the lesson carries over: split for parallel, independent work, not for tidiness.
6. Tools are where the damage happens #
Almost every risk an agent poses runs through a tool. An agent can only read, write, spend or leak what its tools allow. That makes tool design a security and reliability problem, not just a prompting problem.
The lethal trifecta
Simon Willison’s lethal trifecta is the clearest way to think about this. An agent that has access to private data, is exposed to untrusted content and has a way to send data out can be tricked into leaking that data. Models follow instructions wherever they appear, and they can’t reliably tell yours from an attacker’s.
This has already happened. In May 2025, Invariant Labs showed that a malicious issue in a public GitHub repository could make an agent using the GitHub MCP server read the user’s private repositories and publish their contents in a public pull request. Invariant stressed that this wasn’t a bug in the server’s code but a problem with how agents handle untrusted input. In trifecta terms, one agent held all three capabilities at once.
The fix is in how you combine tools, not in a better prompt. Don’t give one agent all three. If it reads untrusted content, limit what it can do afterwards.
Rate tools by risk, and confirm the dangerous ones
OpenAI recommends rating each tool as low, medium or high risk, based on read versus write access, reversibility, required permissions and financial impact, and then pausing or escalating to a human before high-risk actions. The MCP spec says there should always be a human in the loop who can deny tool calls, and Google says to confirm consequential calls with the user before running them.
One trap: MCP tool annotations describe how a tool behaves, but the spec says clients must treat them as untrusted unless they come from trusted servers. A server saying its tool is harmless doesn’t make it so.
Keep credentials out of the model’s reach
Neither the model nor any code it writes should hold real credentials.
- Anthropic’s Managed Agents architecture stores OAuth tokens in a vault. Claude calls MCP tools through a proxy using a session-scoped token, and the proxy fetches the real credentials.
- Cloudflare’s Code Mode keeps API keys in bindings outside the sandbox, so generated code never sees them.
- The MCP spec forbids token passthrough: servers must not accept tokens that weren’t issued for them . It also recommends starting with minimal scopes and requesting more only when a privileged operation is first attempted.
Assume a write can run twice
Agents retry, and harnesses crash and resume. Anthropic’s Managed Agents design keeps an append-only session log so a crashed harness can resume from the last recorded event. That’s good design, but it has a consequence: if a crash happens after a tool runs and before its result is recorded, the resumed agent may call that tool again.
None of the guidance cited here spells this out, so this one is my recommendation: give every write tool an idempotency key, so a repeated refund_order call refunds only once. Put a timeout on every call and a rate limit on every server. The MCP spec requires servers to rate limit tool calls and recommends that clients set timeouts. I wrote about how unbounded retries turn a blip into an outage in Retries as a Load Multiplier. An agent that retries on its own adds one more layer of retries to that picture.
Pin what your tools run on
A tool server is a dependency with access to everything its credentials allow. Corelight found that an MCP server pinned to a specific release produced 15 different environments across 19 builds over five weeks, because its launcher re-resolved dependencies on every rebuild. Install from a lock file, pin to a commit, and review updates as you would any other dependency.
A reference architecture
Here’s how the pieces fit together. The gateway layer is my synthesis of the guidance above, not any single vendor’s product.
The model chooses which tool to call. The gateway decides whether the call is allowed, makes it safe to repeat and records it. The session log makes the whole run recoverable.
7. Evaluate tools the way you test code #
You can’t know whether a tool definition works until you run it against realistic tasks. Anthropic’s process is the most concrete I’ve seen:
- Write realistic tasks, some needing dozens of tool calls, each with an outcome you can verify.
- Track accuracy, runtime, number of tool calls, token use and tool errors.
- Read the transcripts, especially where the agent hesitated or chose the wrong tool.
- Have an agent analyze the transcripts and improve the tools, and keep a held-out test set so you don’t overfit.
Anthropic says most of the advice in its tools post came out of this loop. In its research system, a tool-testing agent that rewrote flawed tool descriptions helped later agents complete tasks 40% faster.
The model decides. Your system answers for it. #
Better models will choose tools more reliably. They won’t make your refunds idempotent, your tokens narrowly scoped or your MCP servers pinned. Those are engineering decisions, and they’re what turn an agent that works in a demo into one you can trust in production.