I integrate tools into a production AI agent for a living — first-party ones, third-party ones, HTTP APIs wrapped as tools, MCP servers. After enough integrations, you notice something uncomfortable: whether a model uses a tool correctly has surprisingly little to do with how well the tool is engineered.
A tool can have clean code, solid auth, perfect uptime — and the model still picks the wrong one, invents arguments, or ignores it entirely. The failures cluster into a handful of patterns. Here are the five I keep seeing.
1. The parameter with no description #
The single most common failure, by a wide margin.
"url": { "type": "string" }
Your type system knows this is a string. The model needs to know which URL — the full page URL? just the domain? with or without protocol? URL-encoded? A model facing an undocumented parameter doesn't stop and ask. It guesses. Sometimes it guesses right. In production, "sometimes" is a bug report.
The root cause is almost always schema generation: zod, Pydantic, or an
OpenAPI converter produces structurally perfect schemas with zero semantics,
and nobody goes back to add .describe()
. The type checker is satisfied. The model is starving.
The fix costs an hour: every parameter gets a description with format and one example value. It is the highest-leverage hour you can spend on agent reliability, and nobody spends it.
2. Tools that compete for the same intent #
get_user
and get_users
. search_docs
and query_docs
. send_message
and post_message
.
To you, the difference is obvious — you wrote them. To a model doing tool selection over a catalog it has never seen before, two near-identical names with near-identical descriptions is a coin flip. And a coin flip at step 2 of a 6-step task is a 50% failure rate you'll never reproduce locally.
Two rules of thumb: if two tools' descriptions could be swapped without anyone noticing, the model can't tell them apart either. And every description should answer when to use this instead of the neighbors — the best catalogs cross-reference ("for bulk queries, use X instead").
3. Prose constraints instead of schema constraints #
"status": {
"type": "string",
"description": "Must be one of: active, inactive, banned"
}
That constraint lives in prose, which means the model can violate it.
Move it into "enum": ["active", "inactive", "banned"]
and it can't — most inference stacks enforce enums at decoding time.
The general principle: anything the schema can express, the schema should express. Descriptions are for semantics; constraints belong in the type system. Every constraint you leave in prose is a runtime error you've chosen to discover later.
4. The catalog tax #
Every tool you expose gets serialized into every single request. A catalog with 25 tools and generous schemas can cost several thousand tokens before the user has typed a word — you pay it on every turn, forever.
Worse than the money: selection accuracy degrades as catalogs grow. The model attends over everything; ten sharply-differentiated tools reliably beat thirty overlapping ones on both cost and correctness.
If your catalog is large, split it by domain, or gate rarely-used tools behind a mode. Your token bill and your error rate will both thank you.
5. Errors that strand the model #
When a model calls your tool wrong, your error message is the only feedback loop it has. Compare:
Internal Server Error
Missing required parameter "query". Expected a free-text search string, e.g. "login bug".
The first strands the model — it will retry the same call, or give up, or hallucinate a workaround. The second lets it self-correct in one turn. Error messages are prompts. Write them like prompts.
There's a nastier variant: the tool that silently accepts an invalid call —
declared required
fields that aren't actually enforced, so the call
"succeeds" with undefined
in the payload. A model can recover from a good error. It cannot recover from a lie.
The common thread #
None of these are engineering problems in the traditional sense. The code is fine. They're writing problems — descriptions, names, error messages — the parts that don't show up in code review because they don't break tests.
The mental model that helps: your tool definition is not an API contract, it's a prompt. It gets read by a language model under time pressure with no documentation open in another tab. Write it the way you'd write for a sharp intern on their first day: what this does, when to use it, what comes back, and what to do when it fails.
The teams that internalize this ship tools that agents use correctly on the first try. The teams that don't file bugs against the model.
Next up: how widespread are these failure modes across the MCP ecosystem, actually? I've been collecting data. It is not flattering. Stay tuned.