{"slug": "day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return", "title": "Day 13/90: Define What Your AI Tools Can Accept—and What They Must Return", "summary": "A developer working through a 90-day AI learning series outlined a practical approach to tool contracts for LLM function calling, arguing that tool selection, argument validity, and permitted execution are separate concerns that must be evaluated independently. Drawing on the Berkeley Function Calling Leaderboard, JSON Schema, Pydantic's strict mode, and the Model Context Protocol's optional outputSchema, the writeup proposes explicit input and output contracts — rejecting malformed arguments like a string \"limit\" or blank query — while noting that schema validation does not establish data access permission or evidence quality.", "body_md": "An LLM chooses a search tool and sends this:\n\n```\n{\n  \"query\": \"What is our refund policy?\",\n  \"limit\": \"a few\"\n}\n```\n\nThe request is understandable to a person. The backend still needs an integer.\n\nShould it guess what “a few” means? Use a default? Return an error?\n\nThis is where tool contracts become useful. While exploring this topic, I looked at function-calling research, JSON Schema, and the Model Context Protocol specification.\n\nOne practical lesson stood out: **choosing a tool, producing valid arguments, and executing an allowed action are separate concerns.**\n\nThe Berkeley Function Calling Leaderboard (BFCL), published at ICML 2025, evaluates whether models can make correct function calls across different settings. It treats function calling as a capability that needs explicit evaluation. A fluent response alone doesn’t demonstrate correct tool use.\n\nMy engineering takeaway is to evaluate the actual call: which tool was selected, which arguments were supplied, and whether execution produced the intended result.\n\nA schema helps with part of that problem.\n\nFor a document search tool, an input contract could require:\n\nJSON Schema provides keywords for declaring properties, required fields, and whether additional properties are allowed. Defining a property does **not** automatically make it required; that needs to be specified separately.\n\nUnder our proposed contract:\n\n| Payload | Expected outcome | \n|---|---|\n| `{\"query\": \"refund policy\", \"limit\": 3}` | Accept | \n| `{\"query\": \"refund policy\"}` | Reject: missing limit | \n| `{\"query\": \"refund policy\", \"limit\": \"a few\"}` | Reject: incorrect type | \n| `{\"query\": \"refund policy\", \"limit\": 100}` | Reject: outside allowed range | \n| `{\"query\": \"   \", \"limit\": 3}` | Reject: blank query | \n\nThese are illustrative cases, not measured experiment results. They make the intended behavior concrete before implementation.\n\nThere’s a subtler case:\n\n```\n{\n  \"query\": \"refund policy\",\n  \"limit\": \"3\"\n}\n```\n\nShould `\"3\"` become `3`, or should validation fail?\n\nThat depends on the validation policy.\n\nPydantic’s documentation distinguishes its usual coercion behavior from strict mode, which restricts conversions and rejects incorrect types more aggressively. Strict behavior also depends on the type and whether validation receives Python objects or JSON.\n\nFor this tool’s integer limit, I would reject the string and return a specific error. The key is to choose that behavior deliberately and test it.\n\nThe application needs predictable results after execution.\n\nA successful search might return:\n\n```\n{\n  \"status\": \"success\",\n  \"results\": [\n    {\n      \"document_id\": \"policy_014\",\n      \"text\": \"Refund requests must be submitted within 30 days.\"\n    }\n  ]\n}\n```\n\nMCP supports an optional `outputSchema` for structured tool results. When one is declared, the specification requires servers to return conforming structured results and recommends that clients validate them.\n\nThat gives downstream code an explicit structure to work with.\n\nIt still doesn’t prove that the retrieved text is accurate or relevant. Structural validity and evidence quality need separate checks.\n\nI would represent these outcomes differently:\n\n```\n{\"status\": \"success\", \"results\": []}\n{\n  \"status\": \"error\",\n  \"error\": {\n    \"code\": \"SEARCH_UNAVAILABLE\"\n  }\n}\n```\n\nThe first means the search completed without matches. The second means the search could not be completed.\n\nThis is an application design choice. It helps the next step decide whether to report missing evidence, retry, or explain a temporary failure.\n\nA perfectly typed request can still ask for a document the user cannot access.\n\nThe MCP specification lists input validation and access controls as separate server responsibilities. Passing schema validation does not establish permission.\n\nFor a retrieval tool, I would therefore check both the argument contract and the caller’s access to the requested data.\n\n**My takeaway: a tool schema defines what can enter and leave a tool. Runtime validation enforces that contract. Authorization and backend checks determine which actions are allowed.**\n\nFor a small experiment, I’d start with accepted and rejected payloads, malformed outputs, empty results, and tool failures—and record the observed behavior for each.\n\nWhat has caused more trouble in your AI applications: unexpected tool arguments or inconsistent tool responses?", "url": "https://wpnews.pro/news/day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return", "canonical_source": "https://dev.to/rimsha_kanwal/day-1390-define-what-your-ai-tools-can-accept-and-what-they-must-return-4gbn", "published_at": "2026-11-02 05:44:10+00:00", "updated_at": "2026-11-02 05:52:11.214870+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "agent-protocols", "large-language-models", "ai-safety"], "entities": ["Model Context Protocol", "Berkeley Function Calling Leaderboard", "Pydantic", "ICML"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return", "markdown": "https://wpnews.pro/news/day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return.md", "text": "https://wpnews.pro/news/day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return.txt", "jsonld": "https://wpnews.pro/news/day-13-90-define-what-your-ai-tools-can-accept-and-what-they-must-return.jsonld"}}