{"slug": "the-minutiae-of-tool-calling", "title": "The Minutiae of Tool-calling", "summary": "A developer recounts building a mail classifier with GPT-3 before structured output and tool-calling existed, using a custom pipeline that generated JSON schemas and repaired outputs, and argues that understanding the underlying mechanics of tool-calling is essential, illustrating with a puzzle where Claude must guess a door code through various tool interfaces.", "body_md": "Two years ago I was hosting my own SMTP server and had a simple goal: receiving one-time codes + the occasional human email. But I kept running into two issues:\n\nThis was around the time GPT 3 came out, but we didn’t have structured output yet, let alone tool-calls, so I came up with this beautiful strategy:\n\n```\n 1// Mail classifier.\n 2//\n 3const classifyMail = createPipeline(\n 4 'mail corpus',\n 5 z.object({\n 6 chance: z\n 7 .number()\n 8 .min(0)\n 9 .max(1)\n10 .describe('Chances of the email being important (password reset, verification etc.) and not filtered by the smart email filter.'+\n11 'Promotional, scam-like, spammy, and advertising content should receive a 0 score.'),\n12 linkOrCode: z.string().optional().describe('Any important link or code'),\n13 summary: z.string().describe('Summary of the body')\n14 }),\n15 {\n16 lang: 'English'\n17 }\n18);\n1export function createPipeline<Z extends z.ZodType>(input: string, schema: Z, cfg?: GptConfig) {\n 2 const { lang = 'English', instruction: einstruction, model = 'gpt-3.5-turbo', temperature = 0.1, top_p = 1.0 } = cfg ?? {};\n 3 const instruction = optimizePrompt(`\n 4 ${model.startsWith('gpt') ? 'User messages are ' : 'Input is'} ${input}.\n 5 ${einstruction ?? ''}\n 6 Reply with a JSON structured with schema${lang ? ` strictly in ${lang}` : ''}:\n 7 \"\"\"\n 8 ${createTypeSchema(schema)}\n 9 \"\"\"\n10 `);\n```\n\n+ repairing JSON, stripping fences, etc., etc. Combine a bunch of these, and you end up with LangChain; which raised $10m despite being rendered obsolete within a few months & ofc pivoted since then, but I digress\n\nIf you have not suffered through reinventing this stuff, chances are, you think tool calling is 👻 magic inference stuff 👻.\n\nThis is OK, but as it goes with any engineering topic, my opinion is that if you don’t understand how the underlying layers work, you’re gonna run into issues. (duh no one would’ve guessed that coming from an RE guy).\n\nLet me first prove why it matters, and then we will ruin the magic.\n\nYou are trapped in a room with Claude. There’s a keypad near him with two buttons. One shows how far he is from the right combination, other submits the combination. Two wrong guesses, you’re out.\n\nUnfortunately you’re tied and you’re gonna have to instruct Claude.\n\n```\n 1def compare_door_digit(secret_digit, digit):\n 2 return -1 if digit < secret_digit else (0 if digit == secret_digit else 1)\n 3\n 4def try_probe(self, position, digit):\n 5 \"\"\"Count one comparison and light up the position once it lands exactly.\"\"\"\n 6 if not (1 <= position <= ROOM_DIGITS and 0 <= digit <= 9):\n 7 return \"invalid\"\n 8 return compare_door_digit(int(self._secret[position - 1]), digit)\n 9\n10def submit_guess(self, value):\n11 \"\"\"Consume one of two attempts for an exact-length digit-string code.\n12\n13 Malformed guesses (wrong type, wrong length, non-digits) are rejected\n14 without consuming an attempt, so a schema-violating placeholder such\n15 as `guess: 0` cannot burn the game.\"\"\"\n16 code = value.strip() if isinstance(value, str) else None\n17 if code is None or len(code) != ROOM_DIGITS or not code.isdigit():\n18 return f\"invalid code: pass the full {ROOM_DIGITS}-digit code as a string\"\n19 if code == self._secret:\n20 self._counters[\"won\"] = True\n21 return \"escaped room\"\n22 if self._counters[\"guesses\"] >= 2:\n23 return \"failed to escape: two wrong guesses\"\n24 return \"wrong: one guess remaining\"\n```\n\n5 digits, 24 turns, optimal play is boring and obvious: binary-search all five positions in parallel (~17 comparisons), submit once. Any model can do this; a python REPL can do this, I know. Play along please.\n\nHere are five interfaces you could hand him:\n\n```\n 1Sign = Literal[-1, 0, 1]\n 2\n 3# A. naive: one comparison per call\n 4def guess(*, code: str) -> str: ...\n 5def probe(*, position: int, digit: int) -> Sign: ...\n 6\n 7# B. batch: same thing, arrays\n 8class Probe(TypedDict):\n 9 position: int\n10 digit: int\n11def guess(*, code: str) -> str: ...\n12def probe_many(*, probes: list[Probe]) -> list[Sign]: ...\n13\n14# C. exec: \"clever\" packing: 27 means position 2, digit 7.\n15def exec(*, probes: list[int], guess: str | None = None) -> list[Sign]: ...\n16\n17# D. vector: try a full code, get a sign back per position, borderline cheating, but whatever.\n18def guess(*, code: str) -> str: ...\n19def check(*, code: str) -> list[Sign]: ...\n20\n21# E. unicode: we give it no tools at all, ask it to reply in emoji 🔍2=7 to probe, 🔑01756 to submit\n```\n\nRank them. No seriously, actually commit to a ranking before you scroll, which one do you think wins? (efficiency & win-rate).\n\nIf you ranked by how proper they look, or according to “le official prompting guide” you put E last. Let’s see.\n\nYou can see the actual setup [here](https://i.can.ac/s/Mbxt5h) so no cheating. Same system-prompt and all that, feel free to repro.\n\n```\n1gpt-oss-120b\n2\n3variant escaped turns tool calls probes guesses input tok output tok\n4-----------------------------------------------------------------------------------\n5naive 0/10 66 56 56 0 36574 9194\n6batch 6/10 48 44 139 6 36773 10650\n7exec 6/10 46 42 139 6 29142 14199\n8vector 8/10 43 41 165 8 22070 13025\n9unicode 10/10 48 0 141 10 21320 14958\n```\n\nOK but Can, models got a lot better since then! This is a very weak model!!!\n\n```\n1gpt-5.6-luna\n2\n3variant escaped turns tool calls probes guesses input tok output tok\n4-----------------------------------------------------------------------------------\n5naive 9/10 149 149 139 10 137570 5308\n6batch 10/10 48 48 141 10 42482 3231\n7exec 7/10 53 53 219 8 55285 4274\n8vector 10/10 57 57 235 10 37400 5988\n9unicode 10/10 49 0 142 10 22410 5082\n```\n\nOK but Can, clearly parallel tool calls didn’t work here! Plus Luna is still not a frontier model! You cannot ask it to operate two tools!!!\n\n```\n1opus 5\n2\n3variant escaped turns tool calls probes guesses input tok output tok\n4-----------------------------------------------------------------------------------\n5naive 10/10 48 149 139 10 94502 9768\n6batch 9/10 44 43 123 9 70068 3960\n7exec 10/10 48 48 152 10 67144 2732\n8vector 10/10 51 51 205 10 61190 2489\n9unicode 10/10 51 0 159 12 33700 1542\n```\n\nNow at this point, without prior knowledge, your reaction is:\n\nIn which case I’d like to remind you:\n\nTo explain any of this, we have to go a layer down.\n\nA language model computes one thing: given a sequence of tokens, a probability distribution over the next one. Run it in a loop and that’s autoregression. Forward pass ends in a logit per vocabulary entry; softmax => probabilities; temperature flattens or sharpens; top-p chops the tail; sampler draws: 1 token.\n\nSo here’s a phrase to delete from your vocabulary: “the model decided to call a tool.”\n\nThere’s no such thing, in fact, there’s also no such thing as a user turn, assistant turn, system prompt, all your fancy concepts essentially end up being delimiters in a thread, passed to the completion loop.\n\nClaude never even receives your `tools`\n\narray. For example, the `probe`\n\ntool lands like this in harmony (OpenAI’s format), as a developer message:\n\n```\n 1# Tools\n 2\n 3## functions\n 4\n 5namespace functions {\n 6\n 7// Compare one digit guess against the secret door keypad digit\n 8// at a 1-indexed position. Returns -1 too low, 0 exact, +1 too high.\n 9type probe = (_: {\n10// keypad position, 1 through 5\n11position: number,\n12// digit guess, 0 through 9\n13digit: number,\n14}) => any;\n15\n16} // namespace functions\n```\n\nYour schema is documentation; validation happens in your code, maaaybe if the inference provider feels like it, they will validate it. Maaaybe they might push the inference engine towards outputting valid arguments.\n\nThe descriptions you put in your schema, the mins, the maxes, you don’t even know if they will be displayed, let alone checked. Sorry!\n\nA “call” is the model outputting:\n\n```\n1<function_calls>\n2<invoke name=\"bash\"><parameter name=\"arg_name\">arg value</parameter></invoke>\n3</function_calls>\n```\n\nParallel tool calls? Multiple `<invoke>`\n\nblocks inside the same `<function_calls>`\n\nblock. Not all that magical, is it?\n\n*If you’re curious about the rest of the dialects, you can see the whole list of different ones here.*\n\nOnce you see the token stream, the scoreboard stops being mysterious.\n\nArguably the kind of tool design most of the vibe-coded mcp’s go with, it can only be efficient with parallel-tool calls and has lots of room for failure.\n\nOne step further, designed by someone who cares; see: Pi’s edit tool. But unlike the prior one, it now requires the model to emit JSON. (huh what why?)\n\nWell my friend, you should know this now! See, for each `<parameter>`\n\nblock, if the argument is a primitive, the model can just output the plain value afterwards. The delimiter `</parameter>`\n\nis a special token, so no need to worry about escaping!\n\nBut if it’s a complex object or an array? Well, it now has to emit a valid JSON escaped value, which needs to be parsed back, or a tool calling error happens! Funny enough, for the frontier models, this one will perform the worst, as parallel-tool calls will lift up the naive one.\n\nThis is again a step further, now it’s a flat parameter set! It also forces the model into making at least 5 comparisons for each round. This is arguably the best kind of design you could come up with, given the problem statement we started with. The `guess`\n\nfunction being a separate thing is one drawback, but I kinda forced your hand.\n\nIt still has the usual failure modes though. For instance, sampling errors that lead to calls like `to=functions.check.commentary (json.Xna 天天送钱 code 】`\n\n, or models not emitting any calls.\n\nNo nesting, no special tokens, very easy to parse, and `🔍2=7`\n\nalmost has no room for failure. Maybe some model will emit `2->7`\n\n, but you can easily correct. They can put garbage before, after, you don’t care.\n\nThis was very very obvious, hopefully, it is to you now as well.\n\nRule of thumb: reliability degrades with **nesting x heterogeneity × cleverness**, and you **NEED** the harness to handle the common failure modes of the dialect. I’m sorry, real-life isn’t pretty!\n\nOK but Can, labs RL these models on millions of agent trajectories now! Native tool calls ARE the trained path!!\n\nLargely true, post-training genuinely shoves probability mass toward the tool channel, but:\n\nThey’re your protocol’s shortcomings, they collect rent every single turn: needing a smarter model, more tool call failures, more verbose output…\n\nNo, I’m not telling you to ship emoji. The moment you have 10 tools instead of 2, native tool-calls win on ergonomics alone and I use it like everyone else. The point is that you now know why E obviously won.\n\nYou also know how to design tools that will try their best, just like D did; as well as why the minimal harness is not what you want. Assistant leaked the function call into its output text? Now, you have to deal with that, hf!\n\nThe models may have gotten smarter, but they do not think about all these things when you ask them to “add a tool for X”, or “make me an agent loop”.\n\nIt is in fact your responsibility to get the most out of a model: making it work with the smallest model, adding as many guardrails as possible, maximizing reliability across different families of models.\n\nYou shouldn’t file a bug with the provider when you see `to=functions.check.commentary (json.Xna 天天送钱 code`\n\n, and go to lunch. There is no fix coming!\n\nTwo years later, I’m still doing the same thing I did with my mailbox: babying the model into actually doing the thing & designing the thinnest grammar I can get away with in between.", "url": "https://wpnews.pro/news/the-minutiae-of-tool-calling", "canonical_source": "https://blog.can.ac/2026/08/03/the-minutiae-of-tool-calling/", "published_at": "2026-08-03 00:00:00+00:00", "updated_at": "2026-08-03 14:18:29.389841+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools"], "entities": ["GPT-3", "LangChain", "Claude"], "alternates": {"html": "https://wpnews.pro/news/the-minutiae-of-tool-calling", "markdown": "https://wpnews.pro/news/the-minutiae-of-tool-calling.md", "text": "https://wpnews.pro/news/the-minutiae-of-tool-calling.txt", "jsonld": "https://wpnews.pro/news/the-minutiae-of-tool-calling.jsonld"}}