# The Minutiae of Tool-calling

> Source: <https://blog.can.ac/2026/08/03/the-minutiae-of-tool-calling/>
> Published: 2026-08-03 00:00:00+00:00

Two years ago I was hosting my own SMTP server and had a simple goal: receiving one-time codes + the occasional human email. But I kept running into two issues:

This was around the time GPT 3 came out, but we didn’t have structured output yet, let alone tool-calls, so I came up with this beautiful strategy:

```
 1// Mail classifier.
 2//
 3const classifyMail = createPipeline(
 4 'mail corpus',
 5 z.object({
 6 chance: z
 7 .number()
 8 .min(0)
 9 .max(1)
10 .describe('Chances of the email being important (password reset, verification etc.) and not filtered by the smart email filter.'+
11 'Promotional, scam-like, spammy, and advertising content should receive a 0 score.'),
12 linkOrCode: z.string().optional().describe('Any important link or code'),
13 summary: z.string().describe('Summary of the body')
14 }),
15 {
16 lang: 'English'
17 }
18);
1export function createPipeline<Z extends z.ZodType>(input: string, schema: Z, cfg?: GptConfig) {
 2 const { lang = 'English', instruction: einstruction, model = 'gpt-3.5-turbo', temperature = 0.1, top_p = 1.0 } = cfg ?? {};
 3 const instruction = optimizePrompt(`
 4 ${model.startsWith('gpt') ? 'User messages are ' : 'Input is'} ${input}.
 5 ${einstruction ?? ''}
 6 Reply with a JSON structured with schema${lang ? ` strictly in ${lang}` : ''}:
 7 """
 8 ${createTypeSchema(schema)}
 9 """
10 `);
```

+ repairing JSON, stripping fences, etc., etc. Combine a bunch of these, and you end up with LangChain; which raised $10m despite being rendered obsolete within a few months & ofc pivoted since then, but I digress

If you have not suffered through reinventing this stuff, chances are, you think tool calling is 👻 magic inference stuff 👻.

This is OK, but as it goes with any engineering topic, my opinion is that if you don’t understand how the underlying layers work, you’re gonna run into issues. (duh no one would’ve guessed that coming from an RE guy).

Let me first prove why it matters, and then we will ruin the magic.

You are trapped in a room with Claude. There’s a keypad near him with two buttons. One shows how far he is from the right combination, other submits the combination. Two wrong guesses, you’re out.

Unfortunately you’re tied and you’re gonna have to instruct Claude.

```
 1def compare_door_digit(secret_digit, digit):
 2 return -1 if digit < secret_digit else (0 if digit == secret_digit else 1)
 3
 4def try_probe(self, position, digit):
 5 """Count one comparison and light up the position once it lands exactly."""
 6 if not (1 <= position <= ROOM_DIGITS and 0 <= digit <= 9):
 7 return "invalid"
 8 return compare_door_digit(int(self._secret[position - 1]), digit)
 9
10def submit_guess(self, value):
11 """Consume one of two attempts for an exact-length digit-string code.
12
13 Malformed guesses (wrong type, wrong length, non-digits) are rejected
14 without consuming an attempt, so a schema-violating placeholder such
15 as `guess: 0` cannot burn the game."""
16 code = value.strip() if isinstance(value, str) else None
17 if code is None or len(code) != ROOM_DIGITS or not code.isdigit():
18 return f"invalid code: pass the full {ROOM_DIGITS}-digit code as a string"
19 if code == self._secret:
20 self._counters["won"] = True
21 return "escaped room"
22 if self._counters["guesses"] >= 2:
23 return "failed to escape: two wrong guesses"
24 return "wrong: one guess remaining"
```

5 digits, 24 turns, optimal play is boring and obvious: binary-search all five positions in parallel (~17 comparisons), submit once. Any model can do this; a python REPL can do this, I know. Play along please.

Here are five interfaces you could hand him:

```
 1Sign = Literal[-1, 0, 1]
 2
 3# A. naive: one comparison per call
 4def guess(*, code: str) -> str: ...
 5def probe(*, position: int, digit: int) -> Sign: ...
 6
 7# B. batch: same thing, arrays
 8class Probe(TypedDict):
 9 position: int
10 digit: int
11def guess(*, code: str) -> str: ...
12def probe_many(*, probes: list[Probe]) -> list[Sign]: ...
13
14# C. exec: "clever" packing: 27 means position 2, digit 7.
15def exec(*, probes: list[int], guess: str | None = None) -> list[Sign]: ...
16
17# D. vector: try a full code, get a sign back per position, borderline cheating, but whatever.
18def guess(*, code: str) -> str: ...
19def check(*, code: str) -> list[Sign]: ...
20
21# E. unicode: we give it no tools at all, ask it to reply in emoji 🔍2=7 to probe, 🔑01756 to submit
```

Rank them. No seriously, actually commit to a ranking before you scroll, which one do you think wins? (efficiency & win-rate).

If you ranked by how proper they look, or according to “le official prompting guide” you put E last. Let’s see.

You can see the actual setup [here](https://i.can.ac/s/Mbxt5h) so no cheating. Same system-prompt and all that, feel free to repro.

```
1gpt-oss-120b
2
3variant escaped turns tool calls probes guesses input tok output tok
4-----------------------------------------------------------------------------------
5naive 0/10 66 56 56 0 36574 9194
6batch 6/10 48 44 139 6 36773 10650
7exec 6/10 46 42 139 6 29142 14199
8vector 8/10 43 41 165 8 22070 13025
9unicode 10/10 48 0 141 10 21320 14958
```

OK but Can, models got a lot better since then! This is a very weak model!!!

```
1gpt-5.6-luna
2
3variant escaped turns tool calls probes guesses input tok output tok
4-----------------------------------------------------------------------------------
5naive 9/10 149 149 139 10 137570 5308
6batch 10/10 48 48 141 10 42482 3231
7exec 7/10 53 53 219 8 55285 4274
8vector 10/10 57 57 235 10 37400 5988
9unicode 10/10 49 0 142 10 22410 5082
```

OK but Can, clearly parallel tool calls didn’t work here! Plus Luna is still not a frontier model! You cannot ask it to operate two tools!!!

```
1opus 5
2
3variant escaped turns tool calls probes guesses input tok output tok
4-----------------------------------------------------------------------------------
5naive 10/10 48 149 139 10 94502 9768
6batch 9/10 44 43 123 9 70068 3960
7exec 10/10 48 48 152 10 67144 2732
8vector 10/10 51 51 205 10 61190 2489
9unicode 10/10 51 0 159 12 33700 1542
```

Now at this point, without prior knowledge, your reaction is:

In which case I’d like to remind you:

To explain any of this, we have to go a layer down.

A language model computes one thing: given a sequence of tokens, a probability distribution over the next one. Run it in a loop and that’s autoregression. Forward pass ends in a logit per vocabulary entry; softmax => probabilities; temperature flattens or sharpens; top-p chops the tail; sampler draws: 1 token.

So here’s a phrase to delete from your vocabulary: “the model decided to call a tool.”

There’s no such thing, in fact, there’s also no such thing as a user turn, assistant turn, system prompt, all your fancy concepts essentially end up being delimiters in a thread, passed to the completion loop.

Claude never even receives your `tools`

array. For example, the `probe`

tool lands like this in harmony (OpenAI’s format), as a developer message:

```
 1# Tools
 2
 3## functions
 4
 5namespace functions {
 6
 7// Compare one digit guess against the secret door keypad digit
 8// at a 1-indexed position. Returns -1 too low, 0 exact, +1 too high.
 9type probe = (_: {
10// keypad position, 1 through 5
11position: number,
12// digit guess, 0 through 9
13digit: number,
14}) => any;
15
16} // namespace functions
```

Your schema is documentation; validation happens in your code, maaaybe if the inference provider feels like it, they will validate it. Maaaybe they might push the inference engine towards outputting valid arguments.

The descriptions you put in your schema, the mins, the maxes, you don’t even know if they will be displayed, let alone checked. Sorry!

A “call” is the model outputting:

```
1<function_calls>
2<invoke name="bash"><parameter name="arg_name">arg value</parameter></invoke>
3</function_calls>
```

Parallel tool calls? Multiple `<invoke>`

blocks inside the same `<function_calls>`

block. Not all that magical, is it?

*If you’re curious about the rest of the dialects, you can see the whole list of different ones here.*

Once you see the token stream, the scoreboard stops being mysterious.

Arguably the kind of tool design most of the vibe-coded mcp’s go with, it can only be efficient with parallel-tool calls and has lots of room for failure.

One step further, designed by someone who cares; see: Pi’s edit tool. But unlike the prior one, it now requires the model to emit JSON. (huh what why?)

Well my friend, you should know this now! See, for each `<parameter>`

block, if the argument is a primitive, the model can just output the plain value afterwards. The delimiter `</parameter>`

is a special token, so no need to worry about escaping!

But if it’s a complex object or an array? Well, it now has to emit a valid JSON escaped value, which needs to be parsed back, or a tool calling error happens! Funny enough, for the frontier models, this one will perform the worst, as parallel-tool calls will lift up the naive one.

This is again a step further, now it’s a flat parameter set! It also forces the model into making at least 5 comparisons for each round. This is arguably the best kind of design you could come up with, given the problem statement we started with. The `guess`

function being a separate thing is one drawback, but I kinda forced your hand.

It still has the usual failure modes though. For instance, sampling errors that lead to calls like `to=functions.check.commentary (json.Xna 天天送钱 code 】`

, or models not emitting any calls.

No nesting, no special tokens, very easy to parse, and `🔍2=7`

almost has no room for failure. Maybe some model will emit `2->7`

, but you can easily correct. They can put garbage before, after, you don’t care.

This was very very obvious, hopefully, it is to you now as well.

Rule of thumb: reliability degrades with **nesting x heterogeneity × cleverness**, and you **NEED** the harness to handle the common failure modes of the dialect. I’m sorry, real-life isn’t pretty!

OK but Can, labs RL these models on millions of agent trajectories now! Native tool calls ARE the trained path!!

Largely true, post-training genuinely shoves probability mass toward the tool channel, but:

They’re your protocol’s shortcomings, they collect rent every single turn: needing a smarter model, more tool call failures, more verbose output…

No, I’m not telling you to ship emoji. The moment you have 10 tools instead of 2, native tool-calls win on ergonomics alone and I use it like everyone else. The point is that you now know why E obviously won.

You also know how to design tools that will try their best, just like D did; as well as why the minimal harness is not what you want. Assistant leaked the function call into its output text? Now, you have to deal with that, hf!

The models may have gotten smarter, but they do not think about all these things when you ask them to “add a tool for X”, or “make me an agent loop”.

It is in fact your responsibility to get the most out of a model: making it work with the smallest model, adding as many guardrails as possible, maximizing reliability across different families of models.

You shouldn’t file a bug with the provider when you see `to=functions.check.commentary (json.Xna 天天送钱 code`

, and go to lunch. There is no fix coming!

Two years later, I’m still doing the same thing I did with my mailbox: babying the model into actually doing the thing & designing the thinnest grammar I can get away with in between.