# Stop Streaming Tools Through Your LLM

> Source: <https://dev.to/shridhar_shah2297/stop-streaming-tools-through-your-llm-2f1i>
> Published: 2026-08-05 16:19:51+00:00

*The 2026 shift from tool-calling to Code Mode: let the agent write one script instead of narrating fifty tool calls — and watch context tokens drop ~99%.*

**TL;DR:** The classic agent loop loads *every* tool definition into the context window and pipes *every* intermediate result back through the model. Connect a few dozen tools and the context is full before the user even speaks. The 2026 move — **Code Mode** — is to let the agent write one short script that calls tools directly; bulk data stays in a sandbox and only the final answer comes back. In a tiny runnable demo it cut context from **36,781 tokens to 222** — a **99.4%** reduction. Same answer, no API key.

**Mental model:** instead of reading a worker every page of 50 manuals and every row of a spreadsheet out loud, you hand them the 3 manuals the job needs and let them do the math at their own desk. You only get back the final answer — not the raw data.

Function/tool-calling is how most agents act today. It works beautifully with five tools. But the whole tool surface gets serialized into the context window *on every request*, and every intermediate result the model asks for is streamed back through the context too.

So a task like *"count open support tickets per plan tier and save the report"* looks like:

`list_tickets`

→ 2,000 rows come back `get_customers`

→ 400 rows come back `save_report`

.The model paid for 50 schemas it mostly didn't need, and for 2,400 rows of raw data it only needed to *aggregate*, not *read*. Anthropic measured a real Google-Drive-to-Salesforce task at **150,000 tokens**; Cloudflare hit **~1.17M tokens** of tool definitions on a 2,500-endpoint API.

Code Mode (Anthropic's "code execution with MCP", Cloudflare's "Code Mode") flips the loop:

Here's the entire script the "model" writes in the demo — the 2,000 tickets and 400 customers are joined *inside the sandbox* and never touch the context:

```
tickets = list_tickets("open")            # 2000 rows: fetched and joined entirely in the sandbox
plan = {c["id"]: c["plan"] for c in get_customers()}
counts = {t: 0 for t in ("free", "pro", "enterprise")}
for tk in tickets:
    counts[plan[tk["customer_id"]]] += 1
rows = [{"tier": t, "open_tickets": counts[t]} for t in counts]
result = save_report("open_by_tier", rows)
```

The sandbox exposes only the tool API — no builtins, no imports — so the script can compose tools but can't reach the rest of the process:

```
api = {"list_tickets": list_tickets, "get_customers": get_customers, "save_report": save_report}
sandbox = {"__builtins__": {}, **api}
exec(script, sandbox)      # only the script text + final answer ever cross the context window
Code Mode — write code that calls tools, don't stream tools through the model

  Task: count open tickets per plan tier over 2000 tickets / 400 customers.
  Tools connected to the agent: 50 (this task needs 3).

   classic tool-calling    36,781 context tokens   (all schemas + raw data pass through)
   code mode                  222 context tokens   (3 signatures + one script + answer)
   ------------------------------------------------
   context reduction        99.4%

  Same result either way: [free: 720, pro: 596, enterprise: 684]
```

The gap isn't a constant — it **compounds** with tool count and data size.

**Reality check:** the exact figure is from the toy model above — treat it as directional, not a benchmark. The *shape* is real and measured in production: Anthropic's Drive→Salesforce task dropped from ~150k to ~2k tokens, and independent reproductions land anywhere from 78% to 99.9% depending on tool count and data size.

The proven part: LLMs are extremely good at writing code, and a code API is a far denser way to express "do these five things and combine the results" than five separate tool-call round trips. Anthropic, Cloudflare, and an [independent study of MCP design choices](https://arxiv.org/abs/2602.15945) all converge on the same finding — token usage becomes roughly **constant in tool count** because the model only reads what it opens.

Where it's heading: as agents connect to hundreds of MCP servers, "load everything up front" simply stops being viable, and the tool-calling layer moves *onto the computer*. Expect the sandbox — not the tool-call — to become the default action primitive, with tool schemas synced to a filesystem and disclosed on demand.

It's a minimal model, not a benchmark: "tokens" are a `chars/4`

proxy, the "model" writes a fixed script, and the sandbox is a restricted `exec`

. Real systems generate the script with an LLM and isolate it far more aggressively (gVisor, containers, V8 isolates) — which is the pattern's main cost: **you now run untrusted, model-written code**, so sandboxing and limits are mandatory. The token economics, though, are exactly what production reports show.

```
python3 demo.py   # standard library only
```

**Papers**

**Engineering blogs**