cd /news/ai-agents/your-mcp-tool-takes-base64-images-th… · home › topics › ai-agents › article
[ARTICLE · art-172682] src=pub.towardsai.net ↗ pub= topic=ai-agents verified=true sentiment=↓ negative

Your Mcp Tool Takes BASE64 Images. The Model Has to Type Every Character

An MCP server's upload_image tool that accepts base64-encoded image strings forces the language model itself to generate every character, and token counts measured with OpenAI's tiktoken library show the cost is prohibitive: a 197,429-byte PNG screenshot becomes 263,240 base64 characters and 176,713 tokens under the o200k_base encoding, more than GPT-4o's entire 128,000-token context window. The JustBlogged MCP server's schema caps the field at 11,184,812 characters (8 MB decoded), which at the measured rate of 1.47 characters per token for random bytes works out to roughly 7.6 million tokens. The author argues the schema's pattern check validates only the base64 alphabet and padding, so a single swapped character in a 263,240-character string still passes and the server stores a broken image, and that the MCP spec's recommendation to show tool inputs to users is impractical at that length.

recommendation

Replace base64 image fields in MCP tool schemas with URL or path inputs the server fetches itself; the source measured 176,713 tokens for one 197 KB screenshot, exceeding GPT-4o's 128,000-token context.

more in playbook: Designing agent systems
by read6 min views1 publishedNov 19, 2026

AI disclosure: I drafted this article with help from an AI assistant. The token counts below were measured with OpenAI’s tiktoken library on the files described, and the script is at the end so you can rerun it.

The MCP server for JustBlogged, the hosted blog platform I build, has an upload_image tool. This is the field that carries the image:

"data": {  "type": "string",  "maxLength": 11184812,  "pattern": "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$",  "description": "Base64-encoded image bytes without a data URL prefix. Maximum decoded size: 8 MB."}

11184812 is 8 MB of bytes after base64's 4/3 expansion, rounded up to whole 4-character groups. As a storage limit it is fine. What the schema doesn't show is who has to produce that string.

The MCP spec says tools are “model-controlled, meaning that the language model can discover and invoke tools automatically.” The client sends tools/call with an arguments object, and in the normal setup the model wrote that object. So a base64 data field isn't uploaded by the client. The model generates it, token by token, as part of its reply.

That makes the size of the string a generation problem, not a transport problem.

I base64-encoded four files and counted tokens with tiktoken’s o200k_base encoding (used by GPT-4o) and cl100k_base (used by GPT-4 and GPT-3.5). Other model families use other tokenizers, so treat these as one tokenizer's numbers, not a universal constant.

File                    | Bytes     | Base64 characters | Tokens (o200k_base) | Characters per token------------------------+-----------+-------------------+---------------------+---------------------100x100 PNG logo        | 1,939     | 2,588             | 1,355               | 1.913200x1600 PNG banner    | 108,858   | 145,144           | 90,998              | 1.601280x800 PNG screenshot | 197,429   | 263,240           | 176,713             | 1.491 MB of random bytes    | 1,048,576 | 1,398,104         | 954,064             | 1.47

cl100k_base counts were about 5% higher for every file.

For comparison, a 10,201-character English article draft came out at 4.29 characters per token with the same encoding. Base64 gets roughly a third of that. The random-bytes row is there because compressed image data (PNG, JPEG, WebP) is close to random from a tokenizer’s point of view, and the two larger PNGs already sit near that line.

At the random-bytes rate, the 8 MB cap in the schema above works out to about 7.6 million tokens.

For scale, OpenAI’s model page for GPT-4o, an older model but a common reference point, lists a 128,000-token context window and 16,384 max output tokens. The 197 KB screenshot needs more tokens than that whole context window.

When you paste a screenshot into a chat client, the model receives it as image input. It can read and describe it. It can’t reproduce the original file byte for byte from that.

The only way a model can emit the real base64 is if the base64 is already in its context as text, for example after a coding agent runs base64 screenshot.png in a shell. Then you pay twice: once to read the string in, once to write it back out.

Copying is also where it breaks. The pattern in the schema checks the alphabet and the padding, not the content. If the model swaps one character in the middle of a 263,240-character string, the argument still validates and the server stores a broken image.

There’s a review problem too. The spec says clients SHOULD “show tool inputs to the user before calling the server, to avoid malicious or accidental data exfiltration.” A quarter of a million characters of base64 is technically shown and practically unreviewable. Nobody can tell from looking at it what bytes it contains.

The right input depends on where the image is when the model wants to use it.

The image is already on the web. Accept a URL and let the server fetch it. JustBlogged’s create_post tool takes the featured image as "path or URL" for this reason. A sample URL like https://example.com/images/launch-screenshot.png is 10 tokens; a CDN URL with a UUID in it was 42. The cost moves to the server, which now fetches arbitrary URLs on a model's say-so, so it needs the usual SSRF guards from the OWASP cheat sheet: allow only http and https, resolve the hostname and reject private, loopback and link-local addresses, cap the download size, set a timeout, and check the content type of what actually arrived.

The image is a local file and the agent has a shell. Coding agents such as Claude Code, Cursor’s agent and Gemini CLI can run commands. Give them a path that doesn’t go through tool arguments. One option is a tool that returns a short-lived upload URL, which the agent fills with curl before passing the returned id to the next tool. The other is a plain REST endpoint. JustBlogged's REST API has POST /upload-image, which takes multipart/form-data, so an agent with an API key can send the file with one curl -F command. Either way the model writes a short command, not the file.

The image was pasted into a chat client with no shell. Neither route works from the model’s side. The tools section of the current spec (2025–11–25) defines image content for tool results, not a file type for tool arguments, so the clean fix lives in the client or in the app’s own upload UI. Be honest in the tool description about this case instead of letting the model try.

The image is tiny. Base64 is fine for logos and icons. The 100x100 logo cost 1,355 tokens.

Where the image is                 | What the tool should accept-----------------------------------+------------------------------------------------Public URL                         | URL, fetched by the server with SSRF guardsLocal file, agent can run commands | Upload URL or REST endpoint, then an id or pathPasted into chat, no shell         | Nothing via the model; use the app's upload UISmall logo or icon                 | Base64 is fine

If you keep a base64 field, set maxLength from a token budget you're willing to let a model spend, not from your storage limit. At about 1.5 characters per token, a 20,000-token budget is 30,000 base64 characters, which is roughly 22 KB of image. That covers logos and small icons and rejects the screenshot early, with a clear validation error, instead of letting a model start a generation it can't finish.

Then say it in the description, since the description is what the model reads when it picks a tool. Something like: “For images larger than 20 KB, pass a public URL or use the REST upload endpoint instead.”

upload_image on JustBlogged's MCP server still takes base64 only today, with the 8 MB limit shown at the top. The REST endpoint takes multipart, and the post tools take a URL for featured images.

encs = {name: tiktoken.get_encoding(name) for name in ("o200k_base", "cl100k_base")}for path in sys.argv[1:]:    raw = open(path, "rb").read()    b64 = base64.b64encode(raw).decode()    counts = {n: len(e.encode(b64)) for n, e in encs.items()}    print(path, len(raw), "bytes", len(b64), "base64 chars",          *(f"{n}={c} ({len(b64)/c:.2f} chars/token)" for n, c in counts.items()))

Your Mcp Tool Takes BASE64 Images. The Model Has to Type Every Character was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

── more in #ai-agents 4 stories · sorted by recency
── more on @justblogged 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/your-mcp-tool-takes-…] indexed:0 read:6min 2026-11-19 · —