{"slug": "json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool", "title": "JSON Mode Silently Stopped All 7 Open Models From Calling a Single Tool", "summary": "A June 2026 paper from Focus Technology's AI lab found that JSON mode caused all seven open-weight models tested to stop calling tools entirely, with tool invocation dropping from 100% to 0% when a JSON schema was attached. The issue stems from a token bitmask that sets logits to -inf for non-schema tokens, and on a five-field extraction schema, the grammar writes 72.4% of response bytes with no free choice until byte 17.", "body_md": "Member-only story\n\n# JSON Mode Silently Stopped All 7 Open Models From Calling a Single Tool\n\nSeven open-weight models. Seven suppression rates of exactly 100%. Tool invocation went from 100% to 0% the moment a JSON schema was attached to the same request, and not one line in any log said so.\n\nThat is the finding in a June 2026 paper out of Focus Technology’s AI lab, and the first time I read it I assumed it was a prompt bug. It is not. It is arithmetic, and once you see the arithmetic you cannot unsee it: `response_format={\"type\":\"json_schema\"}`\n\ncompiles your schema into a token bitmask, and the engine executes exactly one line that ends the argument.\n\n```\nlogits[mask == 0] = -inf\n```\n\nYour model wanted to call a tool. The mask deleted the option before sampling ever ran.\n\nI spent this morning rebuilding the mask math from scratch to check whether the claim survives contact with a calculator. It does, and the numbers are worse than the paper’s headline. On a routine five-field extraction schema, **the grammar writes 72.4% of the response bytes** and the model does not get a single free choice until byte 17. We stopped calling this “generation” a while ago and nobody updated the mental model.", "url": "https://wpnews.pro/news/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool", "canonical_source": "https://pub.towardsai.net/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool-a4f1c8d4731b?source=rss----98111c9905da---4", "published_at": "2026-08-01 16:28:51+00:00", "updated_at": "2026-08-01 16:52:39.507342+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools"], "entities": ["Focus Technology"], "alternates": {"html": "https://wpnews.pro/news/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool", "markdown": "https://wpnews.pro/news/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool.md", "text": "https://wpnews.pro/news/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool.txt", "jsonld": "https://wpnews.pro/news/json-mode-silently-stopped-all-7-open-models-from-calling-a-single-tool.jsonld"}}