JSON Mode Silently Stopped All 7 Open Models From Calling a Single Tool

A June 2026 paper from Focus Technology's AI lab found that JSON mode caused all seven open-weight models tested to stop calling tools entirely, with tool invocation dropping from 100% to 0% when a JSON schema was attached. The issue stems from a token bitmask that sets logits to -inf for non-schema tokens, and on a five-field extraction schema, the grammar writes 72.4% of response bytes with no free choice until byte 17.

Member-only story JSON Mode Silently Stopped All 7 Open Models From Calling a Single Tool Seven open-weight models. Seven suppression rates of exactly 100%. Tool invocation went from 100% to 0% the moment a JSON schema was attached to the same request, and not one line in any log said so. That is the finding in a June 2026 paper out of Focus Technology’s AI lab, and the first time I read it I assumed it was a prompt bug. It is not. It is arithmetic, and once you see the arithmetic you cannot unsee it: response format={"type":"json schema"} compiles your schema into a token bitmask, and the engine executes exactly one line that ends the argument. logits mask == 0 = -inf Your model wanted to call a tool. The mask deleted the option before sampling ever ran. I spent this morning rebuilding the mask math from scratch to check whether the claim survives contact with a calculator. It does, and the numbers are worse than the paper’s headline. On a routine five-field extraction schema, the grammar writes 72.4% of the response bytes and the model does not get a single free choice until byte 17. We stopped calling this “generation” a while ago and nobody updated the mental model.