{"slug": "5-undocumented-rules-for-gemini-structured-output-measured-in-production", "title": "5 Undocumented Rules for Gemini Structured Output, Measured in Production", "summary": "A developer reports undocumented behaviors in Gemini's structured output schema, discovered through production measurements on a document extraction pipeline. The required array controls emission order, which affects correctness, and reordering it fixed a role classification error in 9 of 9 cases, up from 5 of 9. The findings include that enum descriptions are not enforced and that a PROVIDER_EXHAUSTED error after a prompt change indicates a broken schema.", "body_md": "We run a document extraction pipeline on Gemini with a native `responseSchema`\n\nattached, not a \"please reply with JSON\" instruction in the prompt text. Over two months, three separate production problems traced back to behaviours of that schema that are not in Google's documentation.\n\nThese are the rules we ship with now, and the measurement behind each one. The domain is anonymized (no client, no industry, role codes renamed). Every number, date, model name and error string is real.\n\n`required`\n\narray identity, then evidence, then derived. It controls emission order, and emission order controls correctness.`enum`\n\narrays are enforced. Values listed in a `description`\n\nare not constrained at all.`PROVIDER_EXHAUSTED`\n\nright after a prompt change means your schema is broken, not that Google is busy.`required`\n\nidentity, evidence, derived\nGemini emits every `required`\n\nproperty first, in exactly the order the `required`\n\narray lists them, then the optional ones. The declaration order inside `properties`\n\nis ignored for the required set.\n\nThis is empirical. It is not in Google's docs. We measured it on `gemini-3-flash-preview`\n\ntwice, with opposite orders, reading the raw response text:\n\n| schema | position of `role_code` in `required`\n|\nposition in emitted JSON |\n|---|---|---|\n| v18 | 3rd | 3rd |\n| v19 | 14th (last) | 14th |\n\nIn v18 that field was 12th in `properties`\n\nand 3rd in `required`\n\n. It came out 3rd. `properties`\n\nis not the lever.\n\n`propertyOrdering`\n\nis the documented knob, but if you do not set it (we do not, anywhere), the `required`\n\norder is what governs.\n\nA model writing JSON does one forward pass. Whatever it has already emitted is in context. Whatever it has not is not. And an emitted token cannot be revised when a later field contradicts it.\n\nSo a field emitted early is decided with almost no self-generated evidence, and a field emitted late is decided with everything above it visible.\n\nv18 put `role_code`\n\nthird in `required`\n\n, after only `first_name`\n\nand `last_name`\n\n. Its instruction was a priority ladder:\n\n`work_history`\n\nentry (Nine CVs, all advertising the same role. Four came back with `L3-OPS`\n\n, a role from a different department. All four were internally self-contradictory:\n\n```\n{\n  \"role_code\": \"L3-OPS\",\n  \"department\": \"Technical\",\n  \"work_history\": [{ \"title\": \"L3-TECH\", \"...\": \"...\" }]\n}\n```\n\nBoth codes are valid members of the 143-value enum, so nothing rejected the output. `department`\n\n, which agreed with the correct reading in all four cases, was emitted 11th, long after the wrong token was committed. The consistency check that would have caught the error was generated downstream of the error.\n\nRuled out first: environment drift (schemas byte-identical), downstream mapping (the wrong code was already in the raw provider response), a missing enum value (the correct code was present, and used correctly elsewhere in the same responses), and ambiguous source documents (zero matches for any operations wording, 11 to 20 matches for technical wording per document).\n\nReorder `required`\n\n. Nothing else. No type change, no enum change, no shape change.\n\n```\nv18:  first_name, last_name, role_code, contacts, nationalities, date_of_birth,\n      work_history, certifications, documents, education, languages, address,\n      home_airport, department\n\nv19:  first_name, last_name, date_of_birth, nationalities, contacts,\n      work_history, certifications, education, documents, languages, address,\n      home_airport, department, role_code\n```\n\nResult: 9 of 9 correct, up from 5 of 9. Emitted position of `role_code`\n\nmoved 3 to 14, exactly as predicted.\n\n| class | meaning | position |\n|---|---|---|\n| identity | copied off the document, no reasoning (`first_name` , `date_of_birth` ) |\nfirst |\n| evidence | the substantive extracted content (`work_history` , `certifications` ) |\nmiddle |\n| derived | a judgement about the evidence (`role_code` , `department` , any score, total or summary) |\nlast |\n\nThree things that come with the reorder:\n\n**Descriptions must not forward-reference.** Once `department`\n\nmoved ahead of `role_code`\n\n, its old text (\"classify from the stated `role_code`\n\n\") became the same bug in miniature. After any reorder, re-read every description for references to fields that now come later.\n\n**Tell the model the evidence is already there.** Reordering alone is silent. v19's priority 2 became: *\"You have ALREADY emitted the work_history array above. Read the title of its first entry and use it.\"*\n\n**Check for over-anchoring.** The goal is grounding, not echoing. Two candidates whose most recent entry was one level below the applied-for role still correctly emitted the applied-for level, because priority 1 legitimately outranks priority 2. If every derived value suddenly equals `evidence[0]`\n\n, you have over-corrected.\n\nAnd the trap: `required`\n\nis a **set** to a JSON Schema validator. Reordering it is semantically inert, so a formatter that sorts the array, or a tool that round trips the JSON, silently reverts the behaviour with a diff that looks like whitespace and passes every test. Say so in the file.\n\nDo not respond to a wrong derived field by adding more prose first. v18 already carried four bullets of correct guidance for that field and was still wrong about 44% of the time. The instruction was not being disobeyed. It was being evaluated at a token position where its input did not exist.\n\nGemini rejects a schema above an undocumented ceiling on the **total** enum-value count across the whole schema. Google publishes no number, only that \"very large or deeply nested schemas may be rejected\".\n\n| total enum values | result | when |\n|---|---|---|\n| 467 | accepted | v9, production |\n| 610 | accepted, months of clean runs | v10-revised through v14 |\n740 |\nrejected |\nv15, 2026-08-17 |\n| 754 | rejected, reverted | v10-initial, 2026-07-13 |\n\nThe boundary is in (610, 740]. We never bisected it.\n\nThree checkpoints were tried with the 740 schema, one preview and two GA releases:\n\n| model | outcome |\n|---|---|\n`gemini-3-flash-preview` |\n400 invalid argument |\n`gemini-3.5-flash` |\n400 invalid argument |\n`gemini-3.6-flash` |\n400 invalid argument |\n\nIdentical rejection across releases spanning months. This is a property of the constrained-decoding compiler, not of a checkpoint, so waiting for a newer model is not a mitigation.\n\n**You cannot deduplicate your way under the limit.** Gemini's subset has no `$ref`\n\nand no `$defs`\n\n(see Rule 4), so every repeated list is paid for in full. A 145-value list used in three places costs 435, not 145.\n\nPractical consequences:\n\n`description`\n\ninstead. Descriptions cost nothing against the budget. Just know they are not enforced either (Rule 3).A rough counter is worth having in CI:\n\n```\n// Sums every enum array in a schema, nulls included.\nfunction countEnums(node) {\n  if (Array.isArray(node)) return node.reduce((n, v) => n + countEnums(v), 0);\n  if (node && typeof node === 'object') {\n    return Object.entries(node).reduce(\n      (n, [k, v]) => n + (k === 'enum' && Array.isArray(v) ? v.length : countEnums(v)),\n      0\n    );\n  }\n  return 0;\n}\n```\n\n`responseSchema`\n\nguarantees JSON shape and types. It does not guarantee values, with one exception.\n\n| how you express it | enforced? |\n|---|---|\n`\"enum\": [\"L3-TECH\", \"L3-OPS\"]` |\nyes, by constrained decoding |\nallowed values listed in `description`\n|\nno, purely advisory |\n`maxLength` |\nno |\n| array uniqueness | no |\n\nIn July, a schema change meant constrained decoding stopped being applied for five days. Nothing failed, nothing turned red, and 75 values that do not exist in the vocabulary reached production in a field the rest of the system indexes on.\n\nSo:\n\nIf the same prompt may run on more than one provider, store the schema in the **more restrictive** format. Gemini's subset is the floor.\n\n| feature | OpenAI | Gemini |\n|---|---|---|\n`$ref` / `$defs`\n|\nsupported |\nnot supported, inline everything |\n`$schema` , `$id`\n|\nsupported |\nnot supported, strip |\n`oneOf` |\nsupported |\nnot supported, single type + `nullable`\n|\n`[\"string\", \"null\"]` |\nsupported |\nnot supported, use `nullable`\n|\n`exclusiveMinimum` |\nsupported |\nnot supported, use `minimum`\n|\n`pattern` |\nsupported | stripped |\n`format: \"uri\"` |\nsupported | stripped |\n`nullable` |\nnot used |\nrequired for nullable fields |\n| max nesting | no limit | 5 levels |\n| property ordering | not enforced |\n`required` order drives emission |\n\nRejected:\n\n```\n{\n  \"$schema\": \"https://json-schema.org/draft/2020-12/schema\",\n  \"type\": \"object\",\n  \"properties\": {\n    \"role_code\":  { \"$ref\": \"#/$defs/RoleCode\" },\n    \"start_year\": { \"type\": \"integer\", \"exclusiveMinimum\": 1900 },\n    \"email\":      { \"type\": [\"string\", \"null\"] },\n    \"website\":    { \"type\": \"string\", \"format\": \"uri\" },\n    \"ref\":        { \"type\": \"string\", \"pattern\": \"^[A-Z]{2}-\\\\d{4}$\" }\n  }\n}\n```\n\nAccepted:\n\n```\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"start_year\": { \"type\": \"integer\", \"minimum\": 1901 },\n    \"email\":      { \"type\": \"string\", \"nullable\": true },\n    \"website\":    { \"type\": \"string\" },\n    \"ref\":        { \"type\": \"string\", \"description\": \"Two uppercase letters, hyphen, four digits\" },\n    \"role_code\":  { \"type\": \"string\", \"enum\": [\"L3-TECH\", \"L3-OPS\"] }\n  },\n  \"required\": [\"start_year\", \"email\", \"website\", \"ref\", \"role_code\"]\n}\n```\n\nNote `role_code`\n\nis last in `required`\n\n, per Rule 1.\n\nA malformed schema does not degrade politely. On 2026-08-27 a single `\"type\": [\"string\", \"null\"]`\n\nunion in one prompt failed every execution of it until the union was removed.\n\nThis is the one that costs the most hours, because the label sends you to the wrong system.\n\nWhat the provider actually returns:\n\n```\n400 . Request contains an invalid argument.\n```\n\nNo field pointer, no property name, no mention of size or enums. Deterministic on every retry, every key and every service tier.\n\nWhat the operator sees by the time it surfaces:\n\n```\nprocessing_error   PROVIDER_EXHAUSTED: Provider capacity unavailable\nerror_reason_code  PROVIDER_EXHAUSTED\nmodel              (empty)\n```\n\nThe chain is 400, then an API error, then the key circuit opens, then the retry ladder exhausts, then the last event gets reported instead of the first. It reads as a transient capacity shed. It is a permanent schema defect.\n\nTriage table:\n\n| symptom | actual meaning |\n|---|---|\n`PROVIDER_EXHAUSTED` with an empty model field, starting right after a prompt change |\nschema defect, not capacity |\n| the same failure on both STANDARD and FLEX tiers | not a tier or quota problem |\n| identical failure across model checkpoints | constrained-decoding compiler, not the model |\n| valid JSON with out-of-vocabulary values | constrained decoding was not applied at all |\n\nThe real error survives only in a WARN line, on whichever replica ran the worker, which is usually not the replica that logged the submission. Grep all of them:\n\n```\nfor P in $(kubectl -n <ns> get pods -o name | grep -E \"^pod/ai-\" | grep -v db); do\n  kubectl -n <ns> logs $P --since=2h \\\n    | grep -E \"invalid argument|Circuit OPEN|keys exhausted\"\ndone\n```\n\nTime-to-failure tells you nothing. We saw 30s and 165s for the same rejection and briefly read the slow one as \"this model accepted the schema\". It had not. The difference was retry parking.\n\nIf you store parsed responses in a `jsonb`\n\ncolumn, that column **loses key order**. Read the raw response text instead:\n\n``` js\nconst raw = require('fs').readFileSync('raw.json', 'utf8').trim();\nObject.keys(JSON.parse(raw)).forEach((k, i) => console.log(`${i + 1}. ${k}`));\n```\n\nCompare that against your `required`\n\narray. If they diverge, the ordering law has changed for your model family and needs re-measuring.\n\n`required`\n\nordered identity, then evidence, then derived.`description`\n\nreferences a field emitted later.`required`\n\norder is deliberate and must not be sorted.`$ref`\n\n, no `oneOf`\n\n, no type arrays, `nullable`\n\nused, 5 levels max).`PROVIDER_EXHAUSTED`\n\nafter a prompt change means schema first, capacity second.Two of these five rules describe limits Google does not document, and both were established by breaking production. If you are running structured output at any scale, measure them for your own model family and write your own numbers down. The alternative is rediscovering them next quarter at the same price.", "url": "https://wpnews.pro/news/5-undocumented-rules-for-gemini-structured-output-measured-in-production", "canonical_source": "https://dev.to/artyomsv/5-undocumented-rules-for-gemini-structured-output-measured-in-production-3mj", "published_at": "2026-08-28 11:36:48+00:00", "updated_at": "2026-08-28 11:50:31.894996+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-products"], "entities": ["Gemini", "Google"], "alternates": {"html": "https://wpnews.pro/news/5-undocumented-rules-for-gemini-structured-output-measured-in-production", "markdown": "https://wpnews.pro/news/5-undocumented-rules-for-gemini-structured-output-measured-in-production.md", "text": "https://wpnews.pro/news/5-undocumented-rules-for-gemini-structured-output-measured-in-production.txt", "jsonld": "https://wpnews.pro/news/5-undocumented-rules-for-gemini-structured-output-measured-in-production.jsonld"}}