Structured Output That Holds: LLM JSON in Production

Anthropic, OpenAI, and Google now enforce JSON Schema at the sampling level for LLM structured output, but each vendor supports a different subset, breaking portability for shared schemas. Anthropic rejects numeric bounds and caps minItems at 0 or 1, OpenAI forces all fields into required and rejects root-level anyOf, and Gemini supports numeric bounds but discloses no nesting limit. The Vercel AI SDK 6 deprecates generateObject and streamObject in favor of generateText and streamText with an output parameter.

LLM structured output finally grew up in 2026: Anthropic, OpenAI, and Google all now enforce JSON Schema at the sampling level, using grammar-constrained decoding rather than polite prompting. A schema that the vendor accepts will produce output that parses. The catch is the word “accepts” — each provider honors a different subset of JSON Schema, and the differences hide exactly where production pipelines break. The stakes are practical. Extraction pipelines, agent tool calls, CRM enrichment jobs, and report generators all depend on the model returning an object your code can trust. Teams that standardized on one Zod schema and assumed it would travel across providers are discovering that Anthropic rejects numeric bounds, OpenAI rejects root-level unions, and Gemini declines to publish its nesting ceiling at all. This playbook covers each vendor's current structured-output surface, a side-by-side support matrix of what each schema subset actually allows, how streaming partial JSON works, and how the Vercel AI SDK 6 layer — our own house stack — wraps all of it, including the deprecation path away from generateObject . - 01Grammar-constrained JSON is now table stakes.All three major providers constrain token sampling to schema-valid output — Anthropic via strict tool use, OpenAI via Structured Outputs strict mode, Gemini via response format. Prompt-only JSON is a legacy pattern. - 02The schema subsets differ enough to break portability.Anthropic supports no numeric or string-length bounds and caps minItems at 0 or 1. OpenAI forces every field into required and rejects root-level anyOf. Gemini supports numeric bounds but discloses no nesting limit. One shared Zod schema is not guaranteed to validate on all three. - 03Anthropic's guaranteed pattern is tool choice plus strict.Combining tool choice type any with strict: true on the tool definition is Anthropic's documented recipe for a response that always calls a tool and always matches the schema — with caveats around manual extended thinking and Claude Mythos Preview. - 04Gemini's canonical surface moved.Google's current docs teach response format with mime type application/json on the Interactions API — not the older generateContent responseSchema shape most tutorials still show. Treat responseSchema as the legacy pattern. - 05AI SDK 6 deprecates generateObject and streamObject.The dedicated object functions still work, but the forward path is generateText and streamText with an output parameter Output.object, Output.array, and friends . A codemod automates the migration. 01 — The ShiftGrammar constraints made “please return JSON” obsolete . For most of the past three years, reliable JSON from an LLM was a defensive engineering exercise: prompt hard, parse leniently, strip markdown fences, retry on failure. That era is effectively over. Each of the three major providers now offers a mode in which the model is not merely asked to follow your schema — its token sampling is constrained so that schema-invalid output cannot be emitted. Anthropic names the technique directly: grammar-constrained sampling. The three surfaces look different but do the same job. What actually differs — and what this guide is really about — is the slice of JSON Schema each vendor's constraint engine understands. Strict tool use Set strict: true at the top level of a tool definition and the tool use input block is guaranteed to match the JSON Schema. Pair with tool choice any for an always-called, always-valid structured response. A dedicated non-tool structured-outputs path shares the same constraint pipeline. Structured Outputs strict mode On the current Responses API, the schema rides inside the text.format block with strict: true. Every object needs additionalProperties set to false and every field listed as required — optional fields become nullable types instead. Gemini response format The current canonical surface on the Interactions API. The older generateContent + generationConfig.responseSchema shape that most tutorials still teach is the legacy pattern, not what today's docs page shows. The strategic consequence is subtle but important: once validity is guaranteed at the decoding layer, the engineering effort moves up a level. Your JSON will parse. Whether it parses into something correct — the right fields populated, cross-field invariants holding, business rules respected — is still your job, and each vendor's schema-subset gaps determine how much of that job the constraint engine can carry for you. 02 — AnthropicStrict tool use: guaranteed inputs, narrow subset. Anthropic's forced-tool-call control is tool choice , with four modes: auto the default when tools are present , any a tool must be used, no particular one , tool forces one specific tool , and none . Schema enforcement is a separate switch — set "strict": true at the top level of the tool definition, alongside name , description , and input schema . The documented pattern for a fully guaranteed structured response combines the two: tool choice of type any plus a strict tool, so a tool is always called and its input always validates. The feature is available on a wide model range — including Claude Opus 5, Sonnet 5, and Haiku 4.5 — and Anthropic also documents a dedicated non-tool structured-outputs path that reuses the same constraint pipeline, with compiled schemas cached for up to 24 hours after last use. Two interaction caveats matter in production. With manual extended thinking enabled, tool choice modes any and tool error out — only auto and none work; adaptive thinking the default on Opus 5 does support forced tool use. And Claude Mythos Preview does not support forced tool use at all — any or tool return a 400 error on that model. “Strict tool use guarantees Claude’s tool inputs match your JSON Schema by constraining the model’s token sampling to schema-valid outputs a technique called grammar-constrained sampling .”— Anthropic strict tool use documentation The subset is where teams get surprised. Objects must set additionalProperties: false . Array minItems accepts only the values 0 or 1. Numeric constraints minimum , maximum , multipleOf and string-length constraints minLength , maxLength are not supported at all. Recursive schemas and external HTTP $ref are out. What you do get: enum and const , anyOf and allOf though the allOf -plus- $ref combination is unsupported , internal $ref and definitions, default , pattern , and ten string format values — date-time , time , date , duration , email , hostname , uri , ipv4 , ipv6 , and uuid . If your Zod schema says "an array of 3 to 5 tags, each 2 to 40 characters, score between 0 and 100," none of those bounds reach Anthropic's constraint engine. The output will be a valid array of strings and a number — the ranges are your validation layer's problem. never appear in input schema property names, enum or const values, or pattern regexes. Compiled schemas are cached separately from message content and do not receive the same PHI protections as prompts and responses. Treat the schema itself as a non-sensitive artifact by design. 03 — OpenAIStrict mode: hard ceilings and no optional fields . OpenAI's Structured Outputs with strict: true currently rides on the Responses API as text: { format: { type: "json schema", strict: true, schema } } . Its two signature requirements shape how you design schemas: every object must set additionalProperties to false, and every field must appear in required . There are no truly optional fields in strict mode — anything conceptually optional has to be typed as nullable instead. For a full walkthrough of the OpenAI side, see our OpenAI structured-outputs complete guide /blog/openai-structured-outputs-complete-guide . The unsupported-keyword list is the opposite shape from Anthropic's. OpenAI accepts the constraint keywords Anthropic rejects — numeric minimum / maximum / multipleOf and unrestricted array minItems / maxItems — alongside pattern , string format , and enum , which both vendors support. What it rejects is composition: root-level anyOf no discriminated union at the top of your schema , allOf , not , dependentRequired , and if / then / else . Strict mode also has published scale ceilings — the numbers below are hard limits, not guidance. Max properties per schema Total object properties across the entire schema. Generous for extraction jobs, but auto-generated schemas from large TypeScript types can approach it faster than you expect. Max schema nesting Ten levels of nesting is the ceiling. Deeply recursive document structures need flattening or reference redesign before they fit strict mode. Character cap on schema content Total schema string content caps at 120,000 characters, and enum values cap at 1,000 across all properties combined. Large controlled vocabularies belong in your validation layer, not the schema. 04 — Google GeminiThe canonical surface moved — and most tutorials haven't. Here is the finding most existing content gets wrong: Gemini's current structured-output parameter is response format , set on the Interactions API as response format: { type: "text", mime type: "application/json", schema } . The classic generateContent + generationConfig.responseSchema shape that fills most tutorials — and older codebases — is not what today's canonical docs page teaches. The older field may still work for back-compatibility, but new integrations should target response format . Gemini's documented schema subset covers the core types including nullable via "type": "string", "null" , object-level properties / required / additionalProperties , array items / prefixItems / minItems / maxItems , numeric minimum / maximum , and string enum / format . Google's own SDKs also accept Pydantic models in Python or Zod schemas in JavaScript and convert them to the JSON Schema the API expects. Two notable extras: structured outputs stream as valid partial JSON more in section 06 , and combining schema-constrained output with built-in tools — Google Search grounding, URL Context, Code Execution, File Search, Function Calling — is scoped to Gemini 3-series models, with the docs example using gemini-3.1-pro-preview . The limitation language is the honest tell. Google states plainly that “Not all JSON Schema features are supported” and that “Very large or deeply nested schemas may be rejected” — with no numeric nesting or size ceiling disclosed. Where OpenAI gives you hard numbers to design against, Gemini gives you a warning and lets you discover the edge at request time. responseSchema and may keep working on borrowed time. Audit which Gemini surface your integration actually calls, and pin new work to the documented response format shape. 05 — The MatrixOne schema, three engines: the support matrix . The table below assembles each vendor's documented schema subset side by side — the comparison most “they all support JSON Schema now” posts skip. Every cell is sourced from the vendors’ own structured-output documentation as of this post's August 14, 2026 snapshot. | Schema capability | Anthropic · strict tool use | OpenAI · Structured Outputs | Gemini · response format | |---|---|---|---| | Forcing mechanism and baseline requirements | ||| | How you force schema-valid output | strict: true on the tool + tool choice type any | text.format with type json schema and strict: true Responses API | response format with mime type: "application/json" + schema Interactions API | | additionalProperties on objects | false is mandatory | false must be set on every object | Supported at object level; no mandatory-false rule documented | | Optional fields | No all-fields-required rule documented | Every field must be in required ; optionals become nullable types | Nullable via "type": "string", "null" | | Constraint keywords | ||| | Numeric minimum / maximum | Not supported nor multipleOf | Supported, including multipleOf | Supported | | String minLength / maxLength | Not supported | Not listed in the documented constraint set | Not listed in the documented subset | | Array minItems | Only the values 0 or 1 accepted | minItems and maxItems supported | minItems and maxItems supported | | enum | Supported, plus const | Supported — max 1,000 values across all properties | Supported | | Composition and scale | ||| | Unions and composition | anyOf / allOf supported; allOf + $ref combination is not | Root-level anyOf rejected; allOf , not , if / then / else , dependentRequired rejected | Not listed in the documented subset | | Recursion and $ref | Internal $ref / definitions only; recursive schemas and external HTTP $ref unsupported | Not covered by the published unsupported-keyword list | “Not all JSON Schema features are supported” — specifics undisclosed | | Published scale ceilings | None published beyond the minItems quirk | 5,000 properties · 10 nesting levels · 120,000 schema chars · 1,000 enum values | No numeric ceiling — “Very large or deeply nested schemas may be rejected” | Read as a whole, the matrix shows three different philosophies. Anthropic constrains structure tightly but delegates value-range enforcement entirely to your code. OpenAI enforces the most constraint keywords but bans the composition patterns — discriminated unions above all — that typed codebases lean on. Gemini sits in the middle with the least explicit contract: a reasonable subset, softly-documented edges, and no hard numbers. The practical takeaway is that the intersection of all three subsets is the only safely portable schema language : plain objects, required fields, nullable-instead-of-optional, enums, and no bounds you actually depend on. 06 — StreamingStreaming structured output without broken JSON. Streaming and strict schemas coexist better than they used to, but the guarantees differ by vendor and even by SDK. Gemini's docs confirm streaming support directly: “The streamed chunks are valid partial JSON strings that can be concatenated to form the final JSON object” — consumed via stream: true and step.delta events. That means a UI can render a progressively-filling object without waiting for the final token. On the Anthropic side, the one concrete piece of SDK guidance we can cite is scoped to the Java SDK : structured responses there must be fully accumulated via a message accumulator before JSON deserialization — incremental parsing is the caller's responsibility. That guidance is explicitly per-language-binding; don't assume it describes the TypeScript or Python SDKs. In practice, most application teams shouldn't hand-roll partial JSON handling at all — the Vercel AI SDK's streamObject and its successor pattern hands you a typed partial-object stream, plus an elementStream for arrays, and handles chunk concatenation and partial parsing for you across providers. That is the right altitude for product code; raw delta-event handling is for infrastructure layers. 07 — House StackAI SDK 6: generateObject is now the legacy path. The Vercel AI SDK is the abstraction most Next.js teams — ours included; this site runs ai@^6.0.193 — use to avoid coding against three vendor surfaces directly. You hand it a Zod schema, and the SDK translates to whichever provider mechanism is underneath. Field-level .describe calls double as inline prompting hints to the model: js import { generateObject } from "ai"; import { z } from "zod"; const invoiceSchema = z.object { vendor: z.string .describe "Supplier name as printed on the invoice" , totalEur: z.number .describe "Grand total in EUR" , lineItems: z.array z.object { description: z.string , amountEur: z.number , } , } ; const { object } = await generateObject { model, // any AI SDK provider model instance schema: invoiceSchema, prompt: "Extract the invoice fields from the document text below. ...", } ; // object is fully typed as z.infer<typeof invoiceSchema Streaming is the same shape with progressive delivery — the partial-object stream is where the SDK earns its keep: js import { streamObject } from "ai"; const { partialObjectStream } = streamObject { model, schema: invoiceSchema, prompt: "Extract the invoice fields...", } ; for await const partial of partialObjectStream { // typed, progressively-filled partial object — safe to render } Now the caveat: both functions are deprecated in AI SDK 6 — softly. They still exist and still work, but the forward path is generateText / streamText with an output parameter: Output.object { schema } , Output.array { element } , Output.choice { options } , Output.json , or Output.text . An automated codemod npx @ai-sdk/codemod upgrade migrates existing call sites. The correct framing is “legacy, migrate when convenient” — not broken, not urgent. Our AI SDK 5-to-6 migration playbook /blog/vercel-ai-sdk-v5-to-v6-migration-playbook-2026 covers the wider upgrade, and the AI SDK 6 deep dive /blog/vercel-ai-sdk-6-deep-dive-features-tool-calls-2026 goes further into the new tool-calling and streaming surfaces. Error handling is where structured-output code most often stays naive. The object functions throw NoObjectGeneratedError when the response can't be parsed or fails schema validation, and NoOutputGeneratedError when the final step doesn't finish with a stop reason — both preserve the raw text , response , usage , and cause for debugging. Catch them specifically, log the preserved fields, and make the retry decision explicit rather than wrapping everything in a generic try/catch. Teams migrating off LangChain's output parsers /blog/langchain-to-vercel-ai-sdk-migration-playbook-cost-quality-2026 will find this error surface considerably more debuggable than parser-chain failures. 08 — PlaybookThe production hardening checklist. Grammar constraints remove one failure class; they do not remove the need for a validation boundary. Here is how we structure the decisions on client builds. Formatting the model's response Use the structured-output surface strict tools, strict mode, response format . Google's own framing draws the line cleanly: structured outputs are for formatting the final response. The model doing things When the model needs the caller to act during the conversation — look something up, run code, write a record — that is function calling, not output formatting. Keep the two mechanisms distinct in your architecture. One schema, several vendors Design to the intersection subset: flat-ish objects, all fields required, nullable instead of optional, enums for vocabularies, and no numeric or length bounds you rely on. Enforce the bounds in Zod at the boundary instead. Schema conformance over time Vendor subsets are API surface — they change. Pin every production schema in contract tests that run each schema against each provider you route to, so a subset change fails CI instead of production. The layer that ties this together is validate-then-retry. Even with grammar constraints on, parse every response through the same Zod schema you sent — because the constraint engines cannot enforce everything your schema means. Anthropic can't enforce your numeric ranges. OpenAI can't enforce your top-level union. None of them can enforce cross-field invariants like “end date after start date.” Zod catches those post-hoc, and a failed parse becomes a bounded retry with the validation errors appended to the prompt. Schema conformance also belongs in your evaluation suite — our guide to agent eval frameworks /blog/ai-agent-eval-frameworks-testing-guide-2026 covers testing agent outputs systematically. Looking forward, we expect convergence pressure on these subsets — the divergence is a migration tax on every multi-provider team, and vendors know it. But convergence is not here yet: none of the three documents a shared subset today. Until that changes, the durable posture is to treat each provider's schema subset as a versioned API surface: documented in your codebase, tested in CI, and owned by someone. This is exactly the kind of reliability engineering we build into client systems through our AI transformation engagements /services/ai-transformation . 09 — ConclusionValidity is solved. Portability isn't . The JSON now parses. Whether it's portable is the new engineering problem. The 2026 structured-output story is genuinely good news: all three major providers now constrain sampling to schema-valid output, and the parse-retry-pray loop that defined early LLM engineering is a legacy pattern. If your pipeline still strips markdown fences off model responses, you are leaving reliability on the table. But the vendors solved validity, not portability. Anthropic's missing bounds, OpenAI's banned unions and hard ceilings, and Gemini's undisclosed limits mean the same Zod schema can be legal on one provider and rejected — or silently weakened — on another. The teams that ship reliably design to the intersection subset, keep Zod validation at the boundary regardless, and pin provider behavior in contract tests. And keep an eye on your abstraction layer: the AI SDK's move from generateObject to output-parameterized generateText is a reminder that this surface is still settling. The codemod makes the migration cheap; the discipline of validating and testing schema behavior is what makes structured output actually hold in production.