Every developer who has worked with LLMs has been there. You ask the model for JSON. You describe the schema. You say "please only respond with valid JSON." And sometimes, it still breaks.
Your application crashes because the model returned a string where you expected an integer. Or it wrapped the JSON in markdown code blocks. Or it omitted a required field.
Spring AI 2.0 has a solution that treats this like a real engineering problem instead of a prayer.
When you use structured output in Spring AI, the workflow goes like this:
This works well with frontier models like Claude and GPT-4. But smaller open-source models, like Llama 3.2 1B running locally via Ollama, fail more often. They might return null for a primitive field, omit required fields, or produce malformed JSON.
When it fails, you get a deserialization exception. Your endpoint returns a 500 error. Spring AI provides no built-in recovery mechanism.
Consider a conference talk submission system. Speakers submit messy, unstructured abstracts. You want to extract structured data:
public record TalkSubmission(
String title,
String abstractText,
Level level, // BEGINNER, INTERMEDIATE, ADVANCED
Track track,
int duration,
List<String> tags,
String speakerHandle
) {}
Here is what the basic typed response looks like:
@PostMapping("/typed")
public TalkSubmission typed(@RequestBody String rawSubmission) {
return chatClient.prompt()
.system(systemPrompt)
.user(spec -> spec.text("Extract the talk submission: {submission}")
.param("submission", rawSubmission))
.call()
.entity(TalkSubmission.class);
}
You define your type. Spring AI generates the schema and appends it to the prompt. The model gets the instruction. And you hope it works.
Dan Vega, Spring Developer Advocate at Broadcom, puts it bluntly in his video demonstration: "That's not engineering. That's hoping."
Spring AI 2.0 introduces self-correcting schema validation. When the model returns invalid JSON, Spring AI validates the response against the schema, detects the error, feeds the error message back to the model, and asks it to fix the response.
The change is a single method call:
@PostMapping("/validated")
public TalkSubmission validated(@RequestBody String rawSubmission) {
return chatClient.prompt()
.system(systemPrompt)
.user(spec -> spec.text("Extract the talk submission: {submission}")
.param("submission", rawSubmission))
.call()
.entity(TalkSubmission.class, spec -> spec.validateSchema());
}
That is it. The spec -> spec.validateSchema()
consumer turns on the self-correcting retry loop.
According to the official documentation by Christian Tzolov (Spring AI team), the validation loop works as follows:
int
, got null
for field duration
") is appended to the user prompt and the call is re-issuedThis is powered by StructuredOutputValidationAdvisor
, a recursive advisor that is auto-registered when you call validateSchema()
. Default is 3 retry attempts. The model knows exactly what went wrong and can correct it on the next attempt.
To customize the retry count, build your own advisor instance:
var validationAdvisor = StructuredOutputValidationAdvisor.builder()
.outputType(TalkSubmission.class)
.maxRepeatAttempts(5)
.build();
ChatClient chatClient = ChatClient.builder(chatModel)
.defaultAdvisors(validationAdvisor)
.build();
Some frontier models support structured output at the API level. Instead of appending the schema to the prompt text, the schema is sent as an API constraint. The provider's runtime enforces conformance, meaning invalid responses cannot be emitted at all.
Spring AI 2.0 exposes this through useProviderStructuredOutput()
:
TalkSubmission result = chatClient.prompt()
.system(systemPrompt)
.user(spec -> spec.text("Extract the talk submission: {submission}")
.param("submission", rawSubmission))
.call()
.entity(TalkSubmission.class, spec -> spec
.useProviderStructuredOutput()
.validateSchema());
Supported providers as of Spring AI 2.0:
Native structured output is off by default because support varies across models. If a model does not support it, the flag is silently ignored and the prompt-based approach is used instead.
Note: .entity()
is only available on .call()
, not on .stream()
. Typed parsing requires the complete response, so streaming responses cannot be deserialized into a typed object.
OpenAI does not accept top-level JSON arrays. If you need a List<T>
, wrap it in a container record first:
// Does NOT work with OpenAI native structured output:
List<TalkSubmission> list = chatClient.prompt()
.call()
.entity(new ParameterizedTypeReference<List<TalkSubmission>>() {},
spec -> spec.useProviderStructuredOutput()); // fails
// Works: wrap in a container
record SubmissionList(List<TalkSubmission> submissions) {}
SubmissionList result = chatClient.prompt()
.call()
.entity(SubmissionList.class, spec -> spec.useProviderStructuredOutput());
Ollama with reasoning models (like Qwen variants) may emit internal reasoning traces as plain text instead of JSON. Use a non-reasoning model, or combine with validateSchema()
so malformed responses are automatically retried.
Not every scenario needs all features enabled. Based on the official docs and video demonstration:
Frontier models (Claude, GPT-4, Gemini):
useProviderStructuredOutput()
for API-level enforcementvalidateSchema()
as a safety net for edge casesOpen-source models (Llama, Mistral via Ollama):
useProviderStructuredOutput()
may have no effect (model-specific)validateSchema()
is essentialProduction systems:
maxRepeatAttempts
based on your latency budgetThis feature represents a shift in how we think about LLM integration. For too long, the industry treated unreliable model output as a prompt engineering problem. Write a better prompt. Be more specific. Add examples. Pray harder.
Spring AI 2.0 treats it as a systems problem. Validate. Retry. Self-correct. The same principles we apply to any unreliable external service: network calls, database queries, third-party APIs. LLMs are no different.
If you are building production applications with LLMs, schema validation is not optional. It is basic engineering.
Sources: