{"slug": "stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring", "title": "Stop Begging Your LLM for Valid JSON: Self-Correcting Structured Output in Spring AI 2.0", "summary": "Spring AI 2.0 introduces self-correcting structured output that automatically validates JSON responses from LLMs and retries with error feedback when the output is malformed. The feature, demonstrated by Spring Developer Advocate Dan Vega, uses a single method call to enable a retry loop that catches deserialization errors and prompts the model to fix them. Christian Tzolov of the Spring AI team documented the validation loop, which defaults to three retry attempts and can be customized.", "body_md": "Every developer who has worked with LLMs has been there. You ask the model for JSON. You describe the schema. You say \"please only respond with valid JSON.\" And sometimes, it still breaks.\n\nYour application crashes because the model returned a string where you expected an integer. Or it wrapped the JSON in markdown code blocks. Or it omitted a required field.\n\nSpring AI 2.0 has a solution that treats this like a real engineering problem instead of a prayer.\n\nWhen you use structured output in Spring AI, the workflow goes like this:\n\nThis works well with frontier models like Claude and GPT-4. But smaller open-source models, like Llama 3.2 1B running locally via Ollama, fail more often. They might return null for a primitive field, omit required fields, or produce malformed JSON.\n\nWhen it fails, you get a deserialization exception. Your endpoint returns a 500 error. Spring AI provides no built-in recovery mechanism.\n\nConsider a conference talk submission system. Speakers submit messy, unstructured abstracts. You want to extract structured data:\n\n```\npublic record TalkSubmission(\n    String title,\n    String abstractText,\n    Level level,        // BEGINNER, INTERMEDIATE, ADVANCED\n    Track track,\n    int duration,\n    List<String> tags,\n    String speakerHandle\n) {}\n```\n\nHere is what the basic typed response looks like:\n\n```\n@PostMapping(\"/typed\")\npublic TalkSubmission typed(@RequestBody String rawSubmission) {\n    return chatClient.prompt()\n        .system(systemPrompt)\n        .user(spec -> spec.text(\"Extract the talk submission: {submission}\")\n            .param(\"submission\", rawSubmission))\n        .call()\n        .entity(TalkSubmission.class);\n}\n```\n\nYou define your type. Spring AI generates the schema and appends it to the prompt. The model gets the instruction. And you hope it works.\n\nDan Vega, Spring Developer Advocate at Broadcom, puts it bluntly in his [video demonstration](https://www.youtube.com/watch?v=vxOeeNyOtZY): \"That's not engineering. That's hoping.\"\n\nSpring AI 2.0 introduces self-correcting schema validation. When the model returns invalid JSON, Spring AI validates the response against the schema, detects the error, feeds the error message back to the model, and asks it to fix the response.\n\nThe change is a single method call:\n\n```\n@PostMapping(\"/validated\")\npublic TalkSubmission validated(@RequestBody String rawSubmission) {\n    return chatClient.prompt()\n        .system(systemPrompt)\n        .user(spec -> spec.text(\"Extract the talk submission: {submission}\")\n            .param(\"submission\", rawSubmission))\n        .call()\n        .entity(TalkSubmission.class, spec -> spec.validateSchema());\n}\n```\n\nThat is it. The `spec -> spec.validateSchema()`\n\nconsumer turns on the self-correcting retry loop.\n\nAccording to the [official documentation](https://spring.io/blog/2026/06/23/spring-ai-self-correcting-structured-output) by Christian Tzolov (Spring AI team), the validation loop works as follows:\n\n`int`\n\n, got `null`\n\nfor field `duration`\n\n\") is appended to the user prompt and the call is re-issuedThis is powered by `StructuredOutputValidationAdvisor`\n\n, a recursive advisor that is auto-registered when you call `validateSchema()`\n\n. Default is **3 retry attempts**. The model knows exactly what went wrong and can correct it on the next attempt.\n\nTo customize the retry count, build your own advisor instance:\n\n``` js\nvar validationAdvisor = StructuredOutputValidationAdvisor.builder()\n    .outputType(TalkSubmission.class)\n    .maxRepeatAttempts(5)\n    .build();\n\nChatClient chatClient = ChatClient.builder(chatModel)\n    .defaultAdvisors(validationAdvisor)\n    .build();\n```\n\nSome frontier models support structured output at the API level. Instead of appending the schema to the prompt text, the schema is sent as an API constraint. The provider's runtime enforces conformance, meaning invalid responses cannot be emitted at all.\n\nSpring AI 2.0 exposes this through `useProviderStructuredOutput()`\n\n:\n\n```\nTalkSubmission result = chatClient.prompt()\n    .system(systemPrompt)\n    .user(spec -> spec.text(\"Extract the talk submission: {submission}\")\n        .param(\"submission\", rawSubmission))\n    .call()\n    .entity(TalkSubmission.class, spec -> spec\n        .useProviderStructuredOutput()\n        .validateSchema());\n```\n\nSupported providers as of Spring AI 2.0:\n\nNative structured output is off by default because support varies across models. If a model does not support it, the flag is silently ignored and the prompt-based approach is used instead.\n\nNote: `.entity()`\n\nis only available on `.call()`\n\n, not on `.stream()`\n\n. Typed parsing requires the complete response, so streaming responses cannot be deserialized into a typed object.\n\n**OpenAI does not accept top-level JSON arrays.** If you need a `List<T>`\n\n, wrap it in a container record first:\n\n```\n// Does NOT work with OpenAI native structured output:\nList<TalkSubmission> list = chatClient.prompt()\n    .call()\n    .entity(new ParameterizedTypeReference<List<TalkSubmission>>() {},\n            spec -> spec.useProviderStructuredOutput()); // fails\n\n// Works: wrap in a container\nrecord SubmissionList(List<TalkSubmission> submissions) {}\nSubmissionList result = chatClient.prompt()\n    .call()\n    .entity(SubmissionList.class, spec -> spec.useProviderStructuredOutput());\n```\n\n**Ollama with reasoning models** (like Qwen variants) may emit internal reasoning traces as plain text instead of JSON. Use a non-reasoning model, or combine with `validateSchema()`\n\nso malformed responses are automatically retried.\n\nNot every scenario needs all features enabled. Based on the official docs and video demonstration:\n\n**Frontier models (Claude, GPT-4, Gemini):**\n\n`useProviderStructuredOutput()`\n\nfor API-level enforcement`validateSchema()`\n\nas a safety net for edge cases**Open-source models (Llama, Mistral via Ollama):**\n\n`useProviderStructuredOutput()`\n\nmay have no effect (model-specific)`validateSchema()`\n\nis essential**Production systems:**\n\n`maxRepeatAttempts`\n\nbased on your latency budgetThis feature represents a shift in how we think about LLM integration. For too long, the industry treated unreliable model output as a prompt engineering problem. Write a better prompt. Be more specific. Add examples. Pray harder.\n\nSpring AI 2.0 treats it as a systems problem. Validate. Retry. Self-correct. The same principles we apply to any unreliable external service: network calls, database queries, third-party APIs. LLMs are no different.\n\nIf you are building production applications with LLMs, schema validation is not optional. It is basic engineering.\n\n**Sources:**", "url": "https://wpnews.pro/news/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring", "canonical_source": "https://dev.to/jamilxt/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring-ai-20-4hl7", "published_at": "2026-07-18 00:10:15+00:00", "updated_at": "2026-07-18 00:37:09.097970+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "ai-tools"], "entities": ["Spring AI", "Dan Vega", "Broadcom", "Christian Tzolov", "Llama 3.2", "Ollama", "Claude", "GPT-4"], "alternates": {"html": "https://wpnews.pro/news/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring", "markdown": "https://wpnews.pro/news/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring.md", "text": "https://wpnews.pro/news/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring.txt", "jsonld": "https://wpnews.pro/news/stop-begging-your-llm-for-valid-json-self-correcting-structured-output-in-spring.jsonld"}}