# Ollama Structured Outputs in Practice — Getting Type-Safe JSON from Local LLMs with Pydantic

> Source: <https://dev.to/jangwook_kim_e31e7291ad98/ollama-structured-outputs-in-practice-getting-type-safe-json-from-local-llms-with-pydantic-m38>
> Published: 2026-06-17 06:38:48+00:00

`json.loads(response)`

fails at a certain point. You told the model "return JSON only," but it added a

``` json markdown code fence around everything. A quick regex strips it — until that regex hits an edge case, and that edge case blows up in production.

Since Ollama 0.3.0, passing a JSON schema to the `format`

parameter eliminates this problem at the root. The model's inference itself is constrained by the schema, so no code fences, no explanatory text, no mid-thought artifacts. Just parseable JSON.

I ran these tests locally with Gemma4 and Ollama 0.30.7 to see how well it holds up in practice.

The most common problem when running Ollama locally — without a cloud LLM API — is JSON parsing. Two reasons.

First, text generation models are trained toward "natural text." Even if you ask for JSON only, they'll often wrap it in

json ...

` blocks or prepend "Of course! Here is the JSON you requested:" style text. Here's what I reproduced directly:`

```
json
Input: 'Give me 3 Python tips as JSON with keys: tips (array), difficulty (1-5)'
Model output (no format parameter):

``` json
{
  "tips": [
    "Master the fundamentals first...",
    ...
  ]
}
```

JSON parse: FAILED

```
Python's `json.loads()` can't handle the markdown wrapper. The "JSON only" instruction is unreliable in production.

Second, speed. I measured the same query both ways: 32 seconds without structured output, 5 seconds with it. More on why below.

## How the Ollama format Parameter Works

Ollama's `/api/generate` endpoint has a `format` field. Pass a JSON schema object and Ollama applies **constrained decoding** during inference.
```

python

import json

import urllib.request

def ollama_structured(prompt, schema, model="gemma4:e4b"):

payload = {

"model": model,

"prompt": prompt,

"format": schema, # ← pass JSON schema object directly

"stream": False,

"options": {"temperature": 0}

}

data = json.dumps(payload).encode()

req = urllib.request.Request(

"[http://localhost:11434/api/generate](http://localhost:11434/api/generate)",

data=data,

headers={"Content-Type": "application/json"}

)

with urllib.request.urlopen(req, timeout=60) as resp:

result = json.loads(resp.read())

return result["response"]

```
Constrained decoding sets the probability of any token that would violate the schema to zero at each generation step. So even if the model "wants" to generate a markdown fence, the schema makes it physically impossible. That's also where the speed gain comes from — the model doesn't waste tokens on formatting decisions.

Here are the measured numbers:
```

bash

Without structured output:

Raw (first 200 chars):

``` json\n{\n "tips": ["Master the fundamentals first...

Time: 31.84s

JSON parse: FAILED (markdown wrapper)

With structured output:

Structured: {"tips": ["Understand the concept of indentation...", ...], "difficulty": 2}

Time: 4.99s

JSON parse: SUCCESS

```
6.4x difference. Local LLMs are already slow, and adding unreliable parsing on top makes the whole pipeline feel worse.

## Wiring Pydantic Models

Writing JSON schema objects by hand is tedious. With Pydantic models, `model_json_schema()` generates the schema automatically.

``` python
from pydantic import BaseModel
from typing import List, Dict, Any, Literal

class CodeReview(BaseModel):
    severity: str  # "critical", "warning", "info"
    file: str
    line: int
    message: str
    suggestion: str

class ReviewResult(BaseModel):
    total_issues: int
    critical_count: int
    reviews: List[CodeReview]

# Pydantic → JSON schema, automatically
schema = ReviewResult.model_json_schema()

raw = ollama_structured(prompt, schema)

# Parses and validates in one step
result = ReviewResult.model_validate_json(raw)
```

`model_validate_json`

parses the JSON string and runs Pydantic validation simultaneously. If `severity`

gets an integer or `line`

gets a string, it throws `ValidationError`

. Catching that and retrying with a modified prompt is the common pattern in real agents.

Actual output from the code review test:

```
=== Code Review Output ===
Total issues: 3
Critical: 2
  [CRITICAL] process_user_data:2 - SQL Injection Vulnerability (Direct String Formatting)
  [CRITICAL] process_user_data:3 - Storing Passwords in Plain Text (Data Leakage)
  [HIGH] process_user_data:4 - Potential Unused/Incomplete Database Interaction
```

`total_issues: 3`

and `critical_count: 2`

come in as integers. `if result.critical_count > 0`

branches safely.

The strongest use case for structured output is an agent deciding which tool to call next. You pass the tool list and current situation, and get back a type-safe tool call selection.

``` python
from typing import Literal, Dict, Any

class ToolCall(BaseModel):
    tool_name: Literal["web_search", "read_file", "write_file", "execute_code"]
    parameters: Dict[str, Any]
    reasoning: str

schema = ToolCall.model_json_schema()

user_task = "Find the current Bitcoin price and save it to btc_price.txt"
prompt = f"""You are an AI agent. Decide which tool to call next.
Available tools: web_search, read_file, write_file, execute_code
Task: {user_task}
Choose ONE tool call."""

raw = ollama_structured(prompt, schema)
tool_call = ToolCall.model_validate_json(raw)

print(f"Tool: {tool_call.tool_name}")
print(f"Params: {tool_call.parameters}")
=== Agent tool dispatch ===
Tool: web_search
Params: {'query': 'current Bitcoin price'}
Reasoning: The task requires finding real-time information...

Dispatch: OK (type-safe)
```

Because `tool_name`

is typed as `Literal["web_search", "read_file", ...]`

, `tool_call.tool_name`

is always one of those four values. If the model invents a nonexistent tool name, Pydantic throws `ValidationError`

. The `if tool_call.tool_name == "web_search"`

branch is safe to write.

This is architecturally the same as function calling in cloud APIs. Comparing it with [Claude Agent SDK's Tool Use patterns](https://dev.to/en/blog/en/claude-agent-sdk-tool-use-complete-guide-2026) shows an interesting design difference: cloud LLMs handle tool selection natively at the model level, while local Ollama needs an explicit JSON schema + Pydantic validation layer.

Honestly, it doesn't work perfectly in every case. Testing with Gemma4:e4b (4-bit quantized, 4B parameters), I found a few real constraints.

**Deeply nested schemas.** JSON schemas nested 3+ levels deep (`List[Dict[str, List[BaseModel]]]`

) sometimes return empty arrays at intermediate levels. The 12B model (`gemma4:12b-it-qat`

) reduces this, but doesn't eliminate it. This is a fundamental limitation of the model's context handling.

**Optional field handling.** Fields declared as `Optional[str]`

sometimes get filled with empty string `""`

instead of `null`

. Pydantic validation passes, but semantics differ. You need `@validator`

post-processing.

**Schema size.** A large Pydantic model's JSON schema can reach hundreds of tokens. That occupies context window space, reducing the room available for the actual prompt. Complex schemas need stronger models.

Once you've deployed Ollama as an API server (covered in the [Ollama FastAPI production guide](https://dev.to/en/blog/en/ollama-fastapi-production-deployment-guide-2026)), switching models at runtime based on schema complexity becomes a viable optimization.

| Situation | Approach | Why |
|---|---|---|
| Simple data extraction (1-2 levels) |
`format` + `json.loads()`
|
Fast, no overhead |
| Type validation needed |
`format` + Pydantic |
ValidationError catches issues early |
| Agent tool selection |
`format` + Pydantic `Literal`
|
Blocks invalid tool names |
| Complex nested schema | Consider larger model | Small local model limitations |
| Simple text response | No `format`
|
Avoid unnecessary constrained decoding overhead |

I think of this as a switch that moves JSON parse reliability from "unreliable" to "near 100%." There was a time I was appending "JSON only please" to every prompt and hoping for the best. Measuring the actual difference made clear how fragile that approach was.

``` python
import json
import urllib.request
from typing import List, Optional, Dict, Any
from pydantic import BaseModel

def ollama_structured(prompt: str, model_cls: type[BaseModel], 
                      model: str = "gemma4:e4b") -> BaseModel:
    """
    Helper that combines Ollama structured output + Pydantic validation.
    """
    schema = model_cls.model_json_schema()
    payload = {
        "model": model,
        "prompt": prompt,
        "format": schema,
        "stream": False,
        "options": {"temperature": 0}
    }
    data = json.dumps(payload).encode()
    req = urllib.request.Request(
        "http://localhost:11434/api/generate",
        data=data,
        headers={"Content-Type": "application/json"}
    )
    with urllib.request.urlopen(req, timeout=120) as resp:
        result = json.loads(resp.read())
    return model_cls.model_validate_json(result["response"])

# Usage example
class SentimentAnalysis(BaseModel):
    sentiment: str       # "positive", "negative", "neutral"
    confidence: float    # 0.0 ~ 1.0
    key_phrases: List[str]

result = ollama_structured(
    "Analyze sentiment: 'This new MacBook is amazing but too expensive'",
    SentimentAnalysis
)
print(f"Sentiment: {result.sentiment} ({result.confidence:.0%})")
print(f"Key phrases: {result.key_phrases}")
```

This only covers the simplest cases. A real agent needs a bit more.

**Retry logic.** When Pydantic `ValidationError`

fires, retry with a slightly modified prompt — ideally including the error message. Models often self-correct when they can see why they were wrong.

**Streaming.** With `stream: true`

, you can receive the JSON incrementally as it generates. Pair with a streaming JSON parser like `ijson`

for memory-efficient handling of large responses.

**Model switching.** Route simple extractions to `gemma4:e4b`

(fast) and complex nested schemas to `gemma4:12b-it-qat`

(accurate) at runtime. [Structuring an entire agent with Pydantic AI](https://dev.to/en/blog/en/pydantic-ai-type-safe-agent-tutorial-2026) shows how to abstract this decision to the framework level.

If you're already running a Gemma4-based agent locally, adding the `format`

parameter today is a one-line change with a measurable reliability improvement. Especially anywhere in the agent loop where an invalid response immediately causes a downstream error.