{"slug": "stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents", "title": "Stop Prompting, Start Schemas: The Real Fix for Unreliable AI Agents", "summary": "A developer argues that schema-constrained generation, not prompt engineering, is the reliable fix for AI agents that emit malformed output. The writeup explains that prompting is only a soft constraint on a probabilistic model, while masking the vocabulary during decoding so the model cannot emit tokens that violate a JSON Schema, Pydantic model, or grammar guarantees valid output. It recommends validating hard and failing loudly rather than retrying or silently falling back to defaults.", "body_md": "*Originally published on [tamiz.pro](https://tamiz.pro/insights/stop-prompting-start-schemas-ai-agents).*\n\nEvery developer who has shipped an AI agent has hit the same wall. You craft a prompt that looks bulletproof. You instruct the model to \"return JSON only.\" You add \"do not wrap in markdown.\" You even threaten it. And then, in production, the agent returns:\n\n```\nHere's your result:\n{ \"status\": \"success\", \"data\": { ... }\n```\n\nTrailing backticks. A missing brace. A field that's `null` when the schema says `string`. The model \"understands\" the task, but it can't reliably *emit* the contract you asked for. The fix isn't a better prompt — it's a schema.\n\nThis is a deep dive into why schema-constrained generation — not prompt engineering — is the real fix for unreliable AI agents. We'll look at how it works under the hood, why it beats prompt-based parsing, and how to build agents that fail loudly, validate hard, and never silently corrupt data. If you're building production agents with LLMs, this is the mental model you need.\n\nPrompting is a *soft* constraint. It tells the model what you want, but it doesn't give the model a mechanism to guarantee it. The model is a next-token predictor sampling from a probability distribution over the entire vocabulary — and at every step, it can choose a token that breaks your contract.\n\nWhen you say \"return JSON,\" you're asking the model to satisfy a grammar it has only *statistically* learned. The model has seen thousands of JSON examples in training, so it's likely to produce something JSON-shaped. But \"likely\" is not \"guaranteed.\" The failure modes are predictable and familiar:\n\n`max_tokens` limit mid-object, producing invalid JSON.`null` for required fields.`integer` comes back as `\"42\"` (a string), or a list comes back as a single object.\nEach of these is a *distributional* failure. The model's weights encode a tendency, not a rule. And this is the key insight: **you cannot prompt your way out of a probability distribution.** You can nudge the distribution, but you can't force it to respect a grammar.\n\nThe second problem is that prompt-based approaches defer all the work to *post-hoc parsing*. You generate text, then you try to parse it with `json.loads()` or a regex. When parsing fails, you either retry the whole generation (expensive, non-deterministic) or you fall back to a default (silently wrong). Both are bad. Retrying is a band-aid; it doesn't fix the root cause. It just rolls the dice again.\n\nThe root cause is that you're asking a probabilistic system to produce a deterministic artifact, and you're only enforcing the contract *after* the fact.\n\nSchemas flip this around. Instead of generating freely and hoping, you **constrain the generation process itself** so that the model physically cannot emit invalid output. That's a fundamentally different architecture.\n\nA schema is a formal specification of the output's shape: its fields, types, allowed values, nesting, and constraints. JSON Schema is the canonical example, but the same idea applies to Pydantic models, TypeScript interfaces, Zod schemas, or even a context-free grammar.\n\nThe fix is to feed that schema into the *generation loop*, not into the prompt. Instead of letting the model pick any token from the vocabulary at each step, you **mask the vocabulary** to only the tokens that keep the output valid under the schema.\n\nThis is called **constrained decoding**, **structured outputs**, or **grammar-constrained generation**. It's the difference between asking a model to \"draw a circle\" and handing it a compass. With a compass, the circle is guaranteed — the tool enforces the constraint.\n\nHere's the conceptual shift:\n\n| Approach | Constraint location | Failure mode | Cost of failure | \n|---|---|---|---|\n| Prompt-only | Prompt text (soft) | Malformed output, parsing errors | High — retry, or silent corruption | \n| Prompt + retry | Prompt + post-hoc parse | Still fails on hard cases | Medium — repeated calls, latency | \n| Schema-constrained | Generation loop (hard) | Rare — schema violations mostly impossible | Low — deterministic, one shot | \n\nWith schema-constrained generation, the contract is enforced *during* decoding. The model can only choose tokens that advance toward a valid completion. You get what you asked for, every time.\n\nTo understand why this is so much more reliable, you need to know what's happening at the token level. Let's walk through the mechanics.\n\nAt every decoding step, the model produces a probability distribution over the vocabulary — a vector of logits, one per token. Normally, you sample from this full distribution. With constrained decoding, you build a **mask** of allowed tokens and set the logits of disallowed tokens to `-infinity` (or a very large negative number), so they can never be sampled.\n\n``` python\npython\n# Conceptual — not runnable as-is, but shows the mechanism.\nimport torch\n\nlogits = model(input_ids)  # shape: [1, vocab_size]\nallowed_mask = build_allowed_token_mask(state, schema)  # shape: [1, vocab_size]\n\nmasked_logits = logits.masked_fill(~allowed_mask, float('-inf'))\nnext_token = sample(masked_logits)\n```\n\nThe mask is the entire trick. It's not a heuristic; it's a hard constraint. The model *cannot* produce a token that would violate the schema, because that token has zero probability.\n\nBut how do you know which tokens are allowed at each step? You need to track the current state of the partial output and derive the allowed tokens from it. This is where the schema becomes a **finite-state machine** (or a pushdown automaton for nested structures).\n\nImagine the schema:\n\n```\njson\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"name\": { \"type\": \"string\" },\n    \"age\": { \"type\": \"integer\", \"minimum\": 0 }\n  },\n  \"required\": [\"name\", \"age\"]\n}\n```\n\nAfter the model emits `{ \"name\": \"Alice\",`, the parser state knows:\n\n`name` is already satisfied.`age` (since `required` includes it and it's not yet present).`\"age\"` (or whitespace, or `}` if The state machine encodes all the grammar rules of JSON Schema: object key ordering (or unordered keys), string escapes, number formatting, enum membership, array lengths, `minItems`/` maxItems`, `additionalProperties`, and so on. As the model generates, the state advances, and the allowed token set is recomputed from the new state.\n\nFor arbitrary JSON Schema, implementations typically compile the schema into a **context-free grammar (CFG)** and then use a parser (like a GLR or Earley parser) to track the valid continuations. Libraries such as [Outlines](https://dottxt-ai.github.io/outlines/latest/) and [Instructor](https://python.useinstructor.com/) do exactly this: they convert a Pydantic model or JSON Schema into a grammar, then run constrained decoding against any compatible backend (OpenAI, Anthropic, local models via `transformers`, `llama.cpp`, `vLLM`, etc.).\n\n```\npython\n# Outlines — schema-constrained generation against a local model.\nimport outlines\nfrom pydantic import BaseModel\n\nclass User(BaseModel):\n    name: str\n    age: int\n\nmodel = outlines.models.transformers(\"meta-llama/Meta-Llama-3-8B-Instruct\")\ngenerator = outlines.generate.json(model, User)\n\nresult = generator(\"Extract the user from: Alice is 30 years old.\")\nprint(result)  # User(name='Alice', age=30) — always valid, no parsing needed\n```\n\nThe key takeaway: **the schema is not documentation. It is the generator.** The model never has a chance to emit invalid JSON, because invalid tokens are unreachable.\n\nSince late 2024, major providers have shipped this natively. OpenAI's **Structured Outputs** and Anthropic's **tool-use with strict schemas** both implement constrained decoding on their side. You pass a JSON Schema, and the API guarantees the response matches it. This is the easiest adoption path for most teams.\n\n``` python\npython\nfrom openai import OpenAI\nfrom pydantic import BaseModel\n\nclient = OpenAI()\n\nclass User(BaseModel):\n    name: str\n    age: int\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    response_format={\n        \"type\": \"json_schema\",\n        \"json_schema\": {\n            \"name\": \"user_extraction\",\n            \"strict\": True,\n            \"schema\": User.model_json_schema()\n        }\n    },\n    messages=[{\"role\": \"user\", \"content\": \"Alice is 30.\"}]\n)\n\nuser = User.model_validate_json(response.choices[0].message.content)\n```\n\nNotice what changed: there's no `json.loads()` call, no try/except, no regex cleanup, no retry loop. The response is *guaranteed* to be valid JSON matching the schema, so you can deserialize it directly. If the API can't satisfy the schema, it returns an error — it doesn't silently return garbage.\n\nBefore you build, you need to pick your schema source of truth. The choice matters because it affects how you define, validate, and version your contracts.\n\n| Layer | Pros | Cons | Best for | \n|---|---|---|---|\n| JSON Schema (raw) | Universal, provider-native, language-agnostic | Verbose, no type safety in code | Cross-language contracts, API specs | \n| Pydantic (Python) | Type-safe, auto-generates JSON Schema, integrates with FastAPI | Python-only | Python services, FastAPI backends | \n| TypeScript / Zod | Type-safe, compile-time checks | No direct JSON Schema in all providers | TS frontends, edge functions | \n| Function calling | Native to providers, supports multi-tool | Less expressive than full JSON Schema | Tool-using agents, ReAct loops | \n\n**My recommendation for most teams:** define your contract once as a Pydantic model (or Zod schema), and derive the JSON Schema from it. This gives you type safety in code *and* a provider-compatible schema for constrained decoding. You never hand-write JSON Schema by hand — that's how drift happens.\n\n``` python\npython\nfrom pydantic import BaseModel, Field\nfrom typing import Literal\n\nclass Order(BaseModel):\n    order_id: str = Field(description=\"Unique order identifier\")\n    status: Literal[\"pending\", \"paid\", \"shipped\", \"cancelled\"]\n    total_cents: int = Field(ge=0, description=\"Total in cents, non-negative\")\n    items: list[str] = Field(min_length=1, max_length=50)\n```\n\nNow `Order.model_json_schema()` gives you a JSON Schema that both OpenAI and Anthropic can consume directly. And `Order.model_validate_json()` gives you a validated Python object on the way out. One source of truth, two directions.\n\nLet's build a concrete, production-shaped example: an order-processing agent that extracts structured data from a natural-language email and decides whether to approve it. This is the kind of task that breaks naive prompt-based agents constantly.\n\n``` python\npython\nfrom pydantic import BaseModel, Field\nfrom typing import Literal, Optional\n\nclass OrderItem(BaseModel):\n    sku: str\n    quantity: int = Field(ge=1, le=1000)\n    unit_price_cents: int = Field(ge=0)\n\nclass Extraction(BaseModel):\n    \"\"\"Extracted order data from a customer email.\"\"\"\n    customer_email: str\n    order_items: list[OrderItem] = Field(min_length=1)\n    shipping_address: str\n    priority: Literal[\"standard\", \"express\"] = \"standard\"\n    notes: Optional[str] = None\npython\npython\nfrom openai import OpenAI\n\nclient = OpenAI()\n\nSYSTEM_PROMPT = \"\"\"\nYou are an order extraction agent. Extract order information from customer emails.\nReturn ONLY the structured data. Never invent fields that are not in the schema.\nIf a required field is missing from the email, use your best judgment to infer it,\nbut never fabricate a customer email address.\n\"\"\"\n\ndef extract_order(email_body: str) -> Extraction:\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        response_format={\n            \"type\": \"json_schema\",\n            \"json_schema\": {\n                \"name\": \"order_extraction\",\n                \"strict\": True,\n                \"schema\": Extraction.model_json_schema()\n            }\n        },\n        messages=[\n            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n            {\"role\": \"user\", \"content\": email_body}\n        ],\n        temperature=0,  # deterministic, best for extraction\n    )\n\n    # No json.loads(), no try/except, no regex cleanup.\n    # The API guarantees this matches Extraction's schema.\n    return Extraction.model_validate_json(response.choices[0].message.content)\n```\n\nEven with structured outputs, you should still validate. Why? Two reasons: (1) it catches semantic issues that JSON Schema can't express, and (2) it protects you if you ever switch to a provider that doesn't support strict schemas. Validation is cheap; silent corruption is expensive.\n\n```\npython\nclass ExtractionError(Exception):\n    \"\"\"Raised when an extraction fails semantic validation.\"\"\"\n    def __init__(self, message: str, details: dict):\n        super().__init__(message)\n        self.details = details\n\ndef validate_semantics(extraction: Extraction) -> None:\n    \"\"\"Business rules that JSON Schema can't enforce.\"\"\"\n    if not extraction.customer_email.endswith(\"@\"):\n        raise ExtractionError(\"Customer email is malformed\", {\n            \"field\": \"customer_email\",\n            \"value\": extraction.customer_email\n        })\n\n    total = sum(i.unit_price_cents * i.quantity for i in extraction.order_items)\n    if total <= 0:\n        raise ExtractionError(\"Order total must be positive\", {\n            \"field\": \"order_items\",\n            \"value\": total\n        })\n```\n\nNow the full agent loop. Notice the retry logic is *not* for parsing failures — those are impossible now. It's for *semantic* failures and provider errors, which is a much smaller and more predictable surface.\n\n``` python\npython\nimport logging\nfrom tenacity import retry, stop_after_attempt, wait_exponential\n\nlogger = logging.getLogger(__name__)\n\n@retry(\n    stop=stop_after_attempt(3),\n    wait=wait_exponential(multiplier=1, min=1, max=10),\n    reraise=True,\n)\ndef safe_extract(email_body: str) -> Extraction:\n    extraction = extract_order(email_body)\n    validate_semantics(extraction)\n    return extraction\n\n# Usage\ntry:\n    order = safe_extract(\"Hi, please ship 2x SKU-1234 and 1x SKU-5678 to 123 Main St.\")\n    print(f\"Processed order for {order.customer_email}: {len(order.order_items)} items\")\nexcept ExtractionError as e:\n    logger.error(\"Semantic validation failed: %s\", e.details)\n    # Route to human review, don't silently drop\n    raise\n```\n\nThis is the real fix. The prompt still matters — it guides the model's *reasoning* — but it no longer bears the burden of enforcing the *format*. That burden is on the schema, where it belongs.\n\nJSON schemas are just the beginning. The same principle extends to **tool calling** and **multi-step agent loops**, which is where most production agents actually live.\n\nWhen an agent calls a tool, it's not just returning data — it's *selecting an action*. That selection is a structured decision, and it should be schema-constrained too. Provider-native tool calling already does this: you declare each tool's parameters as a JSON Schema, and the model must emit a valid call.\n\n```\npython\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_order_status\",\n            \"description\": \"Get the current status of an order.\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"order_id\": {\"type\": \"string\", \"description\": \"The order ID\"}\n                },\n                \"required\": [\"order_id\"],\n                \"additionalProperties\": False\n            }\n        }\n    }\n]\n```\n\nWith `strict: True` and `additionalProperties: False`, the provider guarantees the tool call is well-formed. The agent can't call `get_order_status` without an `order_id`, and it can't pass extra parameters. This eliminates an entire class of \"tool call hallucination\" bugs.\n\nA ReAct (Reasoning + Acting) agent interleaves reasoning steps and tool calls. The classic bug in naive ReAct implementations is that the model emits a malformed action, the parser fails, and the whole loop crashes. Schema enforcement fixes this at the source.\n\n``` python\npython\nfrom pydantic import BaseModel\nfrom typing import Union\n\nclass Think(BaseModel):\n    thought: str\n\nclass Act(BaseModel):\n    tool_name: str\n    tool_args: dict\n\nclass ReActStep(BaseModel):\n    \"\"\"One step in the ReAct loop. The model must choose exactly one action.\"\"\"\n    step: Union[Think, Act] = Field(description=\"Either a reasoning step or a tool action\")\n    done: bool = Field(description=\"True if the task is complete\")\n```\n\nBy making `step` a discriminated union, you force the model to produce *either* a thought *or* an action — not both, not neither, not a malformed hybrid. The loop then becomes deterministic:\n\n``` php\npython\ndef run_agent(task: str) -> str:\n    history = [{\"role\": \"user\", \"content\": task}]\n    for _ in range(MAX_STEPS):\n        response = client.chat.completions.create(\n            model=\"gpt-4o\",\n            response_format={\n                \"type\": \"json_schema\",\n                \"json_schema\": {\n                    \"name\": \"react_step\",\n                    \"strict\": True,\n                    \"schema\": ReActStep.model_json_schema()\n                }\n            },\n            messages=history,\n        )\n        step = ReActStep.model_validate_json(response.choices[0].message.content)\n\n        if step.done:\n            return final_answer(step)\n\n        if isinstance(step.step, Act):\n            result = execute_tool(step.step.tool_name, step.step.tool_args)\n        else:\n            result = None  # reasoning step, no tool call\n\n        history.append({\"role\": \"assistant\", \"content\": response.choices[0].message.content})\n        history.append({\"role\": \"user\", \"content\": str(result or step.step.thought)})\n\n    raise MaxStepsExceeded()\n```\n\nNotice there's no `json.loads()`, no `if \"tool\" in text` heuristic, no regex extraction. The schema does all the structural work. The model's *reasoning* is still free — that's what the prompt guides — but the *structure* is locked.\n\nSchemas make agents reliable, but reliability in production requires more than just enabling structured outputs. Here's what I've learned shipping these systems.\n\nA schema is an API contract. When you change it, you break every stored response and every downstream consumer. Version it explicitly:\n\n```\npython\nclass ExtractionV2(BaseModel):\n    \"\"\"V2: adds currency field and drops legacy notes field.\"\"\"\n    customer_email: str\n    order_items: list[OrderItem]\n    shipping_address: str\n    currency: Literal[\"USD\", \"EUR\", \"GBP\"] = \"USD\"\n    priority: Literal[\"standard\", \"express\"] = \"standard\"\n```\n\nKeep the old version around for replay and migration. Never silently mutate a schema in place.\n\nThe worst thing an agent can do is return `null` for a required field and have your code treat it as valid. Always validate on the way out, even with structured outputs. If validation fails, raise an error that gets routed to a human — don't log and continue.\n\n```\npython\ntry:\n    extraction = safe_extract(email)\nexcept ExtractionError as e:\n    # Route to human review queue, not to a silent default\n    review_queue.enqueue(email, error=e.details)\n    raise\n```\n\nExtraction is a deterministic task. `temperature=0` gives you the most reproducible results. For open-ended reasoning or creative tasks, you might want a bit more variety — but even then, keep the *structure* constrained and vary the *content*, not the shape.\n\nTrack these metrics, and you'll see the difference schemas make:\n\nJSON Schema can't express everything. It can't say \"this date must be in the future\" or \"this email must exist in our database\" or \"these two fields must be consistent with each other.\" Those are **semantic** constraints, and they belong in your validation layer (Pydantic validators, business rules, etc.). Schema-constrained generation handles *syntax*; your code handles *semantics*. Don't conflate the two.\n\nLet's be honest about the limits. Schema-constrained generation is a massive upgrade, but it's not a silver bullet. There are two classes of failure it doesn't fix:\n\n**Reasoning failures.** The model produces perfectly valid JSON, but the *content* is wrong. It extracts the wrong date, picks the wrong enum value, or infers a fact that isn't in the source. Schema constraints can't fix bad reasoning — they can only make sure the bad reasoning is *well-formatted*. This is where you need better prompts, better context, retrieval, or a more capable model.\n\n**Semantic inconsistency.** Two fields that are individually valid but jointly impossible (e.g., `total_cents` doesn't match the sum of `items`). No schema can express cross-field invariants, so you need validation logic and, when it fails, a recovery strategy: retry with the validation error fed back into the prompt, or escalate to a human.\n\nThe schema guarantees the *shape*. It does not guarantee the *truth*. Those are two different problems, and you need both.\n\nFor failure #2, a good pattern is to feed validation errors back into the model as a corrective message:\n\n``` python\npython\ndef extract_with_self_correction(email_body: str, max_attempts: int = 2) -> Extraction:\n    messages = [{\"role\": \"user\", \"content\": email_body}]\n    for attempt in range(max_attempts):\n        response = client.chat.completions.create(\n            model=\"gpt-4o\",\n            response_format={\"type\": \"json_schema\", \"json_schema\": {...}},\n            messages=messages,\n        )\n        try:\n            extraction = Extraction.model_validate_json(response.choices[0].message.content)\n            validate_semantics(extraction)\n            return extraction\n        except (ValidationError, ExtractionError) as e:\n            if attempt == max_attempts - 1:\n                raise\n            messages.append({\"role\": \"assistant\", \"content\": response.choices[0].message.content})\n            messages.append({\"role\": \"user\", \"content\": f\"Your output failed validation: {e}. Please correct it.\"})\n```\n\nThis keeps the *format* guaranteed by the schema, while giving the model a chance to fix its *reasoning*. It's the best of both worlds.\n\nIf you're using a provider-native structured outputs API (e.g., `response_format` with `json_schema`), malformed JSON should be impossible — the provider enforces it during decoding. If you still see it, you're likely (a) not passing `strict: True`, (b) not setting `additionalProperties: False` on all objects, or (c) using a provider that only *suggests* rather than *enforces* the schema. Some providers return a schema that's a subset of JSON Schema and silently degrade. Always validate on the way out as a safety net.\n\nFunction calling is a specific form of structured output where the schema describes a tool's parameters, and the model's output is a tool invocation. Structured outputs is the broader concept: any JSON Schema can constrain the response. Function calling is often the easiest way to get structured outputs on older models, but the dedicated `json_schema` response format is now more general and more reliable.\n\nUse Pydantic (or Zod in TypeScript) as your source of truth and derive JSON Schema from it. Hand-writing JSON Schema is error-prone and leads to drift between your code types and your API contract. With Pydantic, you get type safety, validation, and auto-generated schemas in one place. Only use raw JSON Schema when you need a language-agnostic contract shared across services.\n\nIf you're building agents that touch production data, make the schema the contract — not the prompt. The prompt guides reasoning; the schema enforces structure. That division of labor is what separates a demo from a system you can actually ship. For more on engineering reliable AI systems, see [Tamiz's Insights](https://tamiz.pro/insights) and the engineering guides at [tamiz.pro](https://tamiz.pro).\n\nValidation is only the first layer of the schema-first approach. The real power emerges when you treat schemas as composable building blocks rather than isolated constraints. Think of it this way: a single schema tells you what a valid response looks like, but a composition of schemas tells you how a system should behave across multiple turns, tools, and failure modes.\n\nLet's move from simple validation to a more sophisticated pattern—schema-driven state machines. Consider a customer support agent that needs to handle a multi-step refund process. Instead of writing a prompt that says \"handle the refund flow,\" you define a schema for each state and a transition map between them.\n\n``` python\npython\nfrom pydantic import BaseModel, Field\nfrom typing import Literal, Union\nfrom enum import Enum\n\nclass RefundState(str, Enum):\n    AWAITING_ORDER_ID = \"awaiting_order_id\"\n    AWAITING_REASON = \"awaiting_reason\"\n    AWAITING_APPROVAL = \"awaiting_approval\"\n    COMPLETED = \"completed\"\n\nclass OrderIDRequest(BaseModel):\n    intent: Literal[\"provide_order_id\"] = \"provide_order_id\"\n    order_id: str = Field(..., pattern=r\"^ORD-\\d{6}$\")\n\nclass RefundReasonRequest(BaseModel):\n    intent: Literal[\"provide_reason\"] = \"provide_reason\"\n    reason: str = Field(..., min_length=10, max_length=500)\n    urgency: Literal[\"low\", \"medium\", \"high\"] = \"medium\"\n\nclass ApprovalDecision(BaseModel):\n    intent: Literal[\"approve_refund\"] = \"approve_refund\"\n    approved: bool\n    notes: str | None = None\n\nclass RefundResponse(BaseModel):\n    state: RefundState\n    data: Union[OrderIDRequest, RefundReasonRequest, ApprovalDecision]\n\n# Define the transition function\ndef next_state(current: RefundState, response: RefundResponse) -> RefundState:\n    if current == RefundState.AWAITING_ORDER_ID:\n        if isinstance(response.data, OrderIDRequest):\n            return RefundState.AWAITING_REASON\n        raise ValueError(\"Expected order ID\")\n\n    if current == RefundState.AWAITING_REASON:\n        if isinstance(response.data, RefundReasonRequest):\n            return RefundState.AWAITING_APPROVAL\n        raise ValueError(\"Expected refund reason\")\n\n    if current == RefundState.AWAITING_APPROVAL:\n        if isinstance(response.data, ApprovalDecision):\n            return RefundState.COMPLETED if response.data.approved else RefundState.AWAITING_REASON\n        raise ValueError(\"Expected approval decision\")\n\n    return RefundState.COMPLETED\n```\n\nThis pattern gives you something that pure prompting can never provide: **provable state transitions**. You can unit-test every path, enforce invariants at compile time, and guarantee that the agent cannot skip a step or hallucinate a state that doesn't exist.\n\nNow that you have schemas, you need an execution layer that actually uses them. The naive approach is to parse the model output and hope it matches. The robust approach is to validate, retry, and fall back through a structured pipeline.\n\nHere's a production-ready agent loop that treats schemas as first-class citizens:\n\n``` python\npython\nimport json\nfrom typing import Type, TypeVar, Generic\nfrom pydantic import BaseModel, ValidationError\n\nT = TypeVar('T', bound=BaseModel)\n\nclass SchemaAgent(Generic[T]):\n    def __init__(\n        self,\n        model: callable,  # LLM callable: (prompt, schema) -> str\n        schema: Type[T],\n        max_retries: int = 3,\n        fallback: T | None = None,\n    ):\n        self.model = model\n        self.schema = schema\n        self.max_retries = max_retries\n        self.fallback = fallback\n\n    def run(self, prompt: str, context: dict | None = None) -> T:\n        last_error = None\n\n        for attempt in range(self.max_retries):\n            try:\n                # Pass the schema as a structured constraint, not just a prompt hint\n                raw_output = self.model(\n                    prompt=prompt,\n                    schema=self.schema.model_json_schema(),\n                    context=context or {},\n                )\n\n                # Validate and coerce\n                validated = self.schema.model_validate_json(raw_output)\n                return validated\n\n            except ValidationError as e:\n                last_error = e\n                # Feed the error back into the prompt for self-correction\n                prompt = self._augment_prompt(prompt, e)\n                continue\n            except json.JSONDecodeError as e:\n                last_error = e\n                prompt = self._augment_prompt(prompt, \"Response was not valid JSON\")\n                continue\n\n        if self.fallback is not None:\n            return self.fallback\n\n        raise RuntimeError(\n            f\"Agent failed after {self.max_retries} attempts. Last error: {last_error}\"\n        )\n\n    def _augment_prompt(self, prompt: str, error: Exception) -> str:\n        return f\"\"\"\n{prompt}\n\nYour previous response was invalid for the following reason:\n{error}\n\nPlease provide a corrected response that strictly adheres to the schema.\n\"\"\"\n```\n\nNotice what's happening here. The schema isn't just appended to the prompt as text—it's passed as a structured parameter to the model. This forces the model to generate output that can be parsed and validated deterministically. The retry loop includes the validation error in the next prompt, which gives the model a concrete correction target rather than a vague \"try again.\"\n\nReal-world agents rarely do one thing. They orchestrate tools, call external APIs, and maintain conversational context. The schema-first approach shines in these scenarios because it gives you a way to define the contract for each step.\n\nHere's a pattern for a multi-step agent that handles a complete user request:\n\n``` python\npython\nfrom typing import List, Dict, Any\nfrom pydantic import BaseModel\n\nclass ToolCall(BaseModel):\n    tool_name: str\n    arguments: Dict[str, Any]\n\nclass AgentStep(BaseModel):\n    step_id: int\n    tool_calls: List[ToolCall]\n    expected_output_schema: Dict[str, Any]\n    next_step: int | None = None\n\nclass WorkflowDefinition(BaseModel):\n    steps: Dict[int, AgentStep]\n    entry_point: int = 0\n\nclass WorkflowExecutor:\n    def __init__(self, workflow: WorkflowDefinition, tools: Dict[str, callable]):\n        self.workflow = workflow\n        self.tools = tools\n\n    def execute(self, initial_input: Dict[str, Any]) -> Dict[str, Any]:\n        current_step = self.workflow.entry_point\n        context = initial_input\n\n        while current_step is not None:\n            step = self.workflow.steps[current_step]\n\n            # Execute each tool call in sequence\n            for tool_call in step.tool_calls:\n                if tool_call.tool_name not in self.tools:\n                    raise ValueError(f\"Unknown tool: {tool_call.tool_name}\")\n\n                result = self.tools[tool_call.tool_name](**tool_call.arguments)\n\n                # Validate the result against the expected schema\n                self._validate_result(result, step.expected_output_schema)\n\n                # Merge result into context\n                context.update(result)\n\n            current_step = step.next_step\n\n        return context\n\n    def _validate_result(self, result: Any, schema: Dict[str, Any]) -> None:\n        # Use a lightweight validator here, or integrate with Pydantic\n        # This is where you'd enforce the schema contract\n        pass\n```\n\nThis pattern is especially valuable when you're building agents that call external APIs. Instead of hoping the LLM correctly formats a function call, you define the exact schema for each tool's input and output. The workflow executor enforces those schemas at runtime, so a malformed tool call fails fast rather than propagating through the system.\n\nOne of the most underrated benefits of schema-first design is that it gives you a structured way to handle errors. When an agent produces invalid output, you don't just retry—you can analyze *why* it failed and take corrective action.\n\n``` python\npython\nfrom pydantic import BaseModel, ValidationError\nfrom typing import Literal, Union\n\nclass RecoveryAction(BaseModel):\n    action: Literal[\"retry\", \"clarify\", \"escalate\"]\n    message: str\n    suggested_schema: Dict[str, Any] | None = None\n\nclass ErrorAnalyzer:\n    def analyze(self, error: ValidationError, attempts: int) -> RecoveryAction:\n        if attempts < 2:\n            return RecoveryAction(\n                action=\"retry\",\n                message=\"Please correct the output to match the schema.\",\n                suggested_schema=error.schema,\n            )\n\n        if self._is_ambiguity_error(error):\n            return RecoveryAction(\n                action=\"clarify\",\n                message=\"The request is ambiguous. Please provide more specific information.\",\n            )\n\n        return RecoveryAction(\n            action=\"escalate\",\n            message=\"Unable to produce valid output. Escalating to human review.\",\n        )\n\n    def _is_ambiguity_error(self, error: ValidationError) -> bool:\n        # Heuristic: ambiguity errors often involve missing fields or wrong types\n        return any(\"missing\" in str(e) or \"type\" in str(e).lower() for e in error.errors())\n```\n\nThis error analyzer gives you a decision tree for handling failures. Instead of blindly retrying, you can escalate to a human when the model consistently fails, or ask for clarification when the input is ambiguous. This is the kind of robustness that makes agents actually deployable in production.\n\nWe started this article with a provocative claim: prompting is not the fix for unreliable AI agents. By now, I hope you see why. Prompts are suggestions; schemas are contracts. Suggestions can be ignored; contracts must be honored.\n\nThe shift to schema-first development changes how you build agents in fundamental ways:\n\nThe tools I've shown here—Pydantic for validation, state machines for workflow control, and structured error recovery—are not exotic. They're the same patterns you'd use for any production software. The only difference is that now you're applying them to systems that generate code and text dynamically.\n\nStop prompting. Start schemas. Your agents will thank you.\n\n", "url": "https://wpnews.pro/news/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents", "canonical_source": "https://dev.to/tamizuddin/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents-50o4", "published_at": "2026-11-13 06:08:33+00:00", "updated_at": "2026-11-13 06:19:59.234647+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "structured-data", "developer-tools"], "entities": ["tamiz.pro", "JSON Schema", "Pydantic", "Zod", "TypeScript"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents", "markdown": "https://wpnews.pro/news/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents.md", "text": "https://wpnews.pro/news/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents.txt", "jsonld": "https://wpnews.pro/news/stop-prompting-start-schemas-the-real-fix-for-unreliable-ai-agents.jsonld"}}