Part 1 left us with a model that talks back beautifully but without structure. This is fine, even pretty cool, for a chatbot used by humans. We understand natural language and fuzziness. Unstructured output is, however, terrible for software. You cannot parse the output into a typed object if the shape is different every time.
To build an application around a model, you must make it respect an interface. This part covers the two skills that turn model output into something you can build on: steering the model with prompts, and enforcing structured output.
A prompt (an input message you give to the model) is best understood as runtime configuration. The model responds to the information, constraints, and examples you provide. Prompting the right way is therefore crucial for taming the model's output. Entire books have been written about prompt engineering. Here are a few quick principles:
Use the system prompt. The first message the model receives is often not the user's first message, but a system prompt that precedes the conversation. In chat interfaces such as ChatGPT, the system prompt is invisible to the user, but it is sent to the model on every turn along with the conversation history. Use it to define the model's overall behavior by specifying rules, constraints, and tone that should apply to every request. For example, "Do not invent fee figures if the knowledge base contains no relevant data" is a business rule that belongs in the system prompt.
Assign a role. Models perform better when they know what expertise to apply. Instead of "Summarize this contract," write "You are a legal assistant. Summarize this contract for a software engineer, highlighting payment terms, termination clauses, and obligations". This role assignment can live in the system prompt.
Show examples. One of the most effective prompting techniques is to show the model examples of the behavior you want. Instead of explaining a format in detail, provide a few input + output examples. Models are extremely good at recognizing and continuing patterns, and examples often define the desired behavior more precisely than prose. This is also called "few shot prompting".
Input: "Refund is requested because the item was damaged."
Output: "Category: Product damage. Outcome: Refund denied."
Input: "Refund is requested because item never arrived."
Output: "Category: Shipment lost. Outcome: Refund accepted."
A model's natural output is text. Humans can interpret text easily, but software systems want contracts. If you need to store a model's result, trigger a workflow, or pass data to another service, dealing with free-form text is difficult. You need an output that can be parsed reliably into a typed object. The solution is the same as in traditional software: define a schema and make the model produce output that conforms to it.
Let's see how PayIQ gets structured output from the model. Create 02_structured_output.py
:
from enum import Enum
from dotenv import load_dotenv
from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model
load_dotenv()
class WinLikelihood(str, Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
class ProbabilityRange(BaseModel):
"""Estimated probability range for winning the dispute."""
low_pct: int = Field(
ge=0,
le=100,
description="Lower bound of estimated win probability percentage"
)
high_pct: int = Field(
ge=0,
le=100,
description="Upper bound of estimated win probability percentage"
)
class DisputeAssessment(BaseModel):
"""Structured verdict on a card chargeback / payment dispute."""
win_likelihood: WinLikelihood = Field(
description="Overall likelihood of winning the dispute"
)
key_evidence: list[str] = Field(
min_length=2,
max_length=4,
description="Most important evidence to submit, written as short phrases"
)
estimated_win_probability: ProbabilityRange
recommended_next_step: str = Field(
description="Single most important action to take before the deadline"
)
SYSTEM_PROMPT = (
"You are a payment operations analyst who assesses card chargebacks for merchants. "
"Judge the case only on the evidence the user provides. "
"If a decisive piece of evidence is missing, say so in the recommended next step "
"instead of assuming it exists."
)
model = init_chat_model("anthropic:claude-sonnet-5")
structured_model = model.with_structured_output(DisputeAssessment)
result: DisputeAssessment = structured_model.invoke([
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": (
"Assess this chargeback: €480 online card payment, reason code 10.4 "
"(fraud, card-absent), customer claims they never made the purchase. "
"We have a 3-D Secure authentication record, an AVS match, and "
"delivery confirmation to the cardholder's billing address. Response "
"deadline in 12 days."
),
},
])
print(result.model_dump_json(indent=2))
Note the two principles from above at work: the system prompt assigns a role and defines failure behavior, and we pass a list of messages, exactly the interface from part 1.
The result is not a string but rather an actual DisputeAssessment
object. Under the hood, with_structured_output
converts your Pydantic class into a JSON schema and hands it to the provider, which constrains generation to that shape. LangChain then parses the response back into your class.
So what happens if the model does not follow the schema and outputs something else? Well, it depends. LangChain raises a validation error by default, and with_structured_output(..., include_raw=True)
lets you inspect the raw response instead of raising. Some implementations also support automatic retries: if parsing fails, the framework sends the error back to the model and asks it to produce corrected output.
Either way, it fails fast rather than silently passing invalid data downstream. This follows the same principle used in reliable backend systems: parse once, then work with typed data. Convert untrusted input into a representation your application can reason about, or throw if it doesn't fit.
The output will look like this:
$ python3 02_structured_output.py
{
"win_likelihood": "high",
"key_evidence": [
"3-D Secure authentication record (liability shift)",
"AVS match on billing address",
"Delivery confirmation to cardholder's billing address",
"Order/session details linking IP or device to cardholder if available"
],
"estimated_win_probability": {
"low_pct": 70,
"high_pct": 85
},
"recommended_next_step": "Confirm the 3-D Secure record shows fully authenticated (frictionless or challenge-completed with ECI value indicating liability shift, e.g., ECI 05/02) rather than attempted-only authentication, since under 10.4 a successful 3DS authentication generally shifts fraud liability to the issuer and is the single most decisive piece of evidence to compile and submit before the 12-day deadline."
}
A schema guarantees structure, not correctness. Therefore, we should also set up additional validation:
Our ProbabilityRange
already rejects a low_pct
of 900
(schema validation), but it happily accepts low_pct: 90
with high_pct: 10
which describes an impossible range. Pydantic (the library we used above) lets you keep business rules close to your schema. For example, a ProbabilityRange
model can enforce that percentages are between 0 and 100 and that low_pct
is never greater than high_pct
:
from pydantic import BaseModel, Field, model_validator
class ProbabilityRange(BaseModel):
low_pct: int = Field(ge=0, le=100)
high_pct: int = Field(ge=0, le=100)
@model_validator(mode="after")
def validate_range(self):
if self.low_pct > self.high_pct:
raise ValueError("low_pct must be <= high_pct")
return self
The goal is not to make the model perfectly reliable. The goal is to build a system where unreliable text generation is surrounded by reliable software boundaries. By harnessing this non-deterministic force, we can build tooling around it.
The next articles build on the same schema-constrained output to create tools for calculations, knowledge retrieval, and even accessing the internet.