# 5‑Second Ad Review with Strict Pydantic Contracts

> Source: <https://promptcube3.com/en/threads/7674/>
> Published: 2026-08-25 16:45:45+00:00

# 5‑Second Ad Review with Strict Pydantic Contracts

A plain chat with an LLM falls apart for this task: the output jumps formats, the model forgets constraints, you get free‑form text, and client data leaks to the cloud. For any real workflow that’s a deal‑breaker.

**Design goals I forced myself to meet**

1. **Strict output contracts** – the schema is a hard wall; if the LLM doesn’t obey it, the call fails, no fake scores appear.

2. **Local inference** – everything runs on‑premise with Ollama, so no data ever leaves the machine.

3. **Prompts as code** – Jinja2 files live in a versioned folder, making them editable without touching the app.

4. **Binary compliance** for any brand‑book violation; a 0 or 10 score removes subjectivity on the highest‑risk rule.

5. **Human‑in‑the‑loop** mode that first extracts structured data from messy input, shows it to the user for correction, then runs the evaluation.

**Architecture at a glance**

```
Brief + Creatives → Jinja2 Prompt → Local LLM (Ollama)  
↓  
Pydantic Validation  
↓  
Score + Verdict + Feedback
```

Key components:

`app/schemas.py`

– strict Pydantic v2 contracts that define the exact JSON shape.`prompts/*.j2`

– versioned prompt templates, treated like source code.`app/main.py`

– orchestrates prompt assembly, LLM call, validation, scoring.`demo/streamlit_app.py`

– thin UI layer for quick testing.

**Engineering decisions that matter**

**Pydantic as the contract**– the model is inherently nondeterministic; treating the schema as absolute guarantees that malformed responses never masquerade as scores.**Binary 0/10 for critical rules**– a creative either fully complies with the brand book or it doesn’t; no “slightly violated” ambiguity.** Jinja2 prompts**– because they’re separate files, you can A/B test wording, review diffs, and keep the app code clean.** Local Ollama inference**– swapping between qwen2.5:7b, qwen2.5:14b or llama3.2 is a one‑line config change, keeping deployment lightweight.** Smart input mode**– managers often paste chat fragments; an initial LLM step normalizes that into structured JSON, which the user can edit before the final audit runs.

**Scoring model**

Each creative gets three numeric scores: brand_alignment (1‑10), constraint_compliance (0‑10), and message_clarity (1‑10). The total combines them with weights:

total = brand × 0.4 + compliance × 0.3 + clarity × 0.3

Verdicts are then derived from the total and any critical failures flagged by the binary check.

**Testing strategy**

I wrote 38 deterministic unit tests, mocking the external LLM so they run in under two seconds. Tests cover schema edge cases, malformed responses, connection hiccups, and the scoring logic, giving a reliable **complete guide** for anyone who wants to replicate the pipeline from scratch.

The whole system lives in a **practical tutorial**‑style repo, with clear **step‑by‑step** instructions for deployment, making it beginner‑friendly while still being a **deep dive** into real‑world LLM agent workflow for prompt engineering.

```
You are an ad compliance analyst. Given a creative description and a brand brief, output ONLY a JSON object that matches the schema below. Do not add any extra text.
{
  "verdict": "PASS|NEEDS_REVISION|FAIL",
  "brand_alignment": 1-10,
  "constraint_compliance": 0-10,
  "message_clarity": 1-10,
  "feedback": "short sentence explaining why"
}
```

CreativeAudit shows that a **reliable LLM pipeline** can replace hours of manual review with a fast, deterministic process that respects data privacy and enforces strict contracts. If you’re building any AI workflow that needs guaranteed output, this **step‑by‑step** setup is worth a look.

[Next Stop trusting raw benchmark scores without looking at the harness →](/en/threads/7530/)