{"slug": "5-second-ad-review-with-strict-pydantic-contracts", "title": "5‑Second Ad Review with Strict Pydantic Contracts", "summary": "A developer built CreativeAudit, a local LLM pipeline using Ollama and Pydantic v2 to enforce strict JSON output contracts for 5-second ad reviews, scoring creatives on brand alignment, constraint compliance, and message clarity with a weighted total formula. The system runs fully on-premise to prevent client data leaks, uses Jinja2 prompt templates as versioned code, and includes 38 deterministic unit tests. It aims to replace manual ad review with a fast, deterministic process that respects data privacy.", "body_md": "# 5‑Second Ad Review with Strict Pydantic Contracts\n\nA plain chat with an LLM falls apart for this task: the output jumps formats, the model forgets constraints, you get free‑form text, and client data leaks to the cloud. For any real workflow that’s a deal‑breaker.\n\n**Design goals I forced myself to meet**\n\n1. **Strict output contracts** – the schema is a hard wall; if the LLM doesn’t obey it, the call fails, no fake scores appear.\n\n2. **Local inference** – everything runs on‑premise with Ollama, so no data ever leaves the machine.\n\n3. **Prompts as code** – Jinja2 files live in a versioned folder, making them editable without touching the app.\n\n4. **Binary compliance** for any brand‑book violation; a 0 or 10 score removes subjectivity on the highest‑risk rule.\n\n5. **Human‑in‑the‑loop** mode that first extracts structured data from messy input, shows it to the user for correction, then runs the evaluation.\n\n**Architecture at a glance**\n\n```\nBrief + Creatives → Jinja2 Prompt → Local LLM (Ollama)  \n↓  \nPydantic Validation  \n↓  \nScore + Verdict + Feedback\n```\n\nKey components:\n\n`app/schemas.py`\n\n– strict Pydantic v2 contracts that define the exact JSON shape.`prompts/*.j2`\n\n– versioned prompt templates, treated like source code.`app/main.py`\n\n– orchestrates prompt assembly, LLM call, validation, scoring.`demo/streamlit_app.py`\n\n– thin UI layer for quick testing.\n\n**Engineering decisions that matter**\n\n**Pydantic as the contract**– the model is inherently nondeterministic; treating the schema as absolute guarantees that malformed responses never masquerade as scores.**Binary 0/10 for critical rules**– a creative either fully complies with the brand book or it doesn’t; no “slightly violated” ambiguity.** Jinja2 prompts**– because they’re separate files, you can A/B test wording, review diffs, and keep the app code clean.** Local Ollama inference**– swapping between qwen2.5:7b, qwen2.5:14b or llama3.2 is a one‑line config change, keeping deployment lightweight.** Smart input mode**– managers often paste chat fragments; an initial LLM step normalizes that into structured JSON, which the user can edit before the final audit runs.\n\n**Scoring model**\n\nEach creative gets three numeric scores: brand_alignment (1‑10), constraint_compliance (0‑10), and message_clarity (1‑10). The total combines them with weights:\n\ntotal = brand × 0.4 + compliance × 0.3 + clarity × 0.3\n\nVerdicts are then derived from the total and any critical failures flagged by the binary check.\n\n**Testing strategy**\n\nI wrote 38 deterministic unit tests, mocking the external LLM so they run in under two seconds. Tests cover schema edge cases, malformed responses, connection hiccups, and the scoring logic, giving a reliable **complete guide** for anyone who wants to replicate the pipeline from scratch.\n\nThe whole system lives in a **practical tutorial**‑style repo, with clear **step‑by‑step** instructions for deployment, making it beginner‑friendly while still being a **deep dive** into real‑world LLM agent workflow for prompt engineering.\n\n```\nYou are an ad compliance analyst. Given a creative description and a brand brief, output ONLY a JSON object that matches the schema below. Do not add any extra text.\n{\n  \"verdict\": \"PASS|NEEDS_REVISION|FAIL\",\n  \"brand_alignment\": 1-10,\n  \"constraint_compliance\": 0-10,\n  \"message_clarity\": 1-10,\n  \"feedback\": \"short sentence explaining why\"\n}\n```\n\nCreativeAudit shows that a **reliable LLM pipeline** can replace hours of manual review with a fast, deterministic process that respects data privacy and enforces strict contracts. If you’re building any AI workflow that needs guaranteed output, this **step‑by‑step** setup is worth a look.\n\n[Next Stop trusting raw benchmark scores without looking at the harness →](/en/threads/7530/)", "url": "https://wpnews.pro/news/5-second-ad-review-with-strict-pydantic-contracts", "canonical_source": "https://promptcube3.com/en/threads/7674/", "published_at": "2026-08-25 16:45:45+00:00", "updated_at": "2026-08-25 17:13:38.716709+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Ollama", "Pydantic", "Jinja2", "CreativeAudit", "qwen2.5:7b", "qwen2.5:14b", "llama3.2"], "alternates": {"html": "https://wpnews.pro/news/5-second-ad-review-with-strict-pydantic-contracts", "markdown": "https://wpnews.pro/news/5-second-ad-review-with-strict-pydantic-contracts.md", "text": "https://wpnews.pro/news/5-second-ad-review-with-strict-pydantic-contracts.txt", "jsonld": "https://wpnews.pro/news/5-second-ad-review-with-strict-pydantic-contracts.jsonld"}}