{"slug": "python-openai-structured-json-tutorial", "title": "Python OpenAI Structured JSON Tutorial", "summary": "Gate of AI published a tutorial on building a Python command-line tool that uses OpenAI's chat model for structured JSON extraction from travel journals, with Pydantic validation. The tool, named travel_journal_analyzer, loads text, Markdown, or CSV files, extracts cities, restaurants, ratings, and sentiment, and writes validated JSON reports. The tutorial emphasizes combining JSON schema with local validation to ensure reliable, typed output for downstream applications.", "body_md": "🚀 Technical Briefing:This tutorial is part of our deep-dive series on Agentic Workflows at[Gate of AI]. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the[original article here].\n\nTutorial\n\nBuild a Python command-line tool that loads travel-journal files, requests JSON-shaped extraction from an OpenAI chat model, validates every response with Pydantic, and writes a reusable report.\n\nThis tutorial builds `travel_journal_analyzer`\n\n, a small but production-minded Python application. It accepts a text, Markdown, CSV file, or a directory containing those formats. It converts each source into a consistent `JournalEntry`\n\n, sends the entry to an OpenAI chat model, validates the returned JSON locally, and writes one JSON report for downstream software.\n\nThe report has a deliberately narrow purpose: identify cities mentioned in an entry, explicitly named restaurants, dishes, ratings where stated, sentiment, practical tips, and a concise summary. The application does not treat a valid JSON shape as proof that a claim is true. Its prompt instructs the model to extract facts only from the supplied entry, while local validation checks the data contract before results are exported.\n\nReliable structured extraction combines two boundaries. The first is the requested JSON schema: it defines the fields an application expects. The second is local validation with Pydantic: it rejects malformed values, such as a rating outside a five-point range. Research on structured generation describes the importance of reliable, typed output for applications that need predictable data rather than unconstrained prose. In practical Python work, these boundaries make testing and maintenance substantially easier.\n\nCreate a project and isolated virtual environment:\n\n```\nmkdir travel-journal-analyzer\ncd travel-journal-analyzer\npython -m venv .venv\n\n# macOS or Linux\nsource .venv/bin/activate\n\n# Windows PowerShell\n# .venv\\Scripts\\Activate.ps1\n\npython -m pip install --upgrade pip\npython -m pip install \"openai>=1.0.0\" \"pydantic>=2.7.0\" \"python-dotenv>=1.0.1\" \"pytest>=8.0.0\"\nmkdir data output tests\n```\n\nCreate `.env`\n\n. Keep this file out of source control. API keys belong in environment variables locally and in a deployment secret manager in production.\n\n```\nOPENAI_API_KEY=your-api-key\nOPENAI_MODEL=your-chat-model\nMAX_ENTRY_CHARACTERS=12000\nREQUEST_TIMEOUT_SECONDS=45\n```\n\nCreate `.gitignore`\n\n:\n\n```\n.env\n.venv/\n__pycache__/\n.pytest_cache/\noutput/\n*.pyc\n```\n\nThis tutorial uses the modern client-based OpenAI Python pattern: `from openai import OpenAI`\n\n, then `client.chat.completions.create(...)`\n\n. Do not use legacy module-level completion calls.\n\nCreate `models.py`\n\n. Nullable fields represent facts that the source did not provide. That is preferable to inventing a city, cuisine, or rating.\n\n``` python\nfrom __future__ import annotations\n\nfrom typing import Literal\nfrom pydantic import BaseModel, Field, field_validator\n\nclass JournalEntry(BaseModel):\n    source_name: str = Field(min_length=1, max_length=255)\n    entry_id: str = Field(min_length=1, max_length=100)\n    text: str = Field(min_length=1)\n\n    @field_validator(\"text\")\n    @classmethod\n    def validate_text(cls, value: str) -> str:\n        cleaned = value.strip()\n        if not cleaned:\n            raise ValueError(\"Journal entry text cannot be blank.\")\n        return cleaned\n\nclass RestaurantFinding(BaseModel):\n    name: str = Field(min_length=1, max_length=200)\n    city: str | None = Field(default=None, max_length=120)\n    country: str | None = Field(default=None, max_length=120)\n    cuisine: str | None = Field(default=None, max_length=120)\n    dishes: list[str] = Field(default_factory=list)\n    rating_out_of_five: float | None = Field(default=None, ge=0, le=5)\n    sentiment: Literal[\"positive\", \"neutral\", \"negative\"]\n    recommendation_reason: str = Field(min_length=1, max_length=600)\n\nclass JournalAnalysis(BaseModel):\n    entry_id: str = Field(min_length=1, max_length=100)\n    cities_mentioned: list[str] = Field(default_factory=list)\n    restaurants: list[RestaurantFinding] = Field(default_factory=list)\n    travel_tips: list[str] = Field(default_factory=list)\n    concise_summary: str = Field(min_length=1, max_length=1000)\n\nclass AnalysisReport(BaseModel):\n    generated_at_utc: str\n    model: str\n    total_entries: int = Field(ge=0)\n    successful_analyses: int = Field(ge=0)\n    failed_entries: list[str] = Field(default_factory=list)\n    analyses: list[JournalAnalysis] = Field(default_factory=list)\n```\n\nThe models are not merely documentation. `JournalAnalysis.model_validate_json()`\n\nturns the model response into a validated object. A response with an invalid sentiment label or a six-point rating fails before it reaches a database, spreadsheet, or customer-facing interface.\n\nCreate `journal_loader.py`\n\n. Plain-text and Markdown files create one entry each. A CSV file requires a `text`\n\ncolumn and creates one entry for every non-empty row. The character limit prevents unexpectedly large requests.\n\n``` python\nfrom __future__ import annotations\n\nimport csv\nfrom pathlib import Path\nfrom models import JournalEntry\n\nSUPPORTED_SUFFIXES = {\".txt\", \".md\", \".csv\"}\n\ndef load_journal_entries(path_value: str, max_characters: int) -> list[JournalEntry]:\n    path = Path(path_value).expanduser().resolve()\n    if not path.exists():\n        raise FileNotFoundError(f\"Input path does not exist: {path}\")\n\n    if path.is_dir():\n        entries: list[JournalEntry] = []\n        for child in sorted(path.iterdir()):\n            if child.is_file() and child.suffix.lower() in SUPPORTED_SUFFIXES:\n                entries.extend(load_journal_entries(str(child), max_characters))\n        if not entries:\n            raise ValueError(\"Directory contains no supported input files.\")\n        return entries\n\n    if path.suffix.lower() in {\".txt\", \".md\"}:\n        text = path.read_text(encoding=\"utf-8\").strip()\n        _check_length(text, path.name, max_characters)\n        return [JournalEntry(source_name=path.name, entry_id=path.stem, text=text)]\n\n    if path.suffix.lower() != \".csv\":\n        raise ValueError(\"Use a .txt, .md, .csv file, or directory.\")\n\n    entries = []\n    with path.open(\"r\", encoding=\"utf-8-sig\", newline=\"\") as handle:\n        reader = csv.DictReader(handle)\n        if not reader.fieldnames or \"text\" not in reader.fieldnames:\n            raise ValueError(\"CSV must contain a column named 'text'.\")\n        for row_number, row in enumerate(reader, start=2):\n            text = (row.get(\"text\") or \"\").strip()\n            if not text:\n                continue\n            _check_length(text, f\"{path.name} row {row_number}\", max_characters)\n            entries.append(JournalEntry(\n                source_name=(row.get(\"source_name\") or path.name).strip(),\n                entry_id=(row.get(\"entry_id\") or f\"{path.stem}-{row_number}\").strip(),\n                text=text,\n            ))\n    if not entries:\n        raise ValueError(\"CSV contains no non-empty text rows.\")\n    return entries\n\ndef _check_length(text: str, label: str, maximum: int) -> None:\n    if not text:\n        raise ValueError(f\"{label} is empty.\")\n    if len(text) > maximum:\n        raise ValueError(f\"{label} exceeds the {maximum}-character limit.\")\n```\n\nSave this sample as `data/journal.txt`\n\n:\n\n```\nWe arrived in Lisbon on a rainy Thursday.\nFor dinner, we booked Taberna da Rua das Flores. The grilled octopus was tender,\nand I would rate the experience 4.5 out of 5. Reserve ahead because it is small.\n\nTwo days later in Porto, Cafe Santiago served a memorable francesinha.\nThe sandwich was rich but the service was slow, so I felt neutral overall.\n```\n\nCreate `analyzer.py`\n\n. The schema below is passed as a requested response format. Your configured model must support the selected response-format capability; verify that support in the current OpenAI documentation for the model available to your account before deploying. Regardless of the upstream response constraint, the code performs local Pydantic validation.\n\n``` python\nfrom __future__ import annotations\n\nfrom openai import OpenAI\nfrom pydantic import ValidationError\n\nfrom models import JournalAnalysis, JournalEntry\n\nSCHEMA = {\n    \"name\": \"travel_journal_analysis\",\n    \"strict\": True,\n    \"schema\": {\n        \"type\": \"object\",\n        \"additionalProperties\": False,\n        \"properties\": {\n            \"entry_id\": {\"type\": \"string\"},\n            \"cities_mentioned\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\n            \"restaurants\": {\"type\": \"array\", \"items\": {\"type\": \"object\", \"additionalProperties\": False, \"properties\": {\n                \"name\": {\"type\": \"string\"}, \"city\": {\"type\": [\"string\", \"null\"]},\n                \"country\": {\"type\": [\"string\", \"null\"]}, \"cuisine\": {\"type\": [\"string\", \"null\"]},\n                \"dishes\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\n                \"rating_out_of_five\": {\"type\": [\"number\", \"null\"]},\n                \"sentiment\": {\"type\": \"string\", \"enum\": [\"positive\", \"neutral\", \"negative\"]},\n                \"recommendation_reason\": {\"type\": \"string\"}\n            }, \"required\": [\"name\", \"city\", \"country\", \"cuisine\", \"dishes\", \"rating_out_of_five\", \"sentiment\", \"recommendation_reason\"]}},\n            \"travel_tips\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\n            \"concise_summary\": {\"type\": \"string\"}\n        },\n        \"required\": [\"entry_id\", \"cities_mentioned\", \"restaurants\", \"travel_tips\", \"concise_summary\"]\n    }\n}\n\nSYSTEM_PROMPT = \"\"\"Extract facts only from the journal entry.\nInclude restaurants only when explicitly named. Never invent facts.\nUse null for unknown scalar values. Return only schema-conforming JSON.\"\"\"\n\nclass JournalAnalyzer:\n    def __init__(self, api_key: str, model: str, timeout: float) -> None:\n        self.model = model\n        self.client = OpenAI(api_key=api_key, timeout=timeout, max_retries=0)\n\n    def analyze(self, entry: JournalEntry) -> JournalAnalysis:\n        completion = self.client.chat.completions.create(\n            model=self.model,\n            temperature=0,\n            response_format={\"type\": \"json_schema\", \"json_schema\": SCHEMA},\n            messages=[\n                {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n                {\"role\": \"user\", \"content\": f\"Required entry_id: {entry.entry_id}\\n\\nJournal:\\n{entry.text}\"},\n            ],\n        )\n        content = completion.choices[0].message.content\n        if not content:\n            raise RuntimeError(\"The model returned an empty response.\")\n        try:\n            result = JournalAnalysis.model_validate_json(content)\n        except ValidationError as error:\n            raise RuntimeError(f\"Local schema validation failed: {error}\") from error\n        if result.entry_id != entry.entry_id:\n            raise RuntimeError(\"Response entry_id does not match the input entry.\")\n        return result\n```\n\nThe extraction policy is intentionally conservative. An unnamed kiosk may be discussed negatively in a journal, but it must not become an invented restaurant record. If unnamed venues matter to the product, add a distinct field and clearly communicate that it represents an unnamed mention, not an official venue identity.\n\n``` python\nfrom __future__ import annotations\n\nimport argparse\nimport os\nfrom datetime import datetime, timezone\nfrom pathlib import Path\nfrom dotenv import load_dotenv\n\nfrom analyzer import JournalAnalyzer\nfrom journal_loader import load_journal_entries\nfrom models import AnalysisReport\n\ndef main() -> int:\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"input_path\")\n    parser.add_argument(\"--output\", default=\"output/travel-analysis.json\")\n    args = parser.parse_args()\n\n    load_dotenv()\n    api_key = os.getenv(\"OPENAI_API_KEY\", \"\").strip()\n    model = os.getenv(\"OPENAI_MODEL\", \"\").strip()\n    if not api_key or not model:\n        raise RuntimeError(\"Set OPENAI_API_KEY and OPENAI_MODEL in .env or your shell.\")\n\n    maximum = int(os.getenv(\"MAX_ENTRY_CHARACTERS\", \"12000\"))\n    timeout = float(os.getenv(\"REQUEST_TIMEOUT_SECONDS\", \"45\"))\n    entries = load_journal_entries(args.input_path, maximum)\n    analyzer = JournalAnalyzer(api_key, model, timeout)\n\n    analyses = []\n    failed = []\n    for entry in entries:\n        try:\n            analyses.append(analyzer.analyze(entry))\n        except RuntimeError as error:\n            failed.append(entry.entry_id)\n            print(f\"Failed {entry.entry_id}: {error}\")\n\n    report = AnalysisReport(\n        generated_at_utc=datetime.now(timezone.utc).isoformat(),\n        model=model,\n        total_entries=len(entries),\n        successful_analyses=len(analyses),\n        failed_entries=failed,\n        analyses=analyses,\n    )\n    destination = Path(args.output)\n    destination.parent.mkdir(parents=True, exist_ok=True)\n    temporary = destination.with_suffix(destination.suffix + \".tmp\")\n    temporary.write_text(report.model_dump_json(indent=2), encoding=\"utf-8\")\n    temporary.replace(destination)\n    print(f\"Wrote {report.successful_analyses}/{report.total_entries} analyses to {destination}\")\n    return 1 if failed else 0\n\nif __name__ == \"__main__\":\n    raise SystemExit(main())\n```\n\nRun the tool and inspect the output:\n\n```\npython main.py data/journal.txt --output output/journal-report.json\npython -m json.tool output/journal-report.json\n```\n\nThe temporary-file replacement avoids leaving a partially written final report if the process stops during serialization. The process returns a non-zero status when one or more entries fail, which is useful in scheduled jobs and CI.\n\nTest deterministic components without calling an API. For example, verify that the loader rejects a CSV without a `text`\n\ncolumn, that blank rows are skipped, and that ratings over five fail Pydantic validation. Mock the OpenAI client for analyzer unit tests, then run a small intentional smoke test with a real credential.\n\n``` python\nfrom models import RestaurantFinding\nfrom pydantic import ValidationError\nimport pytest\n\ndef test_invalid_rating_is_rejected():\n    with pytest.raises(ValidationError):\n        RestaurantFinding(\n            name=\"Example\", sentiment=\"positive\",\n            rating_out_of_five=6,\n            recommendation_reason=\"Unsupported rating range.\",\n        )\n```\n\nSchema validity is not factual validity. Review representative outputs for named-entity grounding, rating fidelity, correct sentiment, and unsupported details. Maintain a permissioned evaluation set when changing prompts, model configuration, schemas, or SDK versions. Avoid logging raw journal content by default because travel notes can contain personal data.\n\nFor GCC and Middle East deployments, apply the same data-handling discipline to Arabic and English input, local privacy obligations, data residency requirements, and organizational retention rules. This tutorial does not make claims about a particular regional provider, initiative, or infrastructure arrangement; teams should validate those requirements with their legal, security, and platform stakeholders before production use.\n\nThe central lesson is simple: use typed contracts and controlled inputs to turn flexible model output into application data, then test both the software boundary and the factual usefulness of the result.", "url": "https://wpnews.pro/news/python-openai-structured-json-tutorial", "canonical_source": "https://dev.to/gateofai/python-openai-structured-json-tutorial-2k4a", "published_at": "2026-08-13 18:08:05+00:00", "updated_at": "2026-08-13 18:19:05.930776+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-products"], "entities": ["OpenAI", "Pydantic", "Gate of AI"], "alternates": {"html": "https://wpnews.pro/news/python-openai-structured-json-tutorial", "markdown": "https://wpnews.pro/news/python-openai-structured-json-tutorial.md", "text": "https://wpnews.pro/news/python-openai-structured-json-tutorial.txt", "jsonld": "https://wpnews.pro/news/python-openai-structured-json-tutorial.jsonld"}}