{"slug": "python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation", "title": "Python Automation Cookbook, Part 2: Using AI to make decisions in your automation pipelines", "summary": "A developer shared a pattern for integrating AI decision-making into Python automation pipelines, using a small AI prompt to classify files before routing them to deterministic scripts. The approach adds a triage step that classifies files as PROCESS, REVIEW, or SKIP, enabling pipelines to handle edge cases without replacing existing automation scripts.", "body_md": "[Part 1](https://dev.to/peytongreen_dev/python-automation-cookbook-part-1-the-25-scripts-i-reach-for-every-week-4fba) covered the 25 scripts I reach for every week — reliable, production-ready automation building blocks. This part covers the part that takes longer to figure out: how to chain them together when the control flow involves a decision.\n\nPure script-chaining is easy. Run A, pipe output to B, done. But real automation pipelines hit decision points: *should this file be processed or skipped? Is this API response an error, a retry, or a success? Does this content need review or can it proceed?*\n\nHistorically you handled this with `if/elif`\n\ntrees. This works fine when the decision logic is clear and stable. It falls apart when the decision requires judgment — parsing ambiguous output, classifying content that doesn't fit neat categories, handling the long tail of edge cases that `if`\n\nstatements don't cover.\n\nThe pattern I've landed on: use the scripts in the Cookbook for the deterministic parts of the pipeline (HTTP calls, file I/O, scheduling, retries), and use a small AI prompt for the judgment calls. The two fit together cleanly because neither is trying to do the other's job.\n\nHere's a three-script pipeline before and after adding an AI decision layer.\n\n**Before — pure script chaining:**\n\n```\n# Download files\npython http_client.py --url $API_ENDPOINT --output raw/\n\n# Process each file\nfor f in raw/*.json; do\n    python csv_converter.py --input $f --output processed/\ndone\n\n# Archive\npython file_archiver.py --source processed/ --destination archive/\n```\n\nThis works if every file from the API is well-formed JSON that maps cleanly to CSV. In practice: some files have malformed encoding, some have unexpected schema changes, some are empty, some are error responses that look like successful ones. The script chain fails silently on these, or fails loudly and stops the whole pipeline.\n\n**After — with an AI triage step:**\n\n```\n# Download files\npython http_client.py --url $API_ENDPOINT --output raw/\n\n# AI triage: classify each file before processing\npython prompt_runner.py \\\n  --prompt \"classify_file_quality\" \\\n  --input-dir raw/ \\\n  --output triage_report.json\n\n# Route based on triage result\npython pipeline_router.py \\\n  --triage triage_report.json \\\n  --good-dest processing/ \\\n  --bad-dest review/ \\\n  --skip-dest archive/\n\n# Process clean files only\nfor f in processing/*.json; do\n    python csv_converter.py --input $f --output processed/\ndone\n```\n\nThe AI triage step doesn't replace the Cookbook scripts — it routes between them. That's the pattern.\n\nThe triage prompt is small. Here's the actual prompt I use for file quality classification:\n\n```\nCLASSIFY_FILE_QUALITY = \"\"\"You are a data quality classifier. Given a JSON file's contents, classify it as one of:\n- PROCESS: well-formed, expected schema, ready for conversion\n- REVIEW: unusual structure, possible schema change, needs human look\n- SKIP: empty, error response, or duplicate of a file already processed\n\nRespond with exactly one word: PROCESS, REVIEW, or SKIP.\nDo not explain. Do not add caveats.\n\nFile contents:\n{file_contents}\n\"\"\"\n```\n\nThree outputs. No explanation. The AI is not being asked to think — it's being asked to classify. This is the right scope for an in-pipeline AI call.\n\nThe key constraints:\n\nThe pattern generalizes. You can use the same shape for:\n\nThe Cookbook includes `prompt_runner.py`\n\n— a lightweight wrapper that handles the boilerplate for in-pipeline AI calls: API client initialization, token budgeting, error handling, output parsing, and retry logic.\n\nHere's how it works:\n\n```\n# Usage:\npython prompt_runner.py \\\n  --prompt classify_file_quality \\\n  --input-file raw/data_2026_03_23.json \\\n  --output-file triage/data_2026_03_23.triage\n\n# Batch mode (processes all files in a directory):\npython prompt_runner.py \\\n  --prompt classify_file_quality \\\n  --input-dir raw/ \\\n  --output-dir triage/ \\\n  --workers 4\n```\n\nThe `--prompt`\n\nflag takes a prompt name from `prompts/`\n\ndirectory (where you store your project's prompt templates). It loads the template, substitutes the input content, calls the API, and writes structured output to the output file.\n\nThe batch mode runs workers in parallel (default 4) with automatic backoff on rate limit errors. For a 100-file batch, it typically runs in 30-60 seconds depending on file size and API response time.\n\nThe pattern I use for deciding whether a decision goes in code or in a prompt:\n\n**Write code if:**\n\n`status == 200`\n\n)**Use a prompt if:**\n\nThe boundary I've found in practice: code handles the deterministic path, prompts handle the long tail. A well-designed pipeline has both.\n\nThe most useful pipeline I've built using this pattern: automated triage of incoming GitHub PR review requests.\n\n**The problem:** I work on a codebase that gets 15-30 PRs per day. Not all of them need deep review — some are docs updates, typo fixes, or dependency bumps. But identifying which ones can be auto-approved vs. which ones need attention requires reading the diff.\n\n**The scripts involved:**\n\n`webhook_receiver.py`\n\n— receives GitHub webhook, writes PR metadata + diff to a queue directory`prompt_runner.py`\n\n— classifies each PR using the triage prompt`pipeline_router.py`\n\n— routes based on classification`notification_sender.py`\n\n— sends Slack notification with routing decision**The triage prompt:**\n\n```\nTRIAGE_PR = \"\"\"You are a senior developer reviewing pull requests for triage priority.\n\nGiven a pull request diff and metadata, classify it as:\n- AUTO: safe to auto-approve (docs, formatting, dependency updates with no API changes, comment additions)\n- REVIEW: needs human review (logic changes, new features, API changes, security-sensitive code)\n- URGENT: needs immediate review (rollback, hotfix, security patch)\n\nRules:\n- If you can't determine the scope from the diff, classify as REVIEW\n- If any test files are modified, classify as at least REVIEW\n- If any auth, security, or credential-handling code is touched, classify as URGENT\n\nRespond with exactly one word: AUTO, REVIEW, or URGENT.\n\nPR title: {title}\nAuthor: {author}\nFiles changed: {file_count}\nDiff summary:\n{diff}\n\"\"\"\n```\n\n**The routing logic:**\n\n```\n# pipeline_router.py takes the triage output and acts on it\nAUTO   → add \"auto-approved\" label, merge if checks pass\nREVIEW → add to review queue, notify team on next batch\nURGENT → page on-call immediately via notification_sender.py\n```\n\n**What it does:** Reduces manual triage from 20-30 minutes per day to 2-3 minutes. AUTO classifications are almost always right (>95%). URGENT classifications have had 0 false negatives in 6 months — it's conservative on purpose.\n\nRunning AI calls in a pipeline has a cost. The math that made this viable for me:\n\nFor most automation pipelines, the cost of AI triage is dominated by the time cost of writing and maintaining the `if/elif`\n\ntree it replaces — which needs to be updated every time a new edge case appears.\n\nThe break-even is roughly: if you'd spend more than 10 minutes writing code to handle the edge case, the API call is cheaper.\n\nThe scripts most useful as AI pipeline components:\n\n| Script | Role in AI pipeline |\n|---|---|\n`http_client.py` |\nFetch data for AI processing; handles retries on API rate limits |\n`file_watcher.py` |\nTrigger AI triage when new files appear |\n`webhook_receiver.py` |\nAccept external events that feed into triage |\n`queue_processor.py` |\nProcess AI-classified items from a queue |\n`notification_sender.py` |\nSend routing decisions and escalations |\n`retry_runner.py` |\nWrap AI calls themselves — handles transient API errors |\n`cron_wrapper.py` |\nSchedule batch AI processing jobs |\n\nThe pipeline pattern:\n\n`http_client.py`\n\nor `webhook_receiver.py`\n\nor `file_watcher.py`\n\n`prompt_runner.py`\n\nwith your classification prompt`pipeline_router.py`\n\nbased on triage output`notification_sender.py`\n\non completion or escalationYou don't always need all five stages. The triage step is what makes it AI-in-the-loop instead of pure script chaining.\n\nIf you have a pipeline that currently uses manual classification or a brittle `if`\n\ntree, here's the migration path:\n\n**Identify the judgment call.** Where in the pipeline does a human or messy code currently make a decision that requires reading unstructured input?\n\n**Define the output enum.** What are the possible classifications? 2-4 choices is ideal. More than 6 and the prompt gets unreliable.\n\n**Write the prompt.** Follow the pattern: role assignment → enumerated choices with definitions → rules for edge cases → required output format (one word). Test it on 10-20 representative examples before deploying.\n\n**Add prompt_runner.py to the pipeline.** Replace the judgment call with a script call. Log outputs for the first week to validate accuracy.\n\n**Tune the rules.** After seeing real output, refine the rules section of the prompt to handle edge cases you didn't anticipate. This is faster than updating code.\n\nThe prompts above are starting points — designed to be modified for your pipeline's specific enum and rules. The general shape is always the same: role assignment → enumerated choices with definitions → rules for edge cases → required one-word output format.\n\nOnce you have a working triage prompt, the marginal cost of adding classification steps to your pipeline is low. Each new decision point is another prompt + another routing branch in `pipeline_router.py`\n\n.\n\nOne thing I keep running into as I expand these pipelines into production services: the pricing model matters as much as the architecture. Pipelines that run on a per-request basis have fundamentally different cost structures than pipelines that run on a per-user seat. If you're building toward a paid product, worth thinking about early.\n\nI wrote a full breakdown of the six pricing models that work for AI agent products — including worked examples, a cost calculator, and the decision framework I actually use:\n\n[Pricing Your Python AI Agent SaaS in 2026 →](https://kazdispatch.gumroad.com/l/ijwtd)\n\n*Six pricing models, Notion cost calculator, worked examples. $39 one-time.*\n\nAnd if you want Part 3 (failure modes, output validation, human-in-the-loop checkpoints) in your inbox when it drops — the list is here:\n\nPart 3 is about failure modes — specifically, what happens when an AI step in your pipeline produces the wrong classification and you don't catch it. It covers: output validation, human-in-the-loop checkpoints for high-stakes decisions, logging patterns for pipeline observability, and how to build a correction loop that improves prompt accuracy over time.\n\nIf you're building AI-in-the-loop pipelines right now, leave a comment with what you're working on — I'm actively building from production use cases, and reader questions drive what goes in Part 3. *(#discuss)*", "url": "https://wpnews.pro/news/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation", "canonical_source": "https://dev.to/peytongreen_dev/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation-pipelines-3o3j", "published_at": "2026-07-10 19:18:48+00:00", "updated_at": "2026-07-10 19:43:11.339481+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "machine-learning", "ai-tools"], "entities": ["Python Automation Cookbook", "prompt_runner.py", "pipeline_router.py", "http_client.py", "csv_converter.py", "file_archiver.py"], "alternates": {"html": "https://wpnews.pro/news/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation", "markdown": "https://wpnews.pro/news/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation.md", "text": "https://wpnews.pro/news/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation.txt", "jsonld": "https://wpnews.pro/news/python-automation-cookbook-part-2-using-ai-to-make-decisions-in-your-automation.jsonld"}}