{"slug": "validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks", "title": "Validate Your LLM Workflow Before It Runs (Python + YAML Topology Checks)", "summary": "A developer built a Python + YAML topology checker for LLM workflows that validates the graph structure before runtime, catching dead ends, missing steps, and illegal loops. The open-source package llm-workflow-router analyzes directed graphs of workflow steps and refuses to start if the topology is invalid, preventing runtime failures in production.", "body_md": "Your LLM app boots fine. It answers the first few messages fine. Then, ten minutes into a real conversation, it quietly walks from the `support`\n\nstep into the `billing`\n\nstep that doesn't exist anymore, or loops `faq → support → faq → support`\n\nuntil something times out. Nothing crashed at startup. The mistake was in the workflow's *shape* the whole time — you just didn't find out until runtime, in production, with a user watching.\n\nThis is the same class of bug as a malformed config file: the error exists the moment you write it, but the tool waits until the worst possible time to tell you. The fix is the same too. Catch it at load time, before a single step executes.\n\nIn this tutorial we'll build a small, declarative LLM workflow as YAML, then validate its **topology** — the graph of which step can hand off to which — before we run anything. We'll catch dead ends, transitions to steps that don't exist, and loops that can never terminate. All the code here runs against a real open-source package, [ llm-workflow-router](https://pypi.org/project/llm-workflow-router/), so you can follow along end to end.\n\nMost guardrails for LLM systems limit *volume*: a max number of tool calls, a timeout, a retry cap. Those are useful, but they treat the symptom. A runaway loop gets cut off after N iterations — it doesn't get prevented, and you learn nothing about *why* the workflow could loop in the first place.\n\nStructural validation asks a different question: **before this workflow ever handles a request, is its graph even valid?** A step that transitions to a nonexistent target is invalid no matter what any model says. A step with no way out is invalid. A cycle through a step that isn't allowed to be re-entered is invalid. None of that depends on content, so none of it needs to wait for runtime to be caught.\n\nThe mental model: your workflow is a directed graph. Nodes are steps (\"containers\"). Edges are the transitions each step permits. Validating the workflow means analyzing that graph and refusing to start if it's broken.\n\nHere's a minimal support flow. A user enters at `support_entry`\n\n, which may hand off to `faq`\n\nor refuse; `faq`\n\nmay only refuse. `REFUSE`\n\nis a terminal state, not a step.\n\n```\n# good_config.yaml\nentrypoints:\n  - support_entry\n\ncontainers:\n  support_entry:\n    allow_transitions:\n      - faq\n      - REFUSE\n    allow_reentry: false\n    max_invocations: 3\n\n  faq:\n    allow_transitions:\n      - REFUSE\n    allow_reentry: true\n    max_invocations: 5\n```\n\nThree things are being declared per step, and each one is a rule the validator can check:\n\n`allow_transitions`\n\n— the `allow_reentry`\n\n— whether the step may be entered more than once. This is what turns \"is there a cycle?\" into \"is there a cycle that's actually a problem?\"`max_invocations`\n\n— a hard ceiling on how many times the step may run. Must be greater than zero, or the step can never execute.`entrypoints`\n\nnames the authoritative roots of the graph. Declaring them explicitly means the validator doesn't have to *guess* where the workflow starts — and it can flag any step that no path can reach.\n\nInstall the package:\n\n```\npip install llm-workflow-router\n```\n\nThen validate the config before you wire it into anything:\n\n```\npython -m router.cli validate good_config.yaml\n{\n  \"ok\": true\n}\n```\n\nClean exit, exit code `0`\n\n. Now let's break it on purpose — the kind of breakage that's easy to introduce and impossible to spot by eye once a config gets large:\n\n```\n# broken_config.yaml\ncontainers:\n  support_entry:\n    allow_transitions:\n      - faq\n      - billing        # typo: this step does not exist\n    allow_reentry: false\n    max_invocations: 3\n\n  faq:\n    allow_transitions:\n      - support_entry   # loops back into a no-reentry step\n    allow_reentry: true\n    max_invocations: 5\n\n  orphaned_step:\n    allow_transitions:\n      - REFUSE\n    allow_reentry: false\n    max_invocations: 1\n```\n\nThere are three separate problems hiding in there. Run `validate`\n\nand it stops you at the first hard error:\n\n```\npython -m router.cli validate broken_config.yaml\n{\n  \"ok\": false,\n  \"error\": \"CONFIG_VALIDATION_ERROR\",\n  \"message\": \"Container 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}.\"\n}\n```\n\nExit code `2`\n\n. Notice what the error message actually does: it names the offending step (`support_entry`\n\n), the bad value (`billing`\n\n), and the *set of things that would have been valid*. That last part is the difference between an error that stops you and an error that helps you. More on that below.\n\n`validate`\n\nis pass/fail — good for CI, where you want a nonzero exit and the first blocking reason. But when you're actually fixing a workflow, you want the whole picture, not one error at a time. That's what `analyze`\n\nis for: it inspects the topology and reports everything, graded by severity, without raising.\n\n```\npython -m router.cli analyze broken_config.yaml\n{\n  \"container_count\": 3,\n  \"entry_containers\": [\n    \"orphaned_step\"\n  ],\n  \"entrypoints_declared\": false,\n  \"cycle_count\": 1,\n  \"cycles\": [\n    [\"faq\", \"support_entry\"]\n  ],\n  \"issues\": [\n    {\n      \"severity\": \"ERROR\",\n      \"code\": \"CYCLE_WITH_REENTRY_BLOCKED\",\n      \"message\": \"Cycle ['faq', 'support_entry'] includes containers with allow_reentry=false: ['support_entry']\"\n    },\n    {\n      \"severity\": \"ERROR\",\n      \"code\": \"UNKNOWN_TARGET\",\n      \"message\": \"Container 'support_entry' transitions to unknown 'billing'.\"\n    },\n    {\n      \"severity\": \"WARNING\",\n      \"code\": \"UNREACHABLE_CONTAINERS\",\n      \"message\": \"Unreachable containers: ['faq', 'support_entry']\"\n    }\n  ]\n}\n```\n\nThis one report catches all three bugs I planted:\n\n`UNKNOWN_TARGET`\n\n`billing`\n\ntypo. A transition to a step that isn't in the graph.`CYCLE_WITH_REENTRY_BLOCKED`\n\n`faq`\n\ntransitions back to `support_entry`\n\n, forming a loop `faq → support_entry → faq`\n\n, but `support_entry`\n\ndeclared `allow_reentry: false`\n\n. The workflow contradicts itself: the topology requires re-entering a step the config forbids re-entering.`entrypoints`\n\nblock, the analyzer had to `orphaned_step`\n\n, so it wrongly became the \"entry.\" Everything else is now unreachable from that inferred root, hence the `UNREACHABLE_CONTAINERS`\n\nwarning. This is a great argument for always declaring `entrypoints`\n\nexplicitly: it removes the guessing.The severity split matters. An `ERROR`\n\nis a contradiction that makes the workflow structurally invalid — it blocks. A `WARNING`\n\nis a smell worth surfacing but not necessarily fatal. Cycles are a nice example of the distinction: a loop through steps that *all* permit re-entry is a `WARNING`\n\n(it can legitimately terminate via `max_invocations`\n\n), while a loop through a step that forbids re-entry is an `ERROR`\n\n(it can't).\n\nThe CLI is a thin wrapper. If you're building the workflow programmatically — or want validation inside your own app's startup path — the same checks are one function call.\n\n``` python\nfrom router.engine import WorkflowEngine\nfrom router.models.config import ContainerConfig, WorkflowConfig\nfrom router.models.enums import InteractionState\nfrom router.models.metadata import InteractionMetadata\nfrom router.validation.config_validator import validate_workflow_config\n\ncfg = WorkflowConfig(\n    containers={\n        \"support_entry\": ContainerConfig(\n            allow_transitions=[\"faq\", \"REFUSE\"],\n            allow_reentry=False,\n            max_invocations=3,\n        ),\n        \"faq\": ContainerConfig(\n            allow_transitions=[\"REFUSE\"],\n            allow_reentry=True,\n            max_invocations=5,\n        ),\n    },\n    entrypoints=[\"support_entry\"],\n)\n\nvalidate_workflow_config(cfg)   # raises ConfigValidationError if the graph is invalid\nprint(\"Config valid ✓\")\n```\n\n`validate_workflow_config`\n\nraises `ConfigValidationError`\n\non any hard error, so the natural place to call it is at load time — the workflow simply refuses to start if its shape is wrong. That's the whole idea: an invalid state is made unrepresentable at runtime because it was rejected before runtime began.\n\nOnce the config is validated, evaluating a step is deterministic — the same metadata always yields the same decision:\n\n```\nengine = WorkflowEngine(cfg)\n\nmetadata = InteractionMetadata(\n    container=\"support_entry\",\n    previous_state=InteractionState.PROCEED,\n    transition_history=[],\n    invocation_depth={\"support_entry\": 1},\n    requested_action=\"faq\",\n    trace_id=\"demo-001\",\n)\n\nprint(engine.evaluate(metadata))\nEvaluationResult(state=<InteractionState.PROCEED: 'PROCEED'>, container='support_entry', reason=None, trace_id='demo-001')\n```\n\nThe engine checks the request against the *validated* rules: is `support_entry`\n\na real step? Under its invocation ceiling? Is `faq`\n\nin its `allow_transitions`\n\n? All yes, so the result is `PROCEED`\n\n. Ask for a transition that isn't allowed and you'd get a `REFUSE`\n\nwith a machine-readable `reason`\n\nlike `INVALID_TRANSITION`\n\n— no exceptions thrown mid-flight, just an explicit, inspectable decision.\n\nGo back and look at that first failure message:\n\nContainer 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}.\n\nIt would have been *easier* to write `ValueError: invalid transition`\n\n. It would also have been useless. A good validation error answers three questions without making the reader go digging: **where** is the problem (`support_entry`\n\n), **what** is wrong (`billing`\n\nisn't a valid target), and **what would be right** (a real step name, or a terminal state). The `analyze`\n\ncodes do the same job for tooling — `UNKNOWN_TARGET`\n\n, `CYCLE_WITH_REENTRY_BLOCKED`\n\n, `ORPHAN_CONTAINER`\n\nare greppable, stable identifiers you can build dashboards or CI rules around.\n\nIf you take one habit away from this, let it be that one. Validation that only says *no* makes people resent your tool. Validation that says *no, here, because of this, and here's what valid looks like* makes people trust it. The effort is a few extra f-strings; the payoff is every future debugging session.\n\nStructural bugs in an LLM workflow — dead ends, phantom transitions, non-terminating loops — are present the instant the config is written. There's no reason to let them surface at runtime. By modeling the workflow as an explicit graph and validating its topology at load time, you convert a whole category of production incidents into a fast, boring failure at startup with a message that tells you exactly what to fix.\n\nEverything here runs against the open-source [ llm-workflow-router](https://pypi.org/project/llm-workflow-router/) (MIT). Try\n\n`validate`\n\nand `analyze`\n\non your own configs — and if you're building anything where an AI system hands off between steps, put the validation call in your startup path before you need it.", "url": "https://wpnews.pro/news/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks", "canonical_source": "https://dev.to/dobybaxter127/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks-35l1", "published_at": "2026-07-13 20:30:50+00:00", "updated_at": "2026-07-13 20:45:40.532969+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-agents", "ai-safety", "mlops"], "entities": ["llm-workflow-router"], "alternates": {"html": "https://wpnews.pro/news/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks", "markdown": "https://wpnews.pro/news/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks.md", "text": "https://wpnews.pro/news/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks.txt", "jsonld": "https://wpnews.pro/news/validate-your-llm-workflow-before-it-runs-python-yaml-topology-checks.jsonld"}}