Validate Your LLM Workflow Before It Runs (Python + YAML Topology Checks)

A developer built a Python + YAML topology checker for LLM workflows that validates the graph structure before runtime, catching dead ends, missing steps, and illegal loops. The open-source package llm-workflow-router analyzes directed graphs of workflow steps and refuses to start if the topology is invalid, preventing runtime failures in production.

Your LLM app boots fine. It answers the first few messages fine. Then, ten minutes into a real conversation, it quietly walks from the support step into the billing step that doesn't exist anymore, or loops faq → support → faq → support until something times out. Nothing crashed at startup. The mistake was in the workflow's shape the whole time — you just didn't find out until runtime, in production, with a user watching. This is the same class of bug as a malformed config file: the error exists the moment you write it, but the tool waits until the worst possible time to tell you. The fix is the same too. Catch it at load time, before a single step executes. In this tutorial we'll build a small, declarative LLM workflow as YAML, then validate its topology — the graph of which step can hand off to which — before we run anything. We'll catch dead ends, transitions to steps that don't exist, and loops that can never terminate. All the code here runs against a real open-source package, llm-workflow-router https://pypi.org/project/llm-workflow-router/ , so you can follow along end to end. Most guardrails for LLM systems limit volume : a max number of tool calls, a timeout, a retry cap. Those are useful, but they treat the symptom. A runaway loop gets cut off after N iterations — it doesn't get prevented, and you learn nothing about why the workflow could loop in the first place. Structural validation asks a different question: before this workflow ever handles a request, is its graph even valid? A step that transitions to a nonexistent target is invalid no matter what any model says. A step with no way out is invalid. A cycle through a step that isn't allowed to be re-entered is invalid. None of that depends on content, so none of it needs to wait for runtime to be caught. The mental model: your workflow is a directed graph. Nodes are steps "containers" . Edges are the transitions each step permits. Validating the workflow means analyzing that graph and refusing to start if it's broken. Here's a minimal support flow. A user enters at support entry , which may hand off to faq or refuse; faq may only refuse. REFUSE is a terminal state, not a step. good config.yaml entrypoints: - support entry containers: support entry: allow transitions: - faq - REFUSE allow reentry: false max invocations: 3 faq: allow transitions: - REFUSE allow reentry: true max invocations: 5 Three things are being declared per step, and each one is a rule the validator can check: allow transitions — the allow reentry — whether the step may be entered more than once. This is what turns "is there a cycle?" into "is there a cycle that's actually a problem?" max invocations — a hard ceiling on how many times the step may run. Must be greater than zero, or the step can never execute. entrypoints names the authoritative roots of the graph. Declaring them explicitly means the validator doesn't have to guess where the workflow starts — and it can flag any step that no path can reach. Install the package: pip install llm-workflow-router Then validate the config before you wire it into anything: python -m router.cli validate good config.yaml { "ok": true } Clean exit, exit code 0 . Now let's break it on purpose — the kind of breakage that's easy to introduce and impossible to spot by eye once a config gets large: broken config.yaml containers: support entry: allow transitions: - faq - billing typo: this step does not exist allow reentry: false max invocations: 3 faq: allow transitions: - support entry loops back into a no-reentry step allow reentry: true max invocations: 5 orphaned step: allow transitions: - REFUSE allow reentry: false max invocations: 1 There are three separate problems hiding in there. Run validate and it stops you at the first hard error: python -m router.cli validate broken config.yaml { "ok": false, "error": "CONFIG VALIDATION ERROR", "message": "Container 'support entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}." } Exit code 2 . Notice what the error message actually does: it names the offending step support entry , the bad value billing , and the set of things that would have been valid . That last part is the difference between an error that stops you and an error that helps you. More on that below. validate is pass/fail — good for CI, where you want a nonzero exit and the first blocking reason. But when you're actually fixing a workflow, you want the whole picture, not one error at a time. That's what analyze is for: it inspects the topology and reports everything, graded by severity, without raising. python -m router.cli analyze broken config.yaml { "container count": 3, "entry containers": "orphaned step" , "entrypoints declared": false, "cycle count": 1, "cycles": "faq", "support entry" , "issues": { "severity": "ERROR", "code": "CYCLE WITH REENTRY BLOCKED", "message": "Cycle 'faq', 'support entry' includes containers with allow reentry=false: 'support entry' " }, { "severity": "ERROR", "code": "UNKNOWN TARGET", "message": "Container 'support entry' transitions to unknown 'billing'." }, { "severity": "WARNING", "code": "UNREACHABLE CONTAINERS", "message": "Unreachable containers: 'faq', 'support entry' " } } This one report catches all three bugs I planted: UNKNOWN TARGET billing typo. A transition to a step that isn't in the graph. CYCLE WITH REENTRY BLOCKED faq transitions back to support entry , forming a loop faq → support entry → faq , but support entry declared allow reentry: false . The workflow contradicts itself: the topology requires re-entering a step the config forbids re-entering. entrypoints block, the analyzer had to orphaned step , so it wrongly became the "entry." Everything else is now unreachable from that inferred root, hence the UNREACHABLE CONTAINERS warning. This is a great argument for always declaring entrypoints explicitly: it removes the guessing.The severity split matters. An ERROR is a contradiction that makes the workflow structurally invalid — it blocks. A WARNING is a smell worth surfacing but not necessarily fatal. Cycles are a nice example of the distinction: a loop through steps that all permit re-entry is a WARNING it can legitimately terminate via max invocations , while a loop through a step that forbids re-entry is an ERROR it can't . The CLI is a thin wrapper. If you're building the workflow programmatically — or want validation inside your own app's startup path — the same checks are one function call. python from router.engine import WorkflowEngine from router.models.config import ContainerConfig, WorkflowConfig from router.models.enums import InteractionState from router.models.metadata import InteractionMetadata from router.validation.config validator import validate workflow config cfg = WorkflowConfig containers={ "support entry": ContainerConfig allow transitions= "faq", "REFUSE" , allow reentry=False, max invocations=3, , "faq": ContainerConfig allow transitions= "REFUSE" , allow reentry=True, max invocations=5, , }, entrypoints= "support entry" , validate workflow config cfg raises ConfigValidationError if the graph is invalid print "Config valid ✓" validate workflow config raises ConfigValidationError on any hard error, so the natural place to call it is at load time — the workflow simply refuses to start if its shape is wrong. That's the whole idea: an invalid state is made unrepresentable at runtime because it was rejected before runtime began. Once the config is validated, evaluating a step is deterministic — the same metadata always yields the same decision: engine = WorkflowEngine cfg metadata = InteractionMetadata container="support entry", previous state=InteractionState.PROCEED, transition history= , invocation depth={"support entry": 1}, requested action="faq", trace id="demo-001", print engine.evaluate metadata EvaluationResult state=<InteractionState.PROCEED: 'PROCEED' , container='support entry', reason=None, trace id='demo-001' The engine checks the request against the validated rules: is support entry a real step? Under its invocation ceiling? Is faq in its allow transitions ? All yes, so the result is PROCEED . Ask for a transition that isn't allowed and you'd get a REFUSE with a machine-readable reason like INVALID TRANSITION — no exceptions thrown mid-flight, just an explicit, inspectable decision. Go back and look at that first failure message: Container 'support entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}. It would have been easier to write ValueError: invalid transition . It would also have been useless. A good validation error answers three questions without making the reader go digging: where is the problem support entry , what is wrong billing isn't a valid target , and what would be right a real step name, or a terminal state . The analyze codes do the same job for tooling — UNKNOWN TARGET , CYCLE WITH REENTRY BLOCKED , ORPHAN CONTAINER are greppable, stable identifiers you can build dashboards or CI rules around. If you take one habit away from this, let it be that one. Validation that only says no makes people resent your tool. Validation that says no, here, because of this, and here's what valid looks like makes people trust it. The effort is a few extra f-strings; the payoff is every future debugging session. Structural bugs in an LLM workflow — dead ends, phantom transitions, non-terminating loops — are present the instant the config is written. There's no reason to let them surface at runtime. By modeling the workflow as an explicit graph and validating its topology at load time, you convert a whole category of production incidents into a fast, boring failure at startup with a message that tells you exactly what to fix. Everything here runs against the open-source llm-workflow-router https://pypi.org/project/llm-workflow-router/ MIT . Try validate and analyze on your own configs — and if you're building anything where an AI system hands off between steps, put the validation call in your startup path before you need it.