{"slug": "retrace-distilling-agent-workflows-into-planner-free-code", "title": "ReTrace: Distilling Agent Workflows into Planner-Free Code", "summary": "A developer built ReTrace, a system that distills LLM agent workflows into planner-free, deterministic code by parsing the reasoning traces agents write before each tool call. ReTrace merges decision conditions across logged runs into a platform-agnostic rule document, then compiles it into a state machine for targets like LangGraph, Temporal, or Prefect, with a fallback to the full agent on uncovered edge cases. On 79 production SOPs the workflows reached 93.8% completion at 24% lower cost per task, and on τ²-Bench they ran in 1.3 seconds versus 170 seconds for the agent.", "body_md": "LLM agents can follow a Standard Operating Procedure written in natural language, but they call an inference endpoint at every decision point. That works for one-off tasks. When you run the same SOP a thousand times a day, the per-step latency and token cost becomes a production problem.\n\nReTrace solves this by tracing agent execution, extracting the decision logic from reasoning traces, and compiling a planner-free workflow. The workflow is deterministic code that selects the next tool call without an LLM in the loop. On production SOPs, ReTrace workflows reach 93.8% completion at 24% lower cost per task. On τ²-Bench, they run in 1.3 seconds instead of 170 seconds.\n\nAgent frameworks like LangGraph and AutoGen orchestrate tool calls by asking the LLM \"what next?\" after every step. This pattern is flexible but expensive:\n\nWorkflow distillation extracts that logic once, then reuses it. The result is a state machine that calls tools directly, falling back to the agent only when it encounters an edge case.\n\nReTrace takes three inputs: an SOP in natural language, a few logged agent runs, and the tool definitions. It produces an executable workflow in three stages.\n\nThe agent writes a reasoning trace before each tool call. ReTrace parses these traces to extract the condition that triggered the action:\n\n`check_refund_policy` with order_id.\"`user_intent == \"refund\"`\n`check_refund_policy(order_id)`\nThis step relies on the agent's natural language explanation, not just the tool call signature. Tool calls alone do not reveal why the agent chose them.\n\nReTrace merges conditions across multiple runs into a compact rule document. If three runs all call `check_refund_policy` when the user mentions \"refund,\" \"return,\" or \"money back,\" the rule becomes:\n\n```\nIF user_intent IN [\"refund\", \"return\", \"money_back\"]\nTHEN check_refund_policy(order_id)\n```\n\nThe rule document is platform-agnostic. It describes the decision logic without committing to a specific orchestration engine.\n\nThe final stage compiles the rule document into executable code for a target platform (LangGraph, Temporal, Prefect). The generated workflow is a state machine with explicit transitions:\n\n``` python\ndef workflow(state):\n    if state[\"user_intent\"] in [\"refund\", \"return\", \"money_back\"]:\n        result = check_refund_policy(state[\"order_id\"])\n        state[\"policy_result\"] = result\n        return \"check_eligibility\"\n    elif state[\"user_intent\"] == \"track_order\":\n        return \"fetch_tracking\"\n    else:\n        return \"fallback_to_agent\"\n```\n\nOnly this last stage is platform-specific. The same rule document can be compiled for different orchestration systems.\n\nThe distilled workflow includes a fallback node. When the workflow encounters a condition not covered by the rule document, it hands control back to the full LLM agent. This happens when:\n\nThe fallback is a safety valve, not a failure. The workflow logs the fallback event, and you can retrain the rule document to cover the new case.\n\nReTrace was tested on three benchmarks:\n\n| Benchmark | Workflow Success | Agent Success | Workflow Latency | Agent Latency | Cost Reduction | \n|---|---|---|---|---|---|\n| 79 production SOPs | 93.8% | 96.8% | Not reported | Not reported | 24% | \n| τ²-Bench | 75.8% | 80.6% | 1.3s | 170s | Not reported | \n| SOP-Bench (8 procedures) | 80.6% (no repair) | 85.5% | 0.40s | 46.6s | Not reported | \n\nWith three rounds of repair on training failures, τ²-Bench workflows improved from 66.7% to 81.2% for k=2 workflows. SOP-Bench workflows reached 87.8% after repair, exceeding the agent's 85.5%.\n\nThe latency reduction is dramatic. On τ²-Bench, workflows run 130× faster. On SOP-Bench, they run 116× faster. This matters when you process thousands of tasks per day.\n\nReTrace includes a repair mechanism. When a workflow fails on a training example, the system:\n\nThree rounds of repair on labeled training failures lifted SOP-Bench workflows from 80.6% to 87.8%. The repair loop is supervised: you provide labeled examples, and ReTrace updates the rule document.\n\nDistilled workflows introduce a new artifact to manage: the rule document. When the underlying SOP changes, you must:\n\nThe paper does not describe a versioning strategy. In production, you will need:\n\nThe rule document is human-readable, so you can diff versions and review changes before deploying.\n\nDistilled workflows fail in predictable ways:\n\nYou can detect these failures by monitoring the fallback rate. If more than 10% of tasks hit the fallback, the rule document needs retraining.\n\nA production deployment of ReTrace requires:\n\nThe rule compiler and repair pipeline run offline. The workflow runtime and fallback router run in the hot path.\n\nThe distilled workflow executes code generated from agent reasoning traces. This introduces a code injection risk if the agent's reasoning is not sanitized. An attacker could craft an input that causes the agent to write a malicious condition, which then gets compiled into the workflow.\n\nMitigations:\n\nThe fallback to the agent is also a security boundary. The agent has access to all tools, while the workflow may have a restricted set. Ensure the fallback does not bypass access controls.\n\nYou need visibility into:\n\nThe rule document itself is an observable artifact. Diff it across versions to understand how the workflow logic evolves.\n\n**Use ReTrace when:**\n\n**Avoid ReTrace when:**\n\nWorkflow distillation trades flexibility for speed and cost. It works when the task is repetitive and the decision logic is extractable from reasoning traces. It fails when the task is too dynamic or the agent's reasoning is too opaque.", "url": "https://wpnews.pro/news/retrace-distilling-agent-workflows-into-planner-free-code", "canonical_source": "https://dev.to/mech_app_ai/retrace-distilling-agent-workflows-into-planner-free-code-2634", "published_at": "2026-11-12 20:09:34+00:00", "updated_at": "2026-11-12 20:23:06.428861+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "mlops", "developer-tools"], "entities": ["ReTrace", "LangGraph", "AutoGen", "Temporal", "Prefect", "τ²-Bench", "SOP-Bench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/retrace-distilling-agent-workflows-into-planner-free-code", "markdown": "https://wpnews.pro/news/retrace-distilling-agent-workflows-into-planner-free-code.md", "text": "https://wpnews.pro/news/retrace-distilling-agent-workflows-into-planner-free-code.txt", "jsonld": "https://wpnews.pro/news/retrace-distilling-agent-workflows-into-planner-free-code.jsonld"}}