ReTrace: Distilling Agent Workflows into Planner-Free Code

A developer built ReTrace, a system that distills LLM agent workflows into planner-free, deterministic code by parsing the reasoning traces agents write before each tool call. ReTrace merges decision conditions across logged runs into a platform-agnostic rule document, then compiles it into a state machine for targets like LangGraph, Temporal, or Prefect, with a fallback to the full agent on uncovered edge cases. On 79 production SOPs the workflows reached 93.8% completion at 24% lower cost per task, and on τ²-Bench they ran in 1.3 seconds versus 170 seconds for the agent.

LLM agents can follow a Standard Operating Procedure written in natural language, but they call an inference endpoint at every decision point. That works for one-off tasks. When you run the same SOP a thousand times a day, the per-step latency and token cost becomes a production problem. ReTrace solves this by tracing agent execution, extracting the decision logic from reasoning traces, and compiling a planner-free workflow. The workflow is deterministic code that selects the next tool call without an LLM in the loop. On production SOPs, ReTrace workflows reach 93.8% completion at 24% lower cost per task. On τ²-Bench, they run in 1.3 seconds instead of 170 seconds. Agent frameworks like LangGraph and AutoGen orchestrate tool calls by asking the LLM "what next?" after every step. This pattern is flexible but expensive: Workflow distillation extracts that logic once, then reuses it. The result is a state machine that calls tools directly, falling back to the agent only when it encounters an edge case. ReTrace takes three inputs: an SOP in natural language, a few logged agent runs, and the tool definitions. It produces an executable workflow in three stages. The agent writes a reasoning trace before each tool call. ReTrace parses these traces to extract the condition that triggered the action: check refund policy with order id." user intent == "refund" check refund policy order id This step relies on the agent's natural language explanation, not just the tool call signature. Tool calls alone do not reveal why the agent chose them. ReTrace merges conditions across multiple runs into a compact rule document. If three runs all call check refund policy when the user mentions "refund," "return," or "money back," the rule becomes: IF user intent IN "refund", "return", "money back" THEN check refund policy order id The rule document is platform-agnostic. It describes the decision logic without committing to a specific orchestration engine. The final stage compiles the rule document into executable code for a target platform LangGraph, Temporal, Prefect . The generated workflow is a state machine with explicit transitions: python def workflow state : if state "user intent" in "refund", "return", "money back" : result = check refund policy state "order id" state "policy result" = result return "check eligibility" elif state "user intent" == "track order": return "fetch tracking" else: return "fallback to agent" Only this last stage is platform-specific. The same rule document can be compiled for different orchestration systems. The distilled workflow includes a fallback node. When the workflow encounters a condition not covered by the rule document, it hands control back to the full LLM agent. This happens when: The fallback is a safety valve, not a failure. The workflow logs the fallback event, and you can retrain the rule document to cover the new case. ReTrace was tested on three benchmarks: | Benchmark | Workflow Success | Agent Success | Workflow Latency | Agent Latency | Cost Reduction | |---|---|---|---|---|---| | 79 production SOPs | 93.8% | 96.8% | Not reported | Not reported | 24% | | τ²-Bench | 75.8% | 80.6% | 1.3s | 170s | Not reported | | SOP-Bench 8 procedures | 80.6% no repair | 85.5% | 0.40s | 46.6s | Not reported | With three rounds of repair on training failures, τ²-Bench workflows improved from 66.7% to 81.2% for k=2 workflows. SOP-Bench workflows reached 87.8% after repair, exceeding the agent's 85.5%. The latency reduction is dramatic. On τ²-Bench, workflows run 130× faster. On SOP-Bench, they run 116× faster. This matters when you process thousands of tasks per day. ReTrace includes a repair mechanism. When a workflow fails on a training example, the system: Three rounds of repair on labeled training failures lifted SOP-Bench workflows from 80.6% to 87.8%. The repair loop is supervised: you provide labeled examples, and ReTrace updates the rule document. Distilled workflows introduce a new artifact to manage: the rule document. When the underlying SOP changes, you must: The paper does not describe a versioning strategy. In production, you will need: The rule document is human-readable, so you can diff versions and review changes before deploying. Distilled workflows fail in predictable ways: You can detect these failures by monitoring the fallback rate. If more than 10% of tasks hit the fallback, the rule document needs retraining. A production deployment of ReTrace requires: The rule compiler and repair pipeline run offline. The workflow runtime and fallback router run in the hot path. The distilled workflow executes code generated from agent reasoning traces. This introduces a code injection risk if the agent's reasoning is not sanitized. An attacker could craft an input that causes the agent to write a malicious condition, which then gets compiled into the workflow. Mitigations: The fallback to the agent is also a security boundary. The agent has access to all tools, while the workflow may have a restricted set. Ensure the fallback does not bypass access controls. You need visibility into: The rule document itself is an observable artifact. Diff it across versions to understand how the workflow logic evolves. Use ReTrace when: Avoid ReTrace when: Workflow distillation trades flexibility for speed and cost. It works when the task is repetitive and the decision logic is extractable from reasoning traces. It fails when the task is too dynamic or the agent's reasoning is too opaque.