Stop pretending your LLM pipeline is an agent if you can already

Software engineer argues that many systems marketed as AI agents are actually fixed pipelines, since the model does not control the runtime flow. The author recommends a 'de-agenting' process, including logging tool calls and using a router pattern with a small model like GPT-4o-mini or Claude 3 Haiku to classify intent and route to fixed pipelines, to avoid nondeterminism, latency, and debuggability issues.

Stop pretending your LLM pipeline is an agent if you can already When I finally dug into the logs, I realized the "agent" was performing the exact same three steps every single time: extract, transform, and respond. It never actually used its autonomy to deviate from a fixed path; I had essentially built a glorified for-loop and called it an agent. The actual distinction between agency and pipelines We need to stop using "agent" as a buzzword for any system that uses an LLM. The only distinction that matters is who controls the flow at runtime. - Real Agency: The model decides the control flow. It chooses which tool to call, whether to loop back to a previous step, or when to terminate the process based on dynamic data it encounters. - Pipelines: The control flow is fixed at design time. Step A leads to Step B, which leads to Step C. The LLM performs a task within the step, but it doesn't choose the step. Using an LLM to extract a field or classify a ticket isn't agency—it's just a smart function call. Agency only exists when you hand the steering wheel to the model and let it pick the route. If you can map out the entire logic on a whiteboard before writing a single line of code, you have a pipeline, not an agent. The cost of fake agency Pretending a pipeline is an agent creates a massive technical debt. When you let a model decide the path for a task that actually has a fixed structure, you pay a "complexity tax" in three specific ways: 1. Nondeterminism: Your bugs become impossible to reproduce because the model might take a different path on run 42 than it did on run 1. 2. Latency and Cost: Reasoning loops like ReAct or Reflexion require multiple LLM passes to accomplish what a single prompt could do if the flow were fixed. 3. Debuggability Collapse: When a fixed pipeline breaks, you know exactly which node failed. When an "autonomous agent" fails, the cause is often a series of three "decisions" made upstream that you can't easily trace or constrain. A practical AI workflow for transition If you're struggling with an unstable agent, I recommend a "de-agenting" process. Start by logging every single tool call and reasoning step. If you see a pattern emerging—like the model always calling search docs followed by summarize text —hardcode that sequence. For those who still need a bit of dynamic behavior without the chaos, I've found that a "Router" pattern works best. Instead of a fully autonomous loop, use a small, fast model like GPT-4o-mini or Claude /en/tags/claude/ 3 Haiku to classify the intent and then route the request to a specific, fixed pipeline. Here is a basic prompt structure I use to turn an "autonomous" mess into a reliable router. Instead of letting the model "reason" about what to do, I force it to output a specific key that maps to a hardcoded function. You are a request router. Your only job is to categorize the user input into one of the following buckets: - DATA EXTRACTION: Use this if the user wants a specific value from a document. - SUMMARY: Use this if the user wants a condensed version of a text. - GENERAL QUERY: Use this for everything else. Output ONLY the key e.g., DATA EXTRACTION . Do not explain your reasoning. Do not provide conversational filler. Input: {{user query}} Output: By moving the logic from "runtime decision" to "design-time routing," you get the best of both worlds: the flexibility of LLMs with the reliability of a traditional software pipeline. Stop over-engineering your workflow with autonomous loops if a simple sequence of three prompts gets the job done. Next Claude Code is making the traditional VC pitch deck feel like a → /en/threads/9041/