Stop pretending your LLM pipeline is an agent if you can already

When I finally dug into the logs, I realized the "agent" was performing the exact same three steps every single time: extract, transform, and respond. It never actually used its autonomy to deviate from a fixed path; I had essentially built a glorified for-loop and called it an agent.

The actual distinction between agency and pipelines #

We need to stop using "agent" as a buzzword for any system that uses an LLM. The only distinction that matters is who controls the flow at runtime.

• Real Agency: The model decides the control flow. It chooses which tool to call, whether to loop back to a previous step, or when to terminate the process based on dynamic data it encounters.
• Pipelines: The control flow is fixed at design time. Step A leads to Step B, which leads to Step C. The LLM performs a taskwithin the step, but it doesn't choose the step.

Using an LLM to extract a field or classify a ticket isn't agency—it's just a smart function call. Agency only exists when you hand the steering wheel to the model and let it pick the route. If you can map out the entire logic on a whiteboard before writing a single line of code, you have a pipeline, not an agent.

The cost of fake agency #

Pretending a pipeline is an agent creates a massive technical debt. When you let a model decide the path for a task that actually has a fixed structure, you pay a "complexity tax" in three specific ways:

1. Nondeterminism: Your bugs become impossible to reproduce because the model might take a different path on run #42 than it did on run #1.

2. Latency and Cost: Reasoning loops (like ReAct or Reflexion) require multiple LLM passes to accomplish what a single prompt could do if the flow were fixed.

3. Debuggability Collapse: When a fixed pipeline breaks, you know exactly which node failed. When an "autonomous agent" fails, the cause is often a series of three "decisions" made upstream that you can't easily trace or constrain.

A practical AI workflow for transition #

If you're struggling with an unstable agent, I recommend a "de-agenting" process. Start by logging every single tool call and reasoning step. If you see a pattern emerging—like the model always calling search_docs followed by summarize_text—hardcode that sequence.

For those who still need a bit of dynamic behavior without the chaos, I've found that a "Router" pattern works best. Instead of a fully autonomous loop, use a small, fast model (like GPT-4o-mini or Claude 3 Haiku) to classify the intent and then route the request to a specific, fixed pipeline.

Here is a basic prompt structure I use to turn an "autonomous" mess into a reliable router. Instead of letting the model "reason" about what to do, I force it to output a specific key that maps to a hardcoded function.

You are a request router. Your only job is to categorize the user input into one of the following buckets:
- DATA_EXTRACTION: Use this if the user wants a specific value from a document.
- SUMMARY: Use this if the user wants a condensed version of a text.
- GENERAL_QUERY: Use this for everything else.

Output ONLY the key (e.g., DATA_EXTRACTION). Do not explain your reasoning. Do not provide conversational filler.

Input: {{user_query}}
Output:

By moving the logic from "runtime decision" to "design-time routing," you get the best of both worlds: the flexibility of LLMs with the reliability of a traditional software pipeline. Stop over-engineering your workflow with autonomous loops if a simple sequence of three prompts gets the job done.

Next Claude Code is making the traditional VC pitch deck feel like a →