{"slug": "stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already", "title": "Stop pretending your LLM pipeline is an agent if you can already", "summary": "Software engineer argues that many systems marketed as AI agents are actually fixed pipelines, since the model does not control the runtime flow. The author recommends a 'de-agenting' process, including logging tool calls and using a router pattern with a small model like GPT-4o-mini or Claude 3 Haiku to classify intent and route to fixed pipelines, to avoid nondeterminism, latency, and debuggability issues.", "body_md": "# Stop pretending your LLM pipeline is an agent if you can already\n\nWhen I finally dug into the logs, I realized the \"agent\" was performing the exact same three steps every single time: extract, transform, and respond. It never actually used its autonomy to deviate from a fixed path; I had essentially built a glorified for-loop and called it an agent.\n\n## The actual distinction between agency and pipelines\n\nWe need to stop using \"agent\" as a buzzword for any system that uses an LLM. The only distinction that matters is who controls the flow at runtime.\n\n- **Real Agency:** The model decides the control flow. It chooses which tool to call, whether to loop back to a previous step, or when to terminate the process based on dynamic data it encounters.\n- **Pipelines:** The control flow is fixed at design time. Step A leads to Step B, which leads to Step C. The LLM performs a task*within* the step, but it doesn't choose the step.\n\nUsing an LLM to extract a field or classify a ticket isn't agency—it's just a smart function call. Agency only exists when you hand the steering wheel to the model and let it pick the route. If you can map out the entire logic on a whiteboard before writing a single line of code, you have a pipeline, not an agent.\n\n## The cost of fake agency\n\nPretending a pipeline is an agent creates a massive technical debt. When you let a model decide the path for a task that actually has a fixed structure, you pay a \"complexity tax\" in three specific ways:\n\n1. **Nondeterminism:** Your bugs become impossible to reproduce because the model might take a different path on run #42 than it did on run #1.\n\n2. **Latency and Cost:** Reasoning loops (like ReAct or Reflexion) require multiple LLM passes to accomplish what a single prompt could do if the flow were fixed.\n\n3. **Debuggability Collapse:** When a fixed pipeline breaks, you know exactly which node failed. When an \"autonomous agent\" fails, the cause is often a series of three \"decisions\" made upstream that you can't easily trace or constrain.\n\n## A practical AI workflow for transition\n\nIf you're struggling with an unstable agent, I recommend a \"de-agenting\" process. Start by logging every single tool call and reasoning step. If you see a pattern emerging—like the model always calling `search_docs` followed by `summarize_text`—hardcode that sequence.\n\nFor those who still need a bit of dynamic behavior without the chaos, I've found that a \"Router\" pattern works best. Instead of a fully autonomous loop, use a small, fast model (like GPT-4o-mini or [Claude](/en/tags/claude/) 3 Haiku) to classify the intent and then route the request to a specific, fixed pipeline.\n\nHere is a basic prompt structure I use to turn an \"autonomous\" mess into a reliable router. Instead of letting the model \"reason\" about what to do, I force it to output a specific key that maps to a hardcoded function.\n\n```\nYou are a request router. Your only job is to categorize the user input into one of the following buckets:\n- DATA_EXTRACTION: Use this if the user wants a specific value from a document.\n- SUMMARY: Use this if the user wants a condensed version of a text.\n- GENERAL_QUERY: Use this for everything else.\n\nOutput ONLY the key (e.g., DATA_EXTRACTION). Do not explain your reasoning. Do not provide conversational filler.\n\nInput: {{user_query}}\nOutput:\n```\n\nBy moving the logic from \"runtime decision\" to \"design-time routing,\" you get the best of both worlds: the flexibility of LLMs with the reliability of a traditional software pipeline. Stop over-engineering your workflow with autonomous loops if a simple sequence of three prompts gets the job done.\n\n[Next Claude Code is making the traditional VC pitch deck feel like a →](/en/threads/9041/)", "url": "https://wpnews.pro/news/stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already", "canonical_source": "https://promptcube3.com/en/threads/9042/", "published_at": "2026-09-08 16:26:35+00:00", "updated_at": "2026-09-08 16:57:04.349418+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools"], "entities": ["GPT-4o-mini", "Claude 3 Haiku"], "alternates": {"html": "https://wpnews.pro/news/stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already", "markdown": "https://wpnews.pro/news/stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already.md", "text": "https://wpnews.pro/news/stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already.txt", "jsonld": "https://wpnews.pro/news/stop-pretending-your-llm-pipeline-is-an-agent-if-you-can-already.jsonld"}}