{"slug": "why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm", "title": "Why Your AI Agent Can't Execute Its Own Plan: Bridging the Gap Between Local LLM Intelligence and Real-World Software Reliability", "summary": "A developer's analysis on tamiz.pro identifies why AI agents fail to execute their own plans, attributing the issue to an architectural mismatch between probabilistic LLMs and deterministic software systems. The post categorizes failure modes such as opaque natural-language plans and state drift, and proposes formal intermediate representations and typed action schemas to improve execution reliability.", "body_md": "*Originally published on tamiz.pro.*\n\nYou prompt your agent to orchestrate a multi-step workflow. It generates a beautifully reasoned plan. Then it fails on step three. Or eight. Or quietly produces wrong output that no one notices until it's too late.\n\nThis isn't a prompt engineering problem. It's an architecture problem — one rooted in the fundamental mismatch between probabilistic language models and deterministic software systems. Understanding *why* agents fail to execute their own plans is the prerequisite to building ones that don't.\n\nThis article dissects the technical failure modes, traces them through the agent stack, and explores architectural patterns that close the gap between LLM-grade reasoning and production-grade execution reliability.\n\nBefore diagnosing the failure modes, we need to be precise about what's actually happening when an agent \"executes a plan.\"\n\nA modern agentic system has two conceptual layers that are conflated in practice:\n\nThe LLM lives entirely in the reasoning layer. When we say an agent \"executes its plan,\" what's really happening is that the LLM generates text that a software harness interprets as instructions. The reliability of the whole system is bounded by whichever of these two layers is weaker.\n\nIn practice, the execution layer is where things collapse. And they collapse in predictable, categorizable ways.\n\nAn LLM-generated plan is a sequence of natural language instructions. When you say \"the agent will execute this plan,\" you're implicitly assuming the plan is executable by something other than another LLM.\n\nBut here's the problem: **the plan is opaque.**\n\n```\n# What the LLM generates as a \"plan\":\nplan = [\n    {\"step\": 1, \"action\": \"query user database\", \"params\": \"active users last 30 days\"},\n    {\"step\": 2, \"action\": \"aggregate metrics\", \"params\": \"count by region\"},\n    {\"step\": 3, \"action\": \"format report\", \"params\": \"PDF with charts\"},\n    {\"step\": 4, \"action\": \"send to stakeholder\", \"params\": \"weekly digest list\"}\n]\n```\n\nStep 3 says \"format report\" with a param of \"PDF with charts.\" That's not a program. That's a description of intent. The execution engine needs to:\n\nEvery one of these decisions is a potential point of failure. The LLM that generated the plan doesn't actually *understand* any of this — it's predicting the next reasonable word in a sequence. The execution engine that *does* need to understand it is a separate system, often hand-written, and almost always incomplete.\n\n**The fix:** Plans must be expressed in a formal intermediate representation (IR), not natural language. This means the LLM generates structured output that maps to concrete, typed actions:\n\n```\n// Formal plan representation\ninterface ExecutablePlan {\n  steps: ExecutionStep[];\n  constraints: PlanConstraints;\n  validation: ValidationRules[];\n}\n\ninterface ExecutionStep {\n  id: string;\n  action: ActionType; // Enum, not string\n  inputs: TypedSchema; // JSON Schema validated\n  dependencies: string[]; // DAG, not implicit ordering\n  retryPolicy: RetryConfig;\n  timeout: Duration;\n}\n\nenum ActionType {\n  DATABASE_QUERY,\n  API_CALL,\n  FILE_WRITE,\n  EMAIL_SEND,\n  // ... exhaustively enumerated\n}\n```\n\nThe LLM fills in parameters, but the *shape* of execution is constrained by the type system. This eliminates the semantic gap between \"what the plan says\" and \"what the code does.\"\n\nEven when the plan is well-formed, a silent killer is **state drift** — the world changes between when the plan is generated and when a step is executed.\n\nConsider this sequence:\n\n```\nT=0s    : Agent plans → \"fetch user X's data from API endpoint /v2/users\"\nT=5s    : API endpoint /v2/users is deprecated; /v3/users is now live\nT=6s    : Agent executes step 1 → 404 error, plan halts\n```\n\nOr worse — a silently wrong result:\n\n```\nT=0s    : Agent plans → \"calculate total revenue from transactions table\"\nT=5s    : A deployment changes the schema; a new `currency` column appears\nT=6s    : Agent executes → queries sum(revenue) but gets mixed currencies\nT=7s    : Agent reports $1.2M revenue to stakeholders → wrong by 3x\n```\n\nThe LLM has no awareness of these state transitions. It generates a plan based on its training data and whatever context you provided. It doesn't know the schema changed yesterday. It doesn't know the API version you meant.\n\n**The fix:** Agents need a **state awareness layer** that validates plan assumptions against current reality before execution:\n\n``` python\nclass PlanValidator:\n    def __init__(self, state_probe: StateProbe):\n        self.probe = state_probe\n\n    def validate_plan(self, plan: ExecutablePlan) -> ValidationReport:\n        for step in plan.steps:\n            # Check that referenced schemas still exist\n            schema = self.probe.get_schema(step.inputs)\n            if not schema.matches(step.params.schema):\n                raise SchemaMismatchError(step.id, schema, step.params.schema)\n\n            # Check that endpoints are reachable\n            if step.action == ActionType.API_CALL:\n                health = self.probe.check_endpoint(step.params.url)\n                if health.status != \"healthy\":\n                    self.flag_risk(step.id, f\"Endpoint degraded: {health.status}\")\n\n        return ValidationReport(completed=True, warnings=self.warnings)\n```\n\nThis turns plan execution from a blind leap into a verified execution. The agent still generates plans naturally, but they're validated against the live system state before any side effect occurs.\n\nLLMs are excellent at generating plans. They're terrible at handling deviations from plans — because error recovery requires *situational awareness* that the planning process doesn't carry forward.\n\nWhen an agent encounters an unexpected error, it has several options:\n\nThe LLM, operating in a stateless request-response loop, has no memory of the original plan's intent beyond what's in the context window. It doesn't know which steps are on the critical path. It doesn't know whether a failure is transient or permanent. It makes a best-guess decision based on whatever context happened to be in the prompt.\n\n**The fix:** Implement **structured error handling** as a first-class component of the agent architecture:\n\n```\nclass ErrorRecoveryEngine:\n    \"\"\"\n    Separates error handling logic from the LLM's planning logic.\n    Uses deterministic rules and bounded LLM calls for recovery.\n    \"\"\"\n\n    RECOVERY_STRATEGIES = {\n        \"timeout\": [\"retry_with_backoff\", \"skip_with_log\", \"abort\"],\n        \"validation_error\": [\"retry_with_corrected_params\", \"ask_for_clarification\", \"abort\"],\n        \"auth_failure\": [\"retry_with_refreshed_token\", \"abort\"],\n        \"dependency_unavailable\": [\"retry_after_delay\", \"use_fallback\", \"abort\"],\n    }\n\n    def handle(self, step: ExecutionStep, error: ExecutionError, context: PlanContext) -> RecoveryDecision:\n        strategy_type = self._classify_error(error)\n        strategies = self.RECOVERY_STRATEGIES.get(strategy_type, [\"abort\"])\n\n        # Deterministic first-pass filtering\n        viable = [s for s in strategies if self._is_viable(step, s, context)]\n\n        # Bounded LLM call for nuanced decisions\n        if len(viable) > 1:\n            decision = self._llm_select_recovery(step, error, viable, context)\n        else:\n            decision = RecoveryDecision(action=viable[0])\n\n        return decision\n\n    def _is_viable(self, step: ExecutionStep, strategy: str, ctx: PlanContext) -> bool:\n        \"\"\"Deterministic checks — no LLM involved.\"\"\"\n        if strategy == \"retry_with_backoff\" and step.retry_count >= step.retry_policy.max_attempts:\n            return False\n        if strategy == \"use_fallback\" and not step.has_fallback:\n            return False\n        if strategy == \"skip_with_log\" and step.is_critical_path:\n            return False\n        return True\n```\n\nThe key insight: **don't ask the LLM to solve everything.** Use deterministic logic for structural decisions (can we retry? have we exhausted retries? is this critical?) and reserve LLM calls for genuinely ambiguous situations where contextual judgment is needed. Each LLM call should be bounded — few tokens, focused question, short response.\n\nYour agent's context window is its short-term memory. And it's a bad one.\n\nAt any given point, the context window contains:\n\nAs the plan executes, this window grows. At some point, it hits the token limit. What gets truncated? Usually, it's the oldest messages — which often includes the *original plan* and the *reasoning that produced it.*\n\nSo the agent is now executing step five, but it can no longer see why it chose the approach in step one. It's making decisions in a vacuum, optimizing for local correctness rather than global coherence.\n\nThis is the **amnesic agent problem**: the agent literally cannot remember its own rationale as it progresses through a long plan.\n\n**The fix:** Explicit **external memory management** — the plan and its rationale must be persisted outside the context window:\n\n```\nclass AgentMemory:\n    \"\"\"\n    External memory store decoupled from the LLM context window.\n    Persists plan state, rationale, and execution history.\n    \"\"\"\n\n    def __init__(self, store: KVStore):\n        self.store = store\n\n    def save_plan_context(self, run_id: str, context: PlanContext):\n        \"\"\"Persist full context for retrieval at any step.\"\"\"\n        self.store.set(f\"plan:{run_id}\", context, ttl=3600)\n\n    def retrieve_relevant_context(self, run_id: str, current_step: int, query: str) -> ContextSnapshot:\n        \"\"\"\n        Retrieve only the context relevant to the current decision point.\n        Uses embedding similarity to avoid loading everything.\n        \"\"\"\n        plan = self.store.get(f\"plan:{run_id}\")\n\n        # Load rationale for nearby steps, not all steps\n        window = range(max(0, current_step - 2), min(len(plan.steps), current_step + 1))\n        relevant_steps = [plan.steps[i] for i in window]\n\n        return ContextSnapshot(\n            original_intent=plan.intent,\n            relevant_steps=relevant_steps,\n            recent_outcomes=plan.executed_steps[-3:],\n            current_query=query\n        )\n```\n\nThis gives you two benefits: first, the agent always has access to its original intent; second, you can control *what* gets loaded into the context window at each step, keeping it lean and focused.\n\nMost agent frameworks treat tool use as a simple function-calling interface:\n\n``` python\n@tool(description=\"Search the knowledge base\")\ndef search_knowledge_base(query: str) -> str:\n    results = kb.search(query)\n    return format_results(results)\n```\n\nThe LLM sees the tool name and description and decides when to call it. But the description is always a natural language approximation of what the tool *actually does*. There's a semantic gap between \"search the knowledge base\" and the actual Elasticsearch query being constructed, the pagination logic, the relevance scoring, the error handling.\n\nWhen the LLM calls a tool with slightly wrong parameters, the result is wrong. When it calls the tool at the wrong time, the plan derails. When the tool returns an error the LLM doesn't recognize, the agent loops or hallucinates a fix.\n\n**The fix:** **Formal tool contracts** — every tool must declare its preconditions, postconditions, and error semantics:\n\n```\ninterface ToolContract {\n  name: string;\n  description: string;\n\n  // What must be true before calling\n  preconditions: precondition[];\n\n  // What the caller can expect after successful execution\n  postconditions: postcondition[];\n\n  // Exhaustive error taxonomy with recovery guidance\n  errors: ErrorCode[];\n\n  // Max latency the caller should expect\n  latencySLA: Duration;\n\n  // Whether results are cached (idempotent?)\n  caching: CachingPolicy;\n}\n\ninterface ErrorCode {\n  code: string;\n  message: string;\n  recovery: RecoveryStrategy; // \"retry\", \"abort\", \"ask_user\", etc.\n  hint: string; // What the LLM should try differently\n}\n```\n\nThis transforms tools from black boxes into verifiable components. The agent's execution engine can check preconditions before calling, interpret errors against a known taxonomy, and apply deterministic recovery strategies rather than hoping the LLM figures it out.\n\nPerhaps the most subtle failure mode: **agents don't learn from execution.**\n\nA plan is generated, executed, and if it fails, the LLM is shown the error and asked to continue. But the LLM doesn't *retain* anything from this failure. The next time a similar plan is generated, it makes the same mistake. The system has no memory of what went wrong, no model of its own failure modes, no way to improve.\n\nThis is especially damaging because LLMs have a well-documented **sycophancy problem** — they tend to double down on their initial reasoning rather than self-correct when presented with contradictory evidence. When an agent fails at step 3 and you feed the error back, the LLM might acknowledge the error but then proceed to make the same type of error at step 4.\n\n**The fix:** Build **execution feedback into the planning pipeline** as a first-class loop, not an afterthought:\n\n```\nclass ExecutionFeedbackLoop:\n    \"\"\"\n    Captures execution outcomes and feeds them back to improve planning.\n    Operates at two levels: online (per-session) and offline (cross-session).\n    \"\"\"\n\n    def __init__(self, session_store: SessionStore, improvement_engine: ImprovementEngine):\n        self.session = session_store\n        self.improve = improvement_engine\n\n    def process_outcome(self, plan: ExecutablePlan, outcome: ExecutionOutcome):\n        # Online: update the current plan with what we learned\n        adjusted_plan = self._adjust_plan(plan, outcome)\n\n        # Offline: accumulate patterns for systemic improvement\n        self.improve.record_f failure_pattern(outcome)\n\n        return adjusted_plan\n\n    def _adjust_plan(self, plan: ExecutablePlan, outcome: ExecutionOutcome) -> ExecutablePlan:\n        \"\"\"\n        Apply deterministic adjustments based on execution results.\n        Not an LLM call — just rule-based plan modification.\n        \"\"\"\n        adjusted = copy.deepcopy(plan)\n\n        if outcome.step_failed:\n            failed_step = outcome.failed_step\n\n            # Add explicit error handling to the failed step\n            failed_step.error_handling = self._infer_error_handler(failed_step, outcome)\n\n            # Add dependency on the failed step succeeding\n            for dependent in adjusted._find_dependents(failed_step.id):\n                adjusted._add_dependency(dependent.id, failed_step.id)\n\n        return adjusted\n```\n\nThe key distinction: the *online* loop makes adjustments to the current plan using deterministic rules (add retry, add error handling, add dependencies). The *offline* loop accumulates failure patterns across sessions and uses them to improve the planner itself — perhaps by fine-tuning, perhaps by improving system prompts, perhaps by building a failure-mode database that the planner consults.\n\nAll of these failure modes share a common root: the LLM is being asked to do something it's architecturally unsuited for — bridging the gap between high-level intent and low-level execution.\n\nThe solution is an **executive layer** — a software component that sits between the LLM (the planner) and the execution environment (the tools, APIs, databases). Its responsibilities are:\n\n```\ninterface ExecutiveLayer {\n  /** Receive a plan from the LLM, validate, and execute */\n  execute(plan: LLMGeneratedPlan): AsyncGenerator<ExecutionEvent>;\n\n  /** Check if the current system state supports the plan's assumptions */\n  validateAssumptions(plan: LLMGeneratedPlan): ValidationResult;\n\n  /** Recover from a failure without restarting the entire plan */\n  recover(step: FailedStep, error: ExecutionError): RecoveryPlan;\n\n  /** Log the full execution trace for debugging and improvement */\n  trace(): ExecutionTrace;\n}\n```\n\nThis layer is *not* an LLM. It's deterministic software. It doesn't need to be smart — it needs to be correct. The LLM provides the creativity and adaptation. The executive layer provides the reliability and accountability.\n\nIf you're building AI agents today, you're almost certainly under-instrumented. The gap between \"the agent planned this\" and \"the agent executed this correctly\" is where your bugs live. Here's where to focus:\n\n**Immediate fixes (this week):**\n\n**Medium-term (this sprint):**\n\n**Longer-term (this quarter):**\n\n**Q: Should I use a smaller model for the execution layer?**\n\nA: No — the executive layer shouldn't use an LLM at all. It should be deterministic code. If you find yourself needing an LLM inside your execution loop, you've likely pushed responsibility onto the model that should belong to your software architecture. The LLM belongs in planning and in genuinely ambiguous error resolution, not in the hot path of execution.\n\n**Q: How do I handle plans that legitimately need LLM judgment during execution?**\n\nA: Limit these to bounded, isolated calls with clear input/output contracts. When the execution engine encounters an ambiguous error state, it should collect all relevant context, make a single focused LLM call (not a free-form conversation), and treat the response as a recommendation — not a command. The executive layer should always be able to override or reject the LLM's suggestion.\n\n**Q: Isn't all of this over-engineering? Why can't the LLM just do it?**\n\nA: Because the LLM *can't*. Not reliably. Not repeatedly. Not at scale. The same properties that make LLMs flexible — probabilistic token generation, context window limitations, lack of persistent state, no native understanding of side effects — are the same properties that make them unreliable as execution engines. You could keep pushing prompt engineering further, but you'll hit a wall. The wall is the difference between *saying* something and *doing* something. Closing that gap requires engineering, not prompting.", "url": "https://wpnews.pro/news/why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm", "canonical_source": "https://dev.to/tamizuddin/why-your-ai-agent-cant-execute-its-own-plan-bridging-the-gap-between-local-llm-intelligence-and-45pc", "published_at": "2026-08-23 12:01:28+00:00", "updated_at": "2026-08-23 12:13:38.508151+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["tamiz.pro"], "alternates": {"html": "https://wpnews.pro/news/why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm", "markdown": "https://wpnews.pro/news/why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm.md", "text": "https://wpnews.pro/news/why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm.txt", "jsonld": "https://wpnews.pro/news/why-your-ai-agent-can-t-execute-its-own-plan-bridging-the-gap-between-local-llm.jsonld"}}