Planning Over Execution: Lessons from 157 Agent Runs and the Rise of Orca-Style Agent Fleets

A developer reports that after 157 agent deployments, planning quality predicts success far better than execution speed or model size. Systems investing 3-5x more tokens in planning achieved 4.2x higher task completion rates and 3.8x fewer rollback cycles. The findings support a hierarchical 'Orca-style' architecture separating strategic planning from specialized execution.

Originally published on tamiz.pro. The field of AI agents has moved rapidly from single-model executors to complex multi-agent orchestration. But after running 157 agent deployments across diverse task domains, one pattern emerged with striking consistency: planning quality predicts success far better than execution speed or model size. This isn't just theoretical—it's a practical lesson that's reshaping how engineers architect agent fleets, giving rise to what we're now calling Orca-style agents : hierarchical, planning-first systems that separate the expensive business of thinking from the cheaper business of doing. Over six months, our team deployed and monitored 157 distinct agent runs across four primary use cases: code generation pipelines, automated testing workflows, infrastructure-as-code provisioning, and data transformation tasks. Each run varied along three dimensions: The results were unambiguous. Systems that invested 3-5x more tokens in planning achieved 4.2x higher task completion rates and 3.8x fewer rollback cycles compared to agents optimized purely for fast execution. The correlation between planning sophistication and success held across every domain. The intuition behind this finding rests on an economic principle of LLM usage: planning is cheap relative to costly mistakes . A well-structured plan reduces the probability of executing the wrong sequence of tools, making incorrect API calls, or generating code that fails integration testing. Consider the token economics: | Phase | Tokens typical | Cost impact | |---|---|---| | Planning intent + decomposition | 800–2,500 | Low | | Execution per subtask | 300–1,200 | Medium | | Correction after failure | 1,500–4,000 | High | Agents that plan thoroughly front-load their costs. Those that rush to execute often pay exponentially more in corrections, retries, and human intervention. The name "Orca" comes from the hierarchical social structure of killer whales: a single matriarch orchestrates, while specialized pod members execute discrete tasks. In agent terms, this translates to: Strategic Planner the matriarch : Holds global context, decomposes goals, assigns subtasks, and validates outcomes. Runs on a stronger model with longer context windows. Specialist Executors the pod : Each handles a narrow domain—code generation, test writing, documentation, validation. Run on smaller, cheaper models optimized for throughput. Shared Memory Layer : A structured knowledge graph or vector store that maintains state across the fleet, preventing redundant work and enabling cross-agent learning. Orchestration Loop : A lightweight controller that routes tasks, aggregates results, and triggers replanning when validation fails. The critical insight is that not all thinking is equal . Strategic decisions—understanding requirements, identifying edge cases, sequencing dependencies—benefit from deep context and reasoning. Tactical decisions—formatting output, calling a specific API, generating a template—are better handled by focused, optimized models. Separating these concerns allows you to: From the 157 runs, several implementation patterns emerged as particularly effective: The planner decomposes a goal into subtasks, each with explicit success criteria. Executors complete subtasks and return structured evidence of completion. The planner validates before proceeding to the next level. php class OrcaPlanner: async def decompose self, goal: str, context: Dict - List Subtask : """Recursive planning with validation gates.""" plan = await self.model.plan goal, context=context validated subtasks = for subtask in plan.subtasks: if subtask.requires decisions : Recursive planning for complex subtasks sub plan = await self.decompose subtask.description, context validated subtasks.extend sub plan else: validated subtasks.append subtask return validated subtasks Each executor carries a "skill card"—a concise description of its capabilities, constraints, and preferred input/output formats. The planner matches subtasks to specialists based on these cards rather than attempting blind routing. Instead of passing raw conversation history, agents exchange structured context objects: constraints discovered, assumptions made, partial results, and confidence scores. This enables better downstream planning and reduces context window waste. Not every design decision landed well. Here are the patterns that correlated with failure: Some teams spent so much time planning that the plan became stale before execution began. The sweet spot was ~20% of total token budget for planning, with dynamic replanning triggered only by validation failures—not on a timer. Creating too many specialists 15+ introduced routing overhead and context fragmentation. The optimal range was 4–8 specialists, each covering a distinct capability domain. When execution failed, some systems silently retried with minor variations. Successful systems explicitly logged failures, triggered replanning at the appropriate abstraction layer, and maintained an audit trail. Planning agents that retained full conversation history from execution agents burned through context windows unnecessarily. Successful implementations used summarized state objects instead. These systems aren't a universal upgrade. Based on the data: | Use Case | Recommendation | |---|---| | Single-shot code generation | Single agent sufficient | | Multi-step workflows with validation | Orca-style strongly recommended | | High-stakes operations infra, payments | Orca-style with human-in-the-loop | | Batch processing 100+ items | Hybrid: planner + parallel executors | | Interactive assistance | Lightweight planning, direct execution | The break-even point appears around 3–5 sequential steps with dependency checking. Below that, the planning overhead outweighs the benefits. For engineers evaluating whether to adopt Orca-style architectures, the cost model matters: Traditional approach : One large model handles everything. Cheap for simple tasks, expensive for complex ones due to rework. Orca approach : Premium model for planning, lighter models for execution. Higher upfront cost, dramatically lower correction costs. In our measurements, the Orca architecture showed a 37% reduction in total token cost for tasks exceeding 10 steps, despite using more expensive models for planning. The savings came from reduced retry loops and fewer human escalations. The 157-agent study reveals a fundamental shift in how we should think about agentic systems. The question is no longer "how fast can the agent execute?" but "how well can the agent plan?" This reframing is driving a new generation of tools focused on planning quality: structured decomposers, validation-aware schedulers, and context-efficient state management. As the field matures, we'll likely see planning becomes a first-class concern in agent frameworks, with libraries and patterns emerging specifically for the strategic layer. The Orca architecture isn't just a pattern—it's an acknowledgment that thinking carefully is the most important part of the job . Q: Can I implement Orca-style planning without a full multi-agent framework? A: Yes. Start with a single planner function that decomposes goals and validates each step before execution. You don't need separate agents initially—a well-structured planning loop within a single process captures 80% of the benefit. Q: How do I choose which model to use for planning vs. execution? A: Planning benefits from strong reasoning and long context e.g., GPT-4-class models . Execution can use smaller, faster models optimized for specific tasks. The key is matching model capability to cognitive demand, not cost alone. Q: What's the minimum viable Orca architecture? A: Three components: 1 a planner that decomposes goals into sequenced subtasks with success criteria, 2 an executor that runs subtasks and returns structured results, and 3 a validator that checks completion evidence before allowing progression. This fits in under 200 lines of code.