Agentic Systems: From Single Agent to Orchestration

A single agent — one model, a system prompt, a handful of tools, and a loop — gets you surprisingly far. It's also where most people stop, because it works for demos and falls over the moment a task needs more than one kind of reasoning happening at once.

Multi-agent orchestration isn't a hype upgrade over a single agent — it's what you reach for when a task naturally decomposes into sub-tasks that benefit from different context, different tools, or genuinely different roles. This post covers when that shift is warranted, the orchestration patterns actually used in production, and a small working example of a coordinator delegating to worker agents.

A single agent's context window and tool list grow with every capability you bolt on. Three failure modes show up as that happens:

Orchestration solves this by splitting one large, blurry job into a coordinator plus specialized workers, each with a narrow role, its own context, and only the tools it actually needs.

There isn't one "multi-agent architecture" — a few concrete patterns cover most real systems:

Frameworks like LangGraph model this as an explicit state graph, CrewAI models it as role-based "crews," and minimal SDKs like the OpenAI Agents SDK treat sub-agents as callable tools from a parent agent. The underlying pattern is the same regardless of framework — what differs is how much structure and observability you get by default.

Here's a minimal hierarchical setup: a coordinator agent receives a research request, decomposes it into sub-tasks, dispatches them to worker agents concurrently, and merges the results.

use futures::future::join_all;
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct SubTask {
    id: String,
    description: String,
    assigned_role: AgentRole,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
enum AgentRole {
    Researcher,
    CodeAnalyst,
    Summarizer,
}

#[derive(Debug, Serialize)]
struct WorkerResult {
    task_id: String,
    output: String,
    tokens_used: u32,
}

struct Coordinator {
    llm_client: LlmClient,
}

impl Coordinator {
    /// Decompose a high-level request into sub-tasks using the coordinator's own LLM call.
    async fn plan(&self, request: &str) -> Vec<SubTask> {
        let planning_prompt = format!(
            "Break this request into 2-4 independent sub-tasks, each assigned \
             to Researcher, CodeAnalyst, or Summarizer. Request: {request}"
        );
        let response = self.llm_client.complete(&planning_prompt).await;
        parse_subtasks(&response) // structured output parsing, omitted for brevity
    }

    /// Dispatch sub-tasks to worker agents concurrently, then merge.
    async fn execute(&self, request: &str) -> String {
        let subtasks = self.plan(request).await;

        let worker_futures = subtasks.into_iter().map(|task| {
            let client = self.llm_client.clone();
            async move { run_worker(client, task).await }
        });

        let results: Vec<WorkerResult> = join_all(worker_futures).await;

        self.merge(results).await
    }

    /// A final LLM call that synthesizes worker outputs into one coherent answer.
    async fn merge(&self, results: Vec<WorkerResult>) -> String {
        let combined = results.iter()
            .map(|r| format!("[{}]: {}", r.task_id, r.output))
            .collect::<Vec<_>>()
            .join("\n\n");

        let merge_prompt = format!(
            "Synthesize these sub-task results into one coherent response:\n\n{combined}"
        );
        self.llm_client.complete(&merge_prompt).await
    }
}

async fn run_worker(client: LlmClient, task: SubTask) -> WorkerResult {
    let role_prompt = match task.assigned_role {
        AgentRole::Researcher => "You are a focused researcher. Be concise and cite sources.",
        AgentRole::CodeAnalyst => "You are a code analyst. Focus only on technical accuracy.",
        AgentRole::Summarizer => "You are a summarizer. Compress without losing key facts.",
    };

    let prompt = format!("{role_prompt}\n\nTask: {}", task.description);
    let output = client.complete(&prompt).await;

    WorkerResult {
        task_id: task.id,
        output,
        tokens_used: estimate_tokens(&output),
    }
}

A few things worth calling out in this shape:

role_prompt

) instead of sharing the coordinator's full context — this is the actual mechanism that fixes context pollution.join_all

runs workers concurrentlyMulti-agent orchestration isn't free — it's worth naming the costs explicitly, since a lot of write-ups skip this part:

The Gartner-cited pattern worth taking seriously: a meaningful share of agentic projects get cancelled not because agents don't work, but because teams reached for orchestration before the task actually needed it, and the coordination overhead outweighed the gain.

Reach for a single agent by default. Move to orchestration when at least one of these is true:

If none of those apply, a single well-designed agent with good tool selection will usually outperform a multi-agent system on both cost and reliability.

The next post in this series steps back to a cost/control framework: when running open-weight models yourself makes sense versus calling a closed API, and how to reason about that tradeoff instead of defaulting to whichever one is trending.

This is part of a series on AI infrastructure patterns, drawing on production work building multi-provider AI gateways in Rust. Follow along for the rest of the series.