We have all hit the "monolithic LLM wall."
You design an incredibly capable AI agent, arm it with a suite of tools, and give it a complex, multi-step task—like writing a comprehensive technical paper complete with data analysis, web research, and code verification. At first, it works beautifully. But as the steps accumulate, the context window fills up. The agent begins to experience "attention drift." It forgets its original instructions, hallucinates tool outputs, and eventually spins out of control, burning through millions of tokens and your API budget.
The problem isn't the LLM's reasoning capacity; it’s the architecture. Trying to solve a complex, multi-domain problem within a single agent’s context window is the modern software equivalent of writing an entire enterprise application inside a single, monolithic main()
function.
To build AI systems that can scale to handle real-world complexity, we must transition from monolithic agents to hierarchical multi-agent orchestration.
By decomposing complex goals into isolated, specialized sub-agents—each operating within its own bounded context and resource budget—we can build resilient, self-improving AI systems that scale indefinitely.
In this post, we will dive deep into the architectural patterns of multi-agent orchestration, explore how to manage agent lifecycles, and write production-grade Python code to spawn and supervise sub-agents.
(The concepts and code demonstrated here are drawn from my ebook Hermes Agent, The Self-Evolving AI Workforce)
Multi-agent orchestration is not just a design convenience; it is an architectural necessity. The theoretical foundation of this approach rests on two pillars: task decomposition and supervisory control. Together, they transform a monolithic agent into a scalable, resilient hierarchy of specialized workers.
Think of a master carpenter building a custom cabinet. The master does not personally cut every dovetail, sand every surface, or install every hinge. Instead, she decomposes the project into distinct sub-tasks: joinery, finishing, and hardware installation.
For each sub-task, she assigns an apprentice with the right tools and expertise. She monitors their progress, checks their quality, and integrates their individual outputs into the final product. If an apprentice hits a snag, she intervenes, provides guidance, or reassigns resources.
In this scenario, the parent agent is the master carpenter, and the sub-agents are the apprentices. Each apprentice operates with their own focused toolset and an independent iteration budget.
+------------------+
| Parent Agent | <-- Master Carpenter (Supervisor)
+--------+---------+
|
+------------------+------------------+
| | |
+--------v-------+ +--------v-------+ +--------v-------+
| Sub-Agent A | | Sub-Agent B | | Sub-Agent C | <-- Apprentices (Workers)
| (Web Searcher) | | (Code Builder) | | (Doc Writer) |
+----------------+ +----------------+ +----------------+
In software engineering, this pattern is everywhere:
Multi-agent orchestration applies these exact principles to AI. The parent agent acts as the Kubernetes orchestrator or OS kernel, sub-agents act as independent processes or microservices, and persistent memory serves as the shared state store.
The parent-agent supervisor pattern is the architectural heart of multi-agent systems. The parent agent (the primary orchestrator instance) is responsible for managing the entire lifecycle of the operation:
This pattern closely mirrors the supervisor-worker model in Erlang/OTP, where supervisor processes monitor worker processes and handle failures gracefully. If a sub-agent fails or gets stuck in an infinite loop, the parent agent can catch the failure, reclaim the resources, and either spawn a replacement or adapt its plan.
One of the biggest risks in autonomous agent systems is the "infinite loop" bug—where an agent repeatedly calls a failing tool or gets stuck in a reasoning loop, draining your API keys. When agents start spawning other agents, this risk multiplies exponentially.
To solve this, we implement a thread-safe, per-agent Iteration Budget.
class IterationBudget:
"""Thread-safe iteration counter for an agent.
Each agent (parent or subagent) gets its own IterationBudget.
The parent's budget is capped at max_iterations (default 90).
Each subagent gets an independent budget capped at
delegation.max_iterations (default 50) — this means total
iterations across parent + subagents can exceed the parent's cap.
"""
An elegant design pattern here is the concept of budget refunds for programmatic execution.
If a sub-agent calls a tool to run a Python script (execute_code
) that takes several steps to execute, those purely computational steps should not consume the agent's reasoning budget. The agent’s "thinking" budget (deciding what to do) should be strictly separated from its "acting" budget (running computations).
By refunding iterations spent on raw code execution, we ensure that complex computational tasks do not penalize the agent's cognitive allocation.
Sub-agents must operate in isolated contexts to keep prompt sizes small, but they still need a way to share state with the parent and their sibling agents. This is achieved through persistent memory—a file-based storage system that survives agent restarts.
This architecture is based on the classical AI Blackboard Pattern:
+-------------------------------------------------------+
| PERSISTENT BLACKBOARD |
| (Shared File-Based Memory) |
+---------------------------^---------------------------+
|
+------------------+------------------+
| | |
+--------v-------+ +--------v-------+ +--------v-------+
| Sub-Agent A | | Sub-Agent B | | Sub-Agent C |
| Writes Search | | Reads Search | | Reads Code |
| Results | | Writes Code | | Writes Final |
| | | Artifacts | | Report |
+----------------+ +----------------+ +----------------+
~/.hermes/
).To prevent memory bloat, a Streaming Context Scrubber is used to compress and summarize large sub-agent outputs before they are passed back up to the parent, keeping the parent's context window clean and focused on high-level strategy.
The true power of this architecture emerges when we apply closed learning loops recursively.
In a multi-agent system, optimization occurs at two distinct layers:
This is the AI equivalent of meta-learning—the system doesn't just get better at doing tasks; it gets better at delegating them.
Let’s translate these theoretical foundations into production-grade Python code.
Below is a complete, robust implementation of a parent agent supervisor that initializes a persistent session database, builds a specialized sub-agent configuration, and manages sub-agent execution.
#!/usr/bin/env python3
"""
Production-Grade Parent-Agent Supervisor and Sub-Agent Spawner.
"""
import logging
import asyncio
import json
from typing import Dict, List, Any, Optional
from pathlib import Path
class IterationBudget:
def __init__(self, limit: int):
self.limit = limit
self.used = 0
def consume(self, amount: int = 1):
self.used += amount
if self.used > self.limit:
raise TimeoutError("Iteration budget exceeded!")
class AIAgent:
def __init__(self, **kwargs):
self.config = kwargs
self.session_id = kwargs.get("session_id")
self.budget = IterationBudget(kwargs.get("max_iterations", 50))
async def run_conversation(self, prompt: str) -> Dict[str, Any]:
await asyncio.sleep(1)
self.budget.consume(5) # Simulate consuming 5 iterations of reasoning
return {
"status": "success",
"output": f"Processed prompt: '{prompt}' using model {self.config.get('model')}",
"iterations_used": self.budget.used
}
class SessionDB:
def __init__(self, db_path: Path):
self.db_path = db_path
self.db_path.mkdir(parents=True, exist_ok=True)
self.sessions_file = self.db_path / "sessions.json"
if not self.sessions_file.exists():
self.sessions_file.write_text("{}")
def ensure_tables(self):
pass
def upsert_session(self, session_id: str, metadata: Dict[str, Any]):
data = json.loads(self.sessions_file.read_text())
data[session_id] = metadata
self.sessions_file.write_text(json.dumps(data, indent=4))
print(f"💾 Session '{session_id}' persisted to database.")
def get_hermes_home() -> Path:
home = Path.home() / ".hermes"
home.mkdir(exist_ok=True)
return home
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("MultiAgentOrchestrator")
parent_config = {
"base_url": "https://api.openai.com/v1",
"api_key": "sk-mock-key",
"model": "gpt-4o",
"provider": "openai",
"api_mode": "chat",
"max_iterations": 90, # Parent gets a generous budget
"tool_delay": 1.0, # Rate-limiting safety delay
"enabled_toolsets": ["filesystem", "web", "terminal", "code_execution"],
"save_trajectories": True,
"session_id": "supervisor_session_101",
}
parent_agent = AIAgent(
base_url=parent_config["base_url"],
api_key=parent_config["api_key"],
model=parent_config["model"],
provider=parent_config["provider"],
api_mode=parent_config["api_mode"],
max_iterations=parent_config["max_iterations"],
tool_delay=parent_config["tool_delay"],
enabled_toolsets=parent_config["enabled_toolsets"],
save_trajectories=parent_config["save_trajectories"],
session_id=parent_config["session_id"],
)
logger.info(f"Supervisor Agent Initialized. Model: {parent_config['model']} | Session: {parent_config['session_id']}")
hermes_home = get_hermes_home()
session_db = SessionDB(db_path=hermes_home / "sessions")
session_db.ensure_tables()
session_db.upsert_session(
session_id=parent_config["session_id"],
metadata={
"role": "supervisor",
"model": parent_config["model"],
"max_iterations": parent_config["max_iterations"],
"status": "active"
}
)
SUB_AGENT_MODEL = "gpt-4-mini" # Using a faster, cheaper model for sub-agents
SUB_AGENT_MAX_ITERATIONS = 50 # Capped iteration budget for safety
def build_sub_agent_config(task_slug: str, specialized_tools: List[str]) -> dict:
"""
Generates a tailored configuration for a specialized sub-agent.
"""
sub_session_id = f"{parent_config['session_id']}_sub_{task_slug}"
return {
"base_url": parent_config["base_url"],
"api_key": parent_config["api_key"],
"model": SUB_AGENT_MODEL,
"provider": parent_config["provider"],
"api_mode": "chat",
"max_iterations": SUB_AGENT_MAX_ITERATIONS,
"tool_delay": 0.5,
"enabled_toolsets": specialized_tools, # Restrict tools to only what is needed!
"save_trajectories": True,
"session_id": sub_session_id,
}
async def orchestrate_sub_task(task_name: str, prompt: str, tools: List[str]) -> Dict[str, Any]:
"""
Spawns, executes, tracks, and terminates a sub-agent.
"""
logger.info(f"🚀 Spawning sub-agent for task: [{task_name}]")
sub_config = build_sub_agent_config(task_name, tools)
session_db.upsert_session(
session_id=sub_config["session_id"],
metadata={
"role": f"worker_{task_name}",
"parent_session_id": parent_config["session_id"],
"model": sub_config["model"],
"max_iterations": sub_config["max_iterations"],
"status": "spawned"
}
)
sub_agent = AIAgent(**sub_config)
try:
logger.info(f"Delegating task to sub-agent [{sub_config['session_id']}]...")
result = await sub_agent.run_conversation(prompt)
session_db.upsert_session(
session_id=sub_config["session_id"],
metadata={"status": "completed", "iterations_used": result["iterations_used"]}
)
logger.info(f"✅ Sub-agent [{task_name}] completed successfully.")
return result
except Exception as e:
logger.error(f"❌ Sub-agent [{task_name}] failed: {str(e)}")
session_db.upsert_session(
session_id=sub_config["session_id"],
metadata={"status": "failed", "error": str(e)}
)
raise e
finally:
logger.info(f"🧹 Terminating sub-agent [{sub_config['session_id']}] and cleaning up resources.")
async def main():
print("\n--- Starting Multi-Agent Orchestration Demo ---\n")
tasks = [
{
"name": "research",
"prompt": "Search the web for the latest advancements in solid-state batteries.",
"tools": ["web"]
},
{
"name": "analysis",
"prompt": "Analyze the research data and generate a Python script to model efficiency curves.",
"tools": ["filesystem", "code_execution"]
}
]
for task in tasks:
try:
result = await orchestrate_sub_task(
task_name=task["name"],
prompt=task["prompt"],
tools=task["tools"]
)
print(f"Result Output: {result['output']}\n")
except Exception:
print(f"Skipping downstream tasks due to failure in task: {task['name']}")
if __name__ == "__main__":
asyncio.run(main())
If you are designing a multi-agent system, keep these core architectural principles in mind:
Leave a comment below with your experiences, and let’s build more resilient AI systems together!
The concepts and code demonstrated here are drawn directly from the comprehensive roadmap laid out in the ebook Hermes Agent, The Self-Evolving AI Workforce: details link, you can find also my programming ebooks with AI here: Programming & AI eBooks.