Most multi-agent demos work great in a notebook and fall apart within two weeks of production traffic. Not because the model is wrong because the orchestration layer was never designed for the failure modes that only show up at scale: partial tool failures, context drift across long conversations, and agents that silently disagree with each other.
Here's what actually breaks, and the patterns that fix it.
A single-agent system that hits an API timeout usually throws an error you can catch. A multi-agent system where Agent A calls Agent B, which calls a tool that returns a malformed but parseable response, tends to produce output that looks reasonable and is wrong. This is the failure mode that costs you trust, not uptime.
The fix isn't more prompt engineering — it's treating every inter-agent handoff like an API contract:
``` python
class AgentResponse(BaseModel):
result: dict
confidence: float
tool_calls: list[ToolCall]
validation_status: Literal["verified", "unverified", "failed"]
def handoff(agent_output: AgentResponse) -> AgentResponse:
if agent_output.validation_status != "verified":
return escalate_to_human_or_retry(agent_output)
return agent_output
```
Every handoff between agents should validate against a schema before the next agent trusts it. This sounds obvious written down; it's the single most common thing missing from agent frameworks used as-is out of the box.
Prompt engineering gets the attention, but the thing that determines whether your agent system survives a 40-turn conversation is how you manage state across agents that each have partial visibility into the task.
Three patterns, in order of how much complexity they can handle:
When two agents produce conflicting outputs (a common failure once you have more than 2–3 agents with overlapping responsibility), most systems just take whichever one finishes last. That's not a resolution strategy, it's a race condition wearing a costume.
Decide up front, per workflow: does disagreement trigger (a) a third arbitration pass, (b) escalation to a human, or (c) a deterministic tie-break rule? Pick one and encode it — don't let it emerge by accident.
Multi-agent orchestration isn't a prompting problem. It's a distributed-systems problem wearing an AI costume — the same discipline you'd apply to any system with partial failures, async handoffs, and conflicting writers applies here, almost unchanged. If you're building this and it feels like you're reinventing microservices patterns, that's because you are.
What's breaking in your agent systems? Curious what patterns others have landed on for state management specifically — drop it in the comments.