Splitting one big agent into a team of specialists feels like good engineering. A planner breaks the task down, a researcher gathers data, a writer drafts, a reviewer approves. Clean separation of concerns. Smaller prompts. Easier to reason about.
Then it ships, and something weird happens: every individual agent looks fine in testing, yet the system as a whole produces wrong answers you can't explain.
The reason is that multi-agent systems mostly fail in the handoffs between agents, not inside any single one. Unclear roles, lost context, a reviewer that quietly stopped reviewing. None of that shows up when you test agents one at a time.
This is Part 1 of a three-part series on monitoring multi-agent systems:
With one agent, one model plans and acts. You read the trace, top to bottom, and the story is right there. Add more agents and three new problems appear.
Each agent often runs in its own process, container or service. Without shared trace context, one user task turns into four disconnected traces in your backend. You can see the researcher's trace and the writer's trace, but nothing tells you they belong together.
trace a1f3… planner ✅ 1.2s
trace 7c09… researcher ✅ 4.8s
trace e44b… writer ✅ 3.1s
trace 2d8a… reviewer ✅ 0.6s
When the final output is wrong, the mistake could sit in any agent, or in a handoff between two of them. Working out which agent caused a failure, and at which step, is hard enough that it's an active research topic.
This is the dangerous one. A small mistake from one agent becomes the next agent's trusted input. The researcher grabs a stale figure; the writer builds a confident paragraph around it; the reviewer checks the prose, not the source. By the end, the original error is buried under several layers of competent-looking work.
It's the same compounding problem single agents have, except now each step is a whole agent with its own reasoning, and none of them is positioned to question what they were handed.
You don't have to guess. Researchers at UC Berkeley annotated more than 1,600 execution traces from seven popular multi-agent frameworks and identified 14 distinct failure modes, grouped into three categories: specification issues, inter-agent misalignment, and task verification (Cemri et al., NeurIPS 2025).
One finding is worth sitting with: many failures came from system design, not just model limits. A better model won't fix a team where nobody owns verification.
Here's how those categories translate into things you can actually monitor:
| Category | What goes wrong | What to monitor |
|---|---|---|
| Specification & design | Unclear roles, ignored constraints, repeated steps | Role violations, duplicate work across agents, steps per agent per task |
| Inter-agent misalignment | Lost context, withheld information, agents talking past each other, conversation resets | Handoff content vs. original task, clarification requests, context dropped between agents |
| Task verification | Ending too early, never ending, no real check of the result | Termination reason, loops between agents, whether the verifier actually checked anything |
Notice how much of the right-hand column is about what passes between agents. You can't monitor any of it unless you record the handoffs.
For every handoff, capture three things:
Many multi-agent failures are plain miscommunication: the task that arrived isn't the task that was sent. Comparing the two is often the fastest route to the root cause.
Here's a minimal schema:
from dataclasses import dataclass, field
from datetime import datetime, timezone
import uuid
@dataclass
class Handoff:
trace_id: str
sender: str # "planner"
receiver: str # "researcher"
task_sent: str # what the sender asked for
task_understood: str | None # receiver's restatement (filled on receipt)
context_keys: list[str] # which pieces of context were passed
context_tokens: int
permissions: list[str] # tools/data the receiver may use for this task
original_goal: str # the user's goal, carried end to end
handoff_id: str = field(default_factory=lambda: uuid.uuid4().hex)
ts: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
The trick that makes this work is task_understood. Ask each receiving agent to restate its task in one sentence before it starts work, and log that restatement. It costs a few tokens and gives you something concrete to compare against.
RESTATE_PROMPT = """Before you start, restate in ONE sentence the task you
have been given and the constraints you must respect. Output only that sentence."""
def receive(handoff: Handoff, agent) -> Handoff:
handoff.task_understood = agent.complete(
system=agent.system_prompt,
messages=[{"role": "user", "content": f"{handoff.task_sent}\n\n{RESTATE_PROMPT}"}],
)
log_handoff(handoff)
return handoff
Once you have task_sent, task_understood and original_goal, you can score how far the task drifted at every hop. Embedding similarity is a decent first pass:
import numpy as np
def cosine(a, b):
return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
def handoff_drift(h: Handoff, embed) -> dict:
sent, understood, goal = map(embed, (h.task_sent, h.task_understood, h.original_goal))
return {
"sent_vs_understood": cosine(sent, understood), # miscommunication
"understood_vs_goal": cosine(understood, goal), # drift from user intent
}
scores = handoff_drift(h, embed)
if scores["sent_vs_understood"] < 0.80 or scores["understood_vs_goal"] < 0.70:
flag(h.trace_id, reason="handoff_task_mismatch", **scores)
The thresholds are placeholders. Calibrate them on a few dozen handoffs you've labelled by hand. And for high-risk flows, an LLM judge asking "does the received task preserve every constraint in the original goal?" catches dropped constraints that embeddings will miss (a missing "under $500" barely moves a vector).
original_goal: "Summarise Q3 churn for EU enterprise customers only"
planner → researcher
task_sent: "Pull Q3 churn data for enterprise customers in the EU"
task_understood: "Pull Q3 churn data for enterprise customers" ⚠️ region dropped
sent_vs_understood: 0.91
understood_vs_goal: 0.74 → flagged
Every downstream agent did its job well. The report was polished, the numbers were real. They were just the wrong numbers, because a constraint fell out at the first hop. Without the handoff record, you'd be debugging the writer.
Monitoring and design reinforce each other. A few habits make multi-agent systems easier to observe and less likely to fail:
In Part 2, we'll stitch every agent into one trace per task with OpenTelemetry, give handoffs their own spans, and write a small root-cause walker that finds the agent that actually broke the task.
If you want the full picture in one place, this guide on what to track when monitoring multi-agent systems covers system, role, handoff and security metrics together.
Have you seen a constraint quietly disappear between two agents? Tell me where it went missing. 🤝