Originally published on tamiz.pro.
The initial promise of Large Language Models (LLMs) centered on single-turn Q&A. Today, the frontier has shifted to Multi-Agent Systems (MAS)—orchestrations of specialized agents that can reason, code, retrieve, and act autonomously. While these systems unlock exponential capability gains, they introduce a new class of failure modes: compounding errors and hallucination propagation.
When one agent hallucinates a non-existent API endpoint, and three subsequent agents build complex logic based on that falsehood, the system doesn't just fail—it fails convincingly. This deep dive explores the architectural patterns, guardrails, and operational lessons learned from deploying MAS at production scale.
In single-agent systems, a hallucination is typically an isolated error. In multi-agent setups, it becomes a virus. We call this Hallucination Drift.
Consider a financial analysis agent swarm:
The user receives a perfectly formatted, confident, and entirely wrong answer. The severity scales with the depth of the chain and the authority of the final output.
LLMs are auto-regressive token predictors, not deterministic databases. They optimize for plausibility, not truth. When an agent provides context to a downstream agent, the downstream model inherits the epistemic uncertainty of the upstream model—and usually amplifies it through "sycophancy," agreeing with the presumed context provided by the system prompt.
To harden MAS, we must move beyond prompt engineering and into system-level design. Here are the three most effective architectural patterns.
Instead of a linear pipeline (Agent A → Agent B → Output), implement a Skeptic Node. Before any agent acts on another agent’s output, a dedicated critic agent evaluates the factual grounding and logical consistency.
def skeptic_loop(upstream_output: str, facts: list[dict]) -> str:
verification_prompt = f"""
Context: {upstream_output}
Ground Truth: {facts}
Identify any unsupported claims or hallucinations.
Return 'VALID' if supported, or list contradictions if not.
"""
result = llm_call(verification_prompt)
return result
If the Skeptic returns contradictions, the system should either: (a) loop back to the Researcher Agent with specific feedback, or (b) flag the section as uncertain for the user.
Never allow LLM-generated content to execute directly if it touches sensitive operations (database writes, code execution, API calls). Use Deterministic Fences to validate structure and intent before execution.
pydantic
or Zod
. If the agent outputs invalid JSON, reject it immediately.Hallucinations often arise when agents lose track of prior context or make incorrect assumptions about shared state. Implement a Centralized State Store (e.g., Redis, Vector DB) that agents query explicitly rather than relying on context window memory.
You cannot harden what you cannot measure. Traditional logging is insufficient for MAS; you need trace-based observability.
Use OpenTelemetry or LangSmith to trace requests across agent boundaries. Each node in the trace should include:
This enables root cause analysis when hallucinations occur: "Which agent introduced the error? Was it a retrieval failure or a reasoning failure?"
Several frameworks now offer built-in hardening features:
Q: Is it possible to eliminate hallucinations entirely in MAS?
A: No. Hallucination is an inherent property of probabilistic language models. The goal is mitigation through architectural controls, not elimination. Always design for failure modes.
Q: How do I balance speed vs. safety in agent loops?
A: Use confidence thresholds. Low-confidence outputs trigger longer Skeptic Loops; high-confidence outputs bypass validation. This optimizes cost and latency while maintaining safety for critical paths.
Q: Should I use separate models for reasoning vs. validation?
A: Often yes. Using a smaller, faster model for validation and a larger, more capable model for reasoning can reduce costs while maintaining accuracy. However, ensure the validation model is not biased by the reasoning model’s style.
Building multi-agent AI systems is less about prompting and more about control theory. By implementing Skeptic Loops, Deterministic Fences, and robust observability, engineers can transform fragile agent swarms into reliable, production-grade systems. The future of AI lies not in bigger models, but in better architectures.