If you're building LLM agents, this is a huge reminder: reading the code isn't enough. You need actual spans and traces to see where the logic is drifting.
The DevSwarm Agent Architecture #
The swarm splits tasks across specialized open-weight models to maximize efficiency:
Planner (GLM-5.2): Converts prompts into typed build plans and API contracts.Frontend (GLM-5.2): Generates a self-contained index.html.Backend (Qwen3-Coder-480B): Builds an Express server based on the contract.Critic (Kimi-K2.7-Code): Reviews both for security and bugs; gates the merge.Doctor (GLM-5.2): Analyzes the swarm's own traces to repair model routing.
The "Critic" is the most critical part of the AI workflow. It acts as a quality gate, routing errors back to the specific agent that caused them for a patch. If it fails after two rounds, it ships with a verdict.
Implementing Telemetry from Scratch #
The team prioritized observability before polishing the code because multi-agent systems fail silently. A request might "succeed" technically, but produce a broken artifact, or a fallback might kick in so seamlessly that you don't realize your primary model is down.
To get this running, they used a self-hosted SigNoz setup. For those looking for a practical tutorial on deployment, they managed their configuration via a casting.yaml
file to ensure the entire observability stack was reproducible.
They tracked every model call as a span named llm.
, following GenAI semantic conventions. This allowed them to move from "I think the model is hallucinating" to "I can see the exact span where the latency tripled and the token count spiked," making the debugging process data-driven rather than anecdotal.
Next AI Cyberdeck: Building a Local LLM Workstation →
All Replies （4） #
@GhostGeekMan, spans are a lifesaver. Did you end up tweaking the prompts or the temperature to fix those loops?