Tracing Multi-Agent LLMs: My experience with otel-swarm

Hand-wiring OpenTelemetry into every single provider call is tedious, so I've been using a library called otel-swarm to handle the instrumentation. It basically wraps the agent logic so you don't have to manually manage spans for every retry path.

The implementation is pretty straightforward. You use task()

for the root span and agent()

for the specific roles.

import { createSwarm } from 'otel-swarm';

const swarm = createSwarm({ service: 'my-swarm', otlpEndpoint: 'http://localhost:4318' });

await swarm.task('generation', { 'my.prompt': prompt }, async (root) => {
 const plan = await swarm.agent('planner', () =>
 swarm.llm('planner', {
 model: 'primary-model-id',
 fallbackModel: 'fallback-model-id',
 call: async (model) => {
 const res = await yourProviderCall(model);
 return { content: res.text, inputTokens: res.usage.in, outputTokens: res.usage.out };
 }
 })
 );

 await swarm.agent('critic', async (span) => {
 swarm.reviewEvents(span, issues); 
 });
});

In my real-world deployment with a code-gen tool, this setup tracked over 240 model calls across 8 different models. The key is that swarm.events

mirrors the span lifecycle, meaning the traceId

allows me to deep-link from a UI row directly to the exact trace in SigNoz.

One huge technical lesson I learned the hard way: never put fallback model info in the attribute you use for grouping.

Initially, I had the system overwrite the gen_ai.request.model

attribute when a fallback kicked in. This sounded logical—the span should "tell the truth" about who answered. But it wrecked my analytics. Every dashboard panel grouping by model attributed the primary model's timeout and wasted latency to the fallback model. My expensive primary looked flawless, while the cheap fallback looked like it had terrible p95 latency.

Now, I record the switch as a fallback_promotion

span event. It keeps the data clean and ensures the primary model gets the "blame" for the failure it actually caused. This is a much better AI workflow for anyone doing a deep dive into LLM observability.

Next Mid-Chain Governance: Fixing the AI Agent Blind Spot →