cd /news/ai-agents/how-opentelemetry-observes-ai-agents… · home › topics › ai-agents › article
[ARTICLE · art-153315] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

How OpenTelemetry observes AI agents without code changes

At KubeCon China 2026, Causely's Endre Sara and Alibaba Cloud's Haibin Zhang demonstrated how OpenTelemetry eBPF Instrumentation (OBI) observes AI agent traffic without code changes, attaching kernel probes and parsing protocols outside the application to capture LLM and MCP calls over TLS. As of OBI v0.9.0, coverage includes OpenAI, Anthropic, Gemini, Bedrock, Qwen, and MCP over JSON-RPC, deployed as a DaemonSet with one process per node. Sara cited a customer whose Bedrock and OpenAI token usage grew 320% in a month with no way to identify the driving agent, arguing that without visibility into token use and tool calls "it's really hard to figure out how to make your agents better.

by read3 min views2 publishedOct 19, 2026

Originally posted to causely.ai by Endre Sara AI agents are hard to operate because instrumentation can't keep up with them. Haibin Zhang of Alibaba Cloud and I made that case at KubeCon China 2026, and showed how OpenTelemetry eBPF Instrumentation (OBI) closes the gap. The full talk is at the end of this post.

A traditional request follows a static path: client, gateway, service, database. An agent request is dynamic. One prompt fans out into embeddings, vector search, reranking, tool calls, and other agents, and the path changes on every run.

Haibin shared an example. One customer's token usage on AWS Bedrock and OpenAI grew 320% in a month, and the team could not tell which agent drove it. Other teams call LLM APIs over raw HTTP without a provider SDK, so SDK instrumentation sees nothing.

Library coverage is also fragmented. In Python, Bedrock instrumentation works but Anthropic may not. MCP instrumentation exists for npm but not for Go.

OBI attaches kprobes and uprobes in the kernel and parses protocols in a separate process outside your application. Because it reads payloads before encryption, it sees LLM and MCP calls over TLS without injecting code. It deploys as a DaemonSet: one process per node, no restarts.

As of OBI v0.9.0, coverage includes OpenAI, Anthropic, Gemini, Bedrock, Qwen, MCP over JSON-RPC, and embedding and rerank providers. When the GenAI semantic conventions change, one OBI upgrade updates every service.

As I said in the talk: "If you cannot observe how many tokens are used, what tools are called, and how many times, it's really hard to figure out how to make your agents better."

OBI complements SDKs. It captures LLM and tool calls from outside the application process, plus what SDKs cannot see: TLS, DNS, and retries. SDKs add in-process detail OBI cannot observe, such as business spans and custom attributes.

Consistent telemetry means two things. Every service reports GenAI calls with the same attributes and semantic convention version, and trace context connects those calls across services and languages. In my own production environment, a Go MCP server calls another Go service, which calls a Python service, which calls Gemini or, through OpenRouter, an Anthropic model. One OBI process per node instruments all of it the same way.

Consistent telemetry still leaves a gap. A trace shows an agent called a tool server dozens of times and slowed down. It does not say why. Answering that takes causal context: a model of how the agent, its tool servers, model providers, and the infrastructure beneath them can fail, and which symptoms each failure produces.

That is the layer we build at Causely. It uses OBI alongside the metrics, logs, traces, and infrastructure data teams already collect, and adds a causal model that checks observed symptoms against known cause signatures, including AI model-specific causes, and returns a diagnosis.

This builds on why GenAI applications are hard to operate and our earlier work instrumenting Java with OBI.

Watch the full talk below, then deploy OBI as a DaemonSet on a cluster running your agents. The project is active and welcomes contributors: join the SIG calls or pick up an open issue in the OBI repository.

── more in #ai-agents 4 stories · sorted by recency
── more on @opentelemetry 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-opentelemetry-ob…] indexed:0 read:3min 2026-10-19 · —