How do you analyze your coding sessions with OpenTelemetry traces from Claude Code?

A developer on the AGNT5 team enabled Claude Code's OpenTelemetry tracing, backed by a self-hosted Jaeger instance, to inspect what happens between prompts during coding sessions. The traces revealed Claude was repeatedly querying package versions from the shell because their skill file cited a version number without saying whether to trust it; adding one line instructing Claude to use the skill's versions eliminated the lookups. The writeup documents the Jaeger and settings.json setup, the span and attribute schema, and eight Python-based questions teams can ask of their own session traces.

We use Claude Code every day to build AGNT5 https://agnt5.com , a platform for running AI agents and workflows. When a session was slow, or the result wasn't what we expected, we couldn't see why. We could read the conversation, but not what happened in between: which commands ran, what the tools returned, whether our skill loaded. A skill is a folder of instructions that Claude Code loads when a task needs it. Ours teaches it how to work with AGNT5. So we turned on Claude Code's OpenTelemetry tracing, which records every prompt, model call and tool call. One of the first things it showed us was something we could fix on our side. Claude was checking package versions online from the shell: tool Bash 62ms ls -la; pip index versions agnt5 | head -2; which agnt5 uv tool Bash 51ms uv pip index versions agnt5 | head -1 || curl -s https://pypi.org/pypi/agnt5/json | … tool Bash 292ms curl -s https://pypi.org/pypi/agnt5/json | …; agnt5 --version That was a sensible thing for it to do. Our skill mentioned a version number and didn't say whether to trust it. We added one line telling Claude to use the versions in the skill, and the lookups stopped. We wouldn't have found this from the conversation, or from counting web searches: there were only 2 in three days. This article shows how to set up the same tracing, how to read a trace, and eight questions you can answer about your own sessions, each with a few lines of Python and the output from ours. Each prompt you send becomes one trace. It's a tree of spans: claude code.interaction ← your prompt → Claude's answer ├── claude code.llm request ← each model call └── claude code.tool ← each tool call: Bash, Read, MCP, Skill… ├── tool.blocked on user ← time waiting for a permission decision └── tool.execution ← time actually running, success true/false Each span carries attributes. These are the ones this article uses: | Span | Attributes | |---|---| | interaction | user prompt | | llm request | model , query source why the call was made , input tokens , output tokens , cache read tokens , ttft ms time to first token , stop reason | | tool | tool name , tool input the command, file path or arguments , skill name | | tool.blocked on user | decision accept or reject , source your settings or you | | tool.execution | success , error | MCP calls are tool spans named mcp <server <tool . A skill load is a tool span named Skill . A subagent's spans sit under the tool span that started it. You need Docker and Claude Code. Nothing leaves your machine. 1. Start Jaeger. It stores the traces and gives you a UI at http://localhost:16686 http://localhost:16686 . Save the config from the appendix as jaeger-config.yaml , then run: docker volume create jaeger-badger docker run -d --name jaeger --restart unless-stopped --user 0 \ -p 16686:16686 -p 4317:4317 -p 4318:4318 \ -v jaeger-badger:/badger \ -v "$PWD/jaeger-config.yaml:/etc/jaeger/config.yaml:ro" \ jaegertracing/jaeger:2.23.0 --config /etc/jaeger/config.yaml The config keeps traces on disk for 30 days. Jaeger's default image keeps them in memory and loses them on restart. 2. Turn on tracing. Add this to ~/.claude/settings.json . It has to be your user settings. A repository's .claude/settings.json can't choose where traces go: we set the endpoints there and the traces still went to the one in our user settings. { "env": { "CLAUDE CODE ENABLE TELEMETRY": "1", "CLAUDE CODE ENHANCED TELEMETRY BETA": "1", "OTEL TRACES EXPORTER": "otlp", "OTEL EXPORTER OTLP TRACES PROTOCOL": "http/protobuf", "OTEL EXPORTER OTLP TRACES ENDPOINT": "http://localhost:4318/v1/traces", "OTEL LOG TOOL DETAILS": "1", "OTEL LOG TOOL CONTENT": "1", "OTEL LOG USER PROMPTS": "1", "ENABLE BETA TRACING DETAILED": "1", "BETA TRACING ENDPOINT": "http://localhost:4318" } } | Lines | What they do | |---|---| | First five | Turn on tracing and send it to Jaeger. On their own they give you timings and success or failure | | The three OTEL LOG | Add content that is left out by default: commands and tool arguments, tool output, and your prompts | | Last two | Turn on detailed tracing: the full input of every tool call, the model's response text and what was sent to the model each turn | We ran the same prompt at each level to see what you get: | Settings on | Spans contain | |---|---| | First five | Timings, tool names, success or failure, token counts. The prompt is recorded as <REDACTED | | First eight | Also the prompt text, the Bash command full command , file paths and tool output | | All ten | Also tool input for every tool MCP arguments included , the model's response text, the system prompt and the list of tools offered | Your email and account IDs are on every span at every level. Detailed tracing is the heaviest setting. Anthropic's docs say to use it for debugging only, and that it exports a lot of data. It works as is in claude -p and the Agent SDK. In interactive terminal sessions, the docs say your organization has to be allowlisted for it. Without it, the snippets below still work, except question 6: it compares tool inputs, which are no longer recorded. 3. Send a prompt. Restart Claude Code, send any prompt, then open Jaeger and pick the service claude-code . Here is one real prompt. It ran a Bash command, read a file and called an MCP server: 7.1 seconds, 16 spans. The timeline. The three model calls take 2.7 s, 1.3 s and 3.0 s. The four tool calls take 4–117 ms each. A tool call. Click a tool span to see its attributes: the exact command and what it returned. Jaeger's UI is the right tool for one trace: why was this prompt slow, what did this command return. Every tool span has the same name in the timeline, though, so questions like "did a skill load?" mean clicking through spans. For those, and for anything across many sessions, use a script. Jaeger has an HTTP API. The loader in the appendix traces.py , about 50 lines pulls recent spans from it and returns each one as a small dictionary. Every snippet below starts from this: python import collections, re from traces import load, tool input spans = load hours=72 our last three days: 240 traces, 5,492 spans tools = s for s in spans if s "name" == "tool" by id = {s "id" : s for s in spans} The output under each snippet is from our own sessions over those three days. Part of that time went to a test where we ran Claude Code headless on a set of tasks, so your numbers will look different. When a session feels slow, the first thing to find out is what kind of slow. Time can go to three places: the model thinking, a tool running, or a tool waiting for approval. Each has its own span type, and every span has a duration. The snippet adds up the seconds for each type. total = collections.Counter for s in spans: if s "name" in "llm request", "tool.execution", "tool.blocked on user" : total s "name" += s "ms" / 1000 for name, sec in total.most common : print f"{name:22} {sec:6.0f} s" llm request 6911 s tool.execution 2355 s tool.blocked on user 694 s The model took about three times as long as all the tools together. A slow session is usually many model calls, not a slow tool. The tool time that does add up comes from commands that start a server or wait on a remote service. Every tool call has a blocked on user span. It measures the time between Claude asking to use the tool and getting a decision. For calls your settings already allow, that's close to zero. For calls that need you, it's however long you took to click. The snippet groups that time by tool, using each span's parent to find the tool's name. waits = collections.Counter for s in spans: if s "name" == "tool.blocked on user" and s "parent" in by id: waits by id s "parent" "a" .get "tool name" += s "ms" / 1000 for tool, sec in waits.most common 3 : print f"{tool:8} {sec:5.0f} s" Bash 401 s Edit 137 s Write 63 s That's 694 seconds waiting on permission decisions, more than half of it for Bash. The source attribute says who decided: in ours, 1,234 calls were accepted by settings and 64 by a person. Those 64 are the ones to look at. If the same safe command keeps waiting for you, add it to your permissions allowlist. Bash can do almost anything, so the tool's name alone tells you little. The command itself is recorded on the span. The snippet collects every command and flags the ones that reach the internet: a curl or wget to an address that isn't local, or a lookup on a package registry. Change the pattern to match the commands you care about. bash = tool input s .get "command", "" for s in tools if s "a" .get "tool name" == "Bash" web = c for c in bash if re.search r" curl|wget \s ^|;& https?:// ? localhost|127\.0\.0\.1 |pip index|npm view", c print len bash , "Bash commands,", len web , "reach the web" 581 Bash commands, 21 reach the web Grouped by where they went: pypi.org 6, pip index 5, npm view 4, restcountries.com 3, api.worldbank.org 2, github.com 1. This is where the finding from the top of the article came from. In the same three days there were 2 WebSearch or WebFetch calls and 21 lookups from Bash, so it's worth reading the commands as well as counting the web tools. Fifteen were checking package versions. Our skill mentioned a version number without saying whether to trust it, so we added a line and those lookups stopped. The five calls to the two public APIs were Claude testing an API that its own code calls, which is exactly right. The pattern finds the commands; you still have to read them. A skill only helps if it loads. Claude decides that from the skill's description, so if the description doesn't match the way you phrase a request, the skill is skipped and nothing tells you. Loading a skill is a tool call named Skill . The snippet counts those calls and which skill each one loaded, then how many prompts had one. loads = s for s in tools if s "a" .get "tool name" == "Skill" print collections.Counter s "a" .get "skill name" for s in loads prompts = s for s in spans if s "name" == "interaction" with skill = {s "trace" for s in loads} print len prompts , "prompts,", sum p "trace" in with skill for p in prompts , "loaded a skill" Counter {'agnt5': 40} 183 prompts, 40 loaded a skill A skill load is either in the trace or it isn't, which makes it easy to test. Send a prompt that should use the skill and check for the span. We did, with a small change request inside one of our projects, and the span wasn't there: the skill's description didn't match the request. We widened the description, sent the same prompt, and the span appeared. Test the other direction too. Send prompts that have nothing to do with the skill and confirm it stays out. MCP tools are small services, and like any service they can be slow or fail. When one fails, Claude may work around it, so you might not notice. Each call's execution span has a duration and a success flag. The snippet builds one row per tool: how many calls, how many failed, and the average time. stats = collections.defaultdict lambda: 0, 0, 0.0 calls, failed, total ms for s in spans: if s "name" == "tool.execution" and s "parent" in by id: name = by id s "parent" "a" .get "tool name", "" if name.startswith "mcp " : st = stats name st 0 += 1 st 1 += str s "a" .get "success" .lower == "false" st 2 += s "ms" for name, n, failed, ms in sorted stats.items , key=lambda x: -x 1 0 :4 : print f"{name:40} {n:3} calls {failed} failed {ms / n:5.0f} ms avg" mcp claude ai AGNT5 get run events 87 calls 2 failed 975 ms avg mcp claude-in-chrome computer 48 calls 0 failed 521 ms avg mcp outline update document 46 calls 4 failed 1272 ms avg mcp claude ai AGNT5 list projects 27 calls 0 failed 740 ms avg One line per tool tells you which is used most, which is slow and which fails. Here, one document tool failed 4 times in 46 calls. The error attribute on the failed spans says why. A common worry about agents is that they get stuck: they run a command, get the same result and try it again. In a trace, that shows up as two tool spans with the same tool and exactly the same input under one prompt. The snippet counts them. calls = collections.Counter s "trace" , s "a" .get "tool name" , s "a" .get "tool input" for s in tools if s "a" .get "tool input" print sum c - 1 for c in calls.values if c 1 , "repeated calls in", len prompts , "prompts" 6 repeated calls in 183 prompts A loop shows up as the same tool with the same input, more than once, inside one prompt. We expected to find loops and found almost none. The six were things like taking a second screenshot or re-reading a document after editing it. An agent can write code and say "done" without running it. The final answer may not make that clear, but the trace records every tool call in order, so you can see what happened after the last file change. The snippet finds the prompts where Claude edited or wrote a file, and counts the ones with no Bash command afterwards. edits = untested = 0 for p in prompts: names = s "a" .get "tool name" for s in tools if s "trace" == p "trace" if "Edit" in names or "Write" in names: edits += 1 last edit = max i for i, n in enumerate names if n in "Edit", "Write" untested += "Bash" not in names last edit + 1: print edits, "prompts changed files;", untested, "ran nothing afterwards" 75 prompts changed files; 27 ran nothing afterwards This is a rough signal, not a verdict. Many of our 27 were edits to documents, where there's nothing to run. But it gives you a short list to read, and it's how we noticed a real pattern: on build tasks, traces ended with a file write and a summary, and no command ran the project. We added one line to our skill, "a change is done only after it runs", and those traces now end with a run. Not every model call answers your prompt. Claude Code also makes calls of its own, and they use tokens too. Each model span has a query source attribute that says why the call was made. The snippet counts the calls by that reason. print collections.Counter s "a" .get "query source" for s in spans if s "name" == "llm request" .most common 3 'sdk', 1008 , None, 145 , 'generate session title', 52 query source says why each model call was made. Most are the calls that answer your prompts. The rest are background calls that Claude Code makes on its own, such as writing a session title, mostly on a smaller model. One of our background calls produced 5 output tokens and read 85,000 cached tokens. They're worth knowing about when you add up token use. To check whether a change helped, run the same prompt before and after and label each run. Any OTEL RESOURCE ATTRIBUTES you set are attached to that session's spans: OTEL RESOURCE ATTRIBUTES="task=fix-login,run=2" claude -p "Fix the failing login test" claude -p runs one prompt without the interactive UI, so it's easy to script. Your script can then pick out exactly those spans and ask the questions above of each run. | What the traces show | What to change | |---|---| | Long permission waits on safe commands | Add them to your permissions allowlist | | Claude exploring or guessing in a repo | Add the missing guidance to CLAUDE.md | | Lookups for things you've already documented | Say in your instructions what to trust | | A skill that doesn't load, or loads on unrelated prompts | Rewrite its description | | Work that's never run | Say when a task counts as done | | An MCP tool that fails often or returns huge results | Fix the tool, or trim its output | Then send the same prompts again and compare. That's the whole loop: look, change one thing, look again. service: extensions: jaeger storage, jaeger query, healthcheckv2 pipelines: traces: receivers: otlp processors: batch exporters: jaeger storage exporter extensions: healthcheckv2: use v2: true http: endpoint: 0.0.0.0:13133 jaeger query: storage: traces: claude store http: endpoint: 0.0.0.0:16686 jaeger storage: backends: claude store: badger: directories: keys: /badger/keys values: /badger/values ephemeral: false ttl: spans: 720h receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: exporters: jaeger storage exporter: trace storage: claude store Save as traces.py next to your scripts. """Load recent Claude Code spans from a local Jaeger. Each span is a dict with: name, trace, id, parent, ms duration and a its attributes . """ import datetime as dt, json, urllib.error, urllib.parse, urllib.request JAEGER = "http://localhost:16686" def load hours=24 : now = dt.datetime.now dt.timezone.utc q = urllib.parse.urlencode { "query.service name": "claude-code", "query.start time min": now - dt.timedelta hours=hours .strftime "%Y-%m-%dT%H:%M:%SZ" , "query.start time max": now.strftime "%Y-%m-%dT%H:%M:%SZ" , "query.num traces": 5000, } try: with urllib.request.urlopen f"{JAEGER}/api/v3/traces?{q}" as r: data = json.load r except urllib.error.HTTPError: Jaeger answers 404 when the window has no traces return spans = {} for rs in data.get "result", {} .get "resourceSpans", : for ss in rs.get "scopeSpans", : for s in ss.get "spans", : a = {x "key" : next iter x "value" .values , None for x in s.get "attributes", if x.get "value" } spans s "spanId" = { the same span can arrive twice; keep one "name": s "name" .removeprefix "claude code." , "trace": s "traceId" , "id": s "spanId" , "parent": s.get "parentSpanId" , "start": int s "startTimeUnixNano" , "ms": int s "endTimeUnixNano" - int s "startTimeUnixNano" / 1e6, "a": a, } return sorted spans.values , key=lambda s: s "start" def tool input span : """A tool span's input as a dict Bash: {"command": ...}, Read: {"file path": ...} .""" a = span "a" raw = a.get "tool input" or "" body = raw.split "\n", 1 1 if raw.startswith " TOOL INPUT" and "\n" in raw else raw try: return json.loads body except ValueError: no detailed tracing: only the command or the file path is recorded return {k: v for k, v in "command", a.get "full command" , "file path", a.get "file path" if v} We're building AGNT5 https://agnt5.com . The skill mentioned in this article is open source: agnt5dev/skills https://github.com/agnt5dev/skills .