4% of Sessions, 65% of the Bill: 30 Days of Claude Code Telemetry

A turn refers to all agent behavior after one user prompt. A step in a turn is one round of Claude thinking, and calling a tool.

Background on OTel Agent Monitoring #

Across organizations, as more and more agentic tools are adopted with increasing capabilities and scope, there is an increased need to monitor and maintain oversight of agent behavior. Furthermore, as functionality like Claude's Dynamic Workflows is rolled out, which allows agents to spawn up to 1000 subagents to complete a task, tracking costs and optimizing usage is becoming a priority.

Claude, and many other agentic solutions, supports OpenTelemetry (OTel). OTel enables administrators to deploy a collector and receive telemetry events from user sessions, covering security-related events like tool call approvals as well as cost-related metrics like the cost of a request. In this experiment, we examine OTel data from a cross-functional team of ten¹ seeking generalizable insights organizations care most about: what risks actually manifest in deployed agents, and what drives costs?

¹ OTel data analyzed was provided through a collaboration with an enterprise partner, anonymized for this publication.

Findings #

9% the time, Claude insecurely handled credentials in plaintext #

To do its work, Claude has to use credentials to act on the user's behalf. If it reads a credential directly, though, three things go wrong:

• The credential is transmitted to Anthropic on every proceeding LLM call.
• It is retained in the session transcript and, depending on configuration, in the OTel logs.
• It lingers in the agent's context, where an indirect prompt injection can target it for exfiltration.

To handle credentials more safely, Claude has been trained to use them without reading them directly, for example, by passing a secret to a child process without printing it, or by programmatically setting it in an HTTP header. Claude did this 2,979 times during the study period.

But Claude wasn't perfect. Claude printed credentials outright 66 times, which put the value into its context, the transcript, and API requests to Anthropic. That is only about 2% of the time credentials were used, but because each printed value stays in context until compaction, plaintext credentials were present 9% of the time: 5,840 of 65,892 LLM API calls to Anthropic, across 32 of 342 sessions.

Of the 66 insecure reads, a little under half fetched a credential from the team's secrets manager, about a third dumped the environment of a running process, and the rest printed a local .env file, a cluster secret, or a macOS keychain entry.

14% of turns, Claude read untrusted external data #

Whenever Claude runs a tool, the tool's output becomes text a model reads. Data from outside one's organization is untrusted and may contain prompt injections capable of manipulating Claude into exfiltrating data, phishing the user, or taking unwanted actions across other integrated tools. Below, we analyze where the 135 MB of external data processed by Claude came from.

The most significant source of untrusted data was MCP servers, followed by web fetch and web search. 14% of turns read untrusted text (1,308 turns). The median such turn read 2.9 KB, one in ten read more than 141 KB, one in a hundred more than 833 KB, and twelve turns read more than 1 MB each.

During the month, 3,500 calls were made to MCP servers, averaging 20 KB each across 49 servers. The largest single MCP result was 1.07 MB. Across the ten team members, they averaged almost 5 unique MCP servers apiece, with 49 total MCP servers used over the month. This aligns with what we have seen recently across industries: reviewing requests to use MCP servers and governing safe MCP server use is becoming a significant to-do.

Assessing Skills, Connectors and Plugins?

Web fetch and web search together returned 61 MB, but only a fraction of that data reached the main agent. Claude's harness uses a smaller model (Haiku) to preprocess web data and extract the parts relevant to the main agent, limiting pollution of its context window by irrelevant web data. While this step ultimately reduces the risk of a prompt injection being processed by the main agent, it does not eliminate the risk, as prompt injections are likely to represent themselves as important and thereby manipulate Haiku to include injected content in the data passed to the main agent.

Beyond web retrieval, 255 external hosts were contacted via bash. These were a mix of APIs for services used by the team, like GitHub, and arbitrary websites retrieved using curl instead of the web-specific tools.

50% of tool use was more than 10 agent steps after the last user interaction #

After a user types a prompt, Claude works in a loop. The LLM processes a prompt, runs a tool, reads the result, and then chooses to stop or run more tools. One 'turn' describes everything the agent does after one user prompt, and one 'step' describes one iteration of the agent thinking and then calling tools, counting the steps of any subagents it spawned.

The median prompt was 86 characters, and the median turn made 3 model calls (LLM processing steps). About 1/100 turns took more than 72 processing steps and the longest turn had 1,102 processing steps.

We measured the proportion of tools called after an increasing number of agent processing steps in order to gauge the risk of agents taking sensitive actions long after a user prompt, when the agent's task may have deviated from the user's original intent.

The median tool call was invoked 10 processing steps after the user's message. 10% of tool calls happened more than 160 processing steps after the last user prompt, and 1% of tool calls happened more than 782 processing steps after the user's last prompt.

Commands that delete files, push code, commit, or reach the network were typically closer to the user, at a median of 5 processing steps, but one in ten was more than 56 processing steps out, and one rm -rf command ran 920 processing steps after the last user interaction.

13% of turns ran for more than five minutes with no human input, 2% for more than fifteen minutes, and under 1% for more than an hour.

Fable represented 54% of spend for only 27% of LLM calls #

During the course of this analysis, eight models were used. 89% of the spend was split between Fable 5 (54% of the total cost, $8,573, used for 27% of model calls) and Opus 5 (36% of the cost, $5,699, used for 50% of model calls).

While Fable was used for 45% fewer tasks than Opus, it cost 50% more. This was largely due to two factors, average context used and price-per-token. LLM calls to Anthropic for Fable 5 carried 319,000 tokens on average compared to 201,000 for Opus 5. Fable also cost $1.31 per million tokens (blended across input, output, and cached tokens) versus Opus' $0.75, contributing to the average model call cost of $0.42 for Fable compared to $0.15 for Opus. Across every model, 97% of tokens were cache reads (the agent re-reading its own conversation), making the cached tokens (billed at one tenth of the input rate, $1.00 per million tokens on Fable 5 and $0.50 on Opus 5) responsible for about 68% of spend.

On the opposite end of the spectrum, the 5% of model calls to Haiku 4.5 accounted for under 1% of total spend. This usage was dominated by auxiliary tasks, like 2,067 calls to preprocess web fetch and web search results (preventing unnecessary context from reaching the main model) and to generate session titles.

Spend was not split evenly across sessions. The cheaper half of all sessions accounted for only 2% of the bill. The 12 sessions that cost more than $300 accounted for 65% of spend, and two sessions alone accounted for 32%. Both primarily used Fable and were kept open and resumed daily for over a week (leading to an average of ~500k cached tokens of context being paid for on every LLM call). This extended a pattern in which a small proportion of agent tasks and a small subset of power users on the team represented the vast majority of spend.