Sentry's Double-Counted Tool Calls Are a Warning for AI Telemetry

A bug in Sentry's JavaScript SDK double-counts tool calls in streaming mode for Google's Gemini integration, recording each invocation twice with inconsistent parameter keys (`args` vs. `arguments`). The issue, identified via DEV's Summer Bug Smash and verified on Sentry's `develop` branch, remains unfixed as PR #23432 is still an open draft, skewing tool-call telemetry for teams monitoring agent loops and model regressions. Sentry's Google GenAI integration streams tool calls into span state from both the SDK's `chunk.functionCalls` accessor and manual iteration over content parts, duplicating entries only on the Gemini streaming path.

Dev Tools https://sourcefeed.dev/c/dev-tools Article Sentry's Double-Counted Tool Calls Are a Warning for AI Telemetry A streaming double-count in Sentry's Google GenAI integration shows why instrumentation deserves the same rigor as production code. Lenn Voss https://sourcefeed.dev/u/lennart voss There's a bug sitting in Sentry https://sentry.io 's JavaScript SDK that will never throw an exception, never fail a request, and never page anyone. When you call @google/genai https://github.com/googleapis/js-genai in streaming mode and Gemini asks to run a tool, the SDK's instrumentation records that tool call to the span twice — one invocation in, two entries out on the gen ai.response.tool calls attribute. Worse, the two entries don't even agree on their shape: one keys the parameters under args , the other under arguments . It's a small bug with a small fix. It's also one of the clearest illustrations I've seen of why AI observability code deserves more scrutiny than it's getting. Anatomy of a double-dip The bug lives in the streaming handler of Sentry's Google GenAI integration packages/server-utils/src/ai/google-genai/streaming.ts . As each streamed chunk arrives, the instrumentation accumulates tool calls into span state — from two places: // Source one: the SDK's convenience accessor if Array.isArray chunk.functionCalls { state.toolCalls.push ...chunk.functionCalls ; } // Source two: manually walking the candidate's content parts if part.functionCall { state.toolCalls.push { type: 'function', id: part.functionCall.id, name: part.functionCall.name, arguments: part.functionCall.args, } ; } The catch: chunk.functionCalls isn't independent data. It's a getter on the @google/genai response object that filters the first candidate's content.parts for function calls — the exact same parts the second block iterates by hand. Same call, same id, pushed twice, in two different schemas. The non-streaming path doesn't have this problem; it reads the accessor once and moves on. The bug surfaced via DEV's Summer Bug Smash https://dev.to/zkasuran/one-tool-call-counted-twice-a-google-genai-streaming-double-dip-in-sentrys-js-sdk-4l4p , and the author's fix — PR 23432 https://github.com/getsentry/sentry-javascript/pull/23432 — deletes the manual iteration and trusts the accessor, matching how Sentry's OpenAI and Anthropic integrations behave. As of this writing the PR is still an open draft, so if you're running the Google GenAI integration with streaming and tools today, your spans are still double-reporting. I've verified the double-push exists on Sentry's develop branch myself. This isn't a hypothetical. Why "harmless" telemetry bugs aren't The instinct is to shrug. Duplicate span attributes — who cares? Here's who: anyone treating gen ai. attributes as data rather than decoration, which is increasingly everyone building agents. Tool-call telemetry is feeding real decisions now. Teams count tool invocations to detect runaway agent loops. They diff tool-call rates across model versions to catch regressions. Some pipe span data into eval sets. A silent 2x on tool-call counts — but only for the Gemini streaming path, not OpenAI, not Anthropic, not non-streaming Gemini — is precisely the kind of skew that survives for months because every individual dashboard still looks plausible. Your Gemini agent didn't get twice as tool-happy after you enabled streaming. Your instrumentation did. The schema mismatch is arguably worse than the count. Anything parsing gen ai.response.tool calls downstream now has to handle both args and arguments for the same field, and code that grew that tolerance will keep it forever, papering over the next inconsistency too. This is how observability data rots: not through outages, but through consumers quietly learning to accept garbage. And that's the structural problem. Telemetry code has no natural failure signal. Production code that double-counts breaks a test or a user; instrumentation that double-counts just emits confident, well-formed, wrong data. The only defense is treating instrumentation as production code — which, to Sentry's credit, is exactly what the fixing PR does, adding a streaming-vs-non-streaming parity test that fails on develop and passes with the fix. Parity tests between code paths that should agree are the single highest-value test you can write for instrumentation, and most SDKs don't have them. The pattern to steal and the one to avoid If you're writing your own LLM instrumentation — and plenty of teams are, given how young this space is — the transferable lesson is: pick one canonical source per fact. The Sentry bug is a belt-and-suspenders anti-pattern: reading both a convenience accessor and the raw wire data it's derived from, presumably because streaming chunk shapes are confusing and grabbing both felt safe. In parsing, redundancy isn't safety; it's a duplication bug waiting for the case where both sources populate. Streaming APIs make this trap easy to fall into. Accessors like functionCalls exist precisely because the raw chunk structure is awkward, so instrumentation authors end up half-trusting them. Decide: either the accessor is the interface, or the wire format is. Never both. There's a second, more forward-looking reason not to over-invest in gen ai.response.tool calls specifically: it's on its way out. The OpenTelemetry GenAI semantic conventions https://opentelemetry.io/docs/specs/semconv/gen-ai/ — still officially in development — have been consolidating toward structured gen ai.output.messages , and Sentry has an open tracking issue to deprecate gen ai.response.text and gen ai.response.tool calls in favor of it. If your pipeline hard-codes today's attribute names, budget for churn. The conventions underneath AI observability are wet cement. What to do about it Concretely: if you use Sentry's JS SDK with @google/genai , streaming, and tool calling, assume tool-call counts on those spans are inflated until the fix ships — watch the sentry-javascript changelog for it. If you've built alerting or evals on those numbers, dedupe by tool-call id at the consumer, or baseline against non-streaming traffic, which reports correctly. More broadly, spot-check your AI telemetry against ground truth once in a while — actual API responses, actual provider billing. LLM instrumentation across the ecosystem is months old, built against streaming formats that shift under it, emitting attributes whose spec isn't finished. It'll mature. Until then, "the span says so" is not evidence. Sources & further reading - One tool call, counted twice: a Google GenAI streaming double-dip in Sentry's JS SDK https://dev.to/zkasuran/one-tool-call-counted-twice-a-google-genai-streaming-double-dip-in-sentrys-js-sdk-4l4p — dev.to - fix server-utils : Deduplicate Google GenAI streaming tool calls https://github.com/getsentry/sentry-javascript/pull/23432 — github.com - google-genai streaming instrumentation develop branch https://github.com/getsentry/sentry-javascript/blob/develop/packages/server-utils/src/ai/google-genai/streaming.ts — github.com - Deprecate gen ai.response.text and gen ai.response.tool calls in favor of gen ai.output.messages https://github.com/getsentry/sentry-javascript/issues/19734 — github.com - OpenTelemetry Semantic Conventions for Generative AI https://opentelemetry.io/docs/specs/semconv/gen-ai/ — opentelemetry.io Lenn Voss https://sourcefeed.dev/u/lennart voss · Cloud & Infrastructure Writer Lenn writes about cloud platforms, Kubernetes internals, and the infrastructure decisions that quietly make or break engineering organizations. Based in Berlin's vibrant tech scene, they have a talent for turning dense platform-engineering topics into prose that people actually finish reading. Discussion 0 No comments yet Be the first to weigh in.