Dev ToolsArticle
A streaming double-count in Sentry's Google GenAI integration shows why instrumentation deserves the same rigor as production code.
There's a bug sitting in Sentry's JavaScript SDK that will never throw an exception, never fail a request, and never page anyone. When you call @google/genai in streaming mode and Gemini asks to run a tool, the SDK's instrumentation records that tool call to the span twice — one invocation in, two entries out on the gen_ai.response.tool_calls
attribute. Worse, the two entries don't even agree on their shape: one keys the parameters under args
, the other under arguments
.
It's a small bug with a small fix. It's also one of the clearest illustrations I've seen of why AI observability code deserves more scrutiny than it's getting.
Anatomy of a double-dip #
The bug lives in the streaming handler of Sentry's Google GenAI integration (packages/server-utils/src/ai/google-genai/streaming.ts
). As each streamed chunk arrives, the instrumentation accumulates tool calls into span state — from two places:
// Source one: the SDK's convenience accessor
if (Array.isArray(chunk.functionCalls)) {
state.toolCalls.push(...chunk.functionCalls);
}
// Source two: manually walking the candidate's content parts
if (part.functionCall) {
state.toolCalls.push({
type: 'function',
id: part.functionCall.id,
name: part.functionCall.name,
arguments: part.functionCall.args,
});
}
The catch: chunk.functionCalls
isn't independent data. It's a getter on the @google/genai
response object that filters the first candidate's content.parts
for function calls — the exact same parts the second block iterates by hand. Same call, same id, pushed twice, in two different schemas. The non-streaming path doesn't have this problem; it reads the accessor once and moves on.
The bug surfaced via DEV's Summer Bug Smash, and the author's fix — PR #23432 — deletes the manual iteration and trusts the accessor, matching how Sentry's OpenAI and Anthropic integrations behave. As of this writing the PR is still an open draft, so if you're running the Google GenAI integration with streaming and tools today, your spans are still double-reporting.
I've verified the double-push exists on Sentry's develop
branch myself. This isn't a hypothetical.
Why "harmless" telemetry bugs aren't #
The instinct is to shrug. Duplicate span attributes — who cares? Here's who: anyone treating gen_ai.*
attributes as data rather than decoration, which is increasingly everyone building agents.
Tool-call telemetry is feeding real decisions now. Teams count tool invocations to detect runaway agent loops. They diff tool-call rates across model versions to catch regressions. Some pipe span data into eval sets. A silent 2x on tool-call counts — but only for the Gemini streaming path, not OpenAI, not Anthropic, not non-streaming Gemini — is precisely the kind of skew that survives for months because every individual dashboard still looks plausible. Your Gemini agent didn't get twice as tool-happy after you enabled streaming. Your instrumentation did.
The schema mismatch is arguably worse than the count. Anything parsing gen_ai.response.tool_calls
downstream now has to handle both args
and arguments
for the same field, and code that grew that tolerance will keep it forever, papering over the next inconsistency too. This is how observability data rots: not through outages, but through consumers quietly learning to accept garbage.
And that's the structural problem. Telemetry code has no natural failure signal. Production code that double-counts breaks a test or a user; instrumentation that double-counts just emits confident, well-formed, wrong data. The only defense is treating instrumentation as production code — which, to Sentry's credit, is exactly what the fixing PR does, adding a streaming-vs-non-streaming parity test that fails on develop
and passes with the fix. Parity tests between code paths that should agree are the single highest-value test you can write for instrumentation, and most SDKs don't have them.
The pattern to steal (and the one to avoid) #
If you're writing your own LLM instrumentation — and plenty of teams are, given how young this space is — the transferable lesson is: pick one canonical source per fact. The Sentry bug is a belt-and-suspenders anti-pattern: reading both a convenience accessor and the raw wire data it's derived from, presumably because streaming chunk shapes are confusing and grabbing both felt safe. In parsing, redundancy isn't safety; it's a duplication bug waiting for the case where both sources populate.
Streaming APIs make this trap easy to fall into. Accessors like functionCalls
exist precisely because the raw chunk structure is awkward, so instrumentation authors end up half-trusting them. Decide: either the accessor is the interface, or the wire format is. Never both.
There's a second, more forward-looking reason not to over-invest in gen_ai.response.tool_calls
specifically: it's on its way out. The OpenTelemetry GenAI semantic conventions — still officially in development — have been consolidating toward structured gen_ai.output.messages
, and Sentry has an open tracking issue to deprecate gen_ai.response.text
and gen_ai.response.tool_calls
in favor of it. If your pipeline hard-codes today's attribute names, budget for churn. The conventions underneath AI observability are wet cement.
What to do about it #
Concretely: if you use Sentry's JS SDK with @google/genai
, streaming, and tool calling, assume tool-call counts on those spans are inflated until the fix ships — watch the sentry-javascript changelog for it. If you've built alerting or evals on those numbers, dedupe by tool-call id
at the consumer, or baseline against non-streaming traffic, which reports correctly.
More broadly, spot-check your AI telemetry against ground truth once in a while — actual API responses, actual provider billing. LLM instrumentation across the ecosystem is months old, built against streaming formats that shift under it, emitting attributes whose spec isn't finished. It'll mature. Until then, "the span says so" is not evidence.
Sources & further reading #
One tool call, counted twice: a Google GenAI streaming double-dip in Sentry's JS SDK— dev.to - fix(server-utils): Deduplicate Google GenAI streaming tool calls— github.com - google-genai streaming instrumentation (develop branch)— github.com - Deprecate gen_ai.response.text and gen_ai.response.tool_calls in favor of gen_ai.output.messages— github.com - OpenTelemetry Semantic Conventions for Generative AI— opentelemetry.io
Lenn Voss· Cloud & Infrastructure Writer
Lenn writes about cloud platforms, Kubernetes internals, and the infrastructure decisions that quietly make or break engineering organizations. Based in Berlin's vibrant tech scene, they have a talent for turning dense platform-engineering topics into prose that people actually finish reading.
Discussion 0 #
No comments yet
Be the first to weigh in.