AI Traces Query API - Query LLM Cost and Tool Calls

SigNoz introduced a `builder_ai_query` query type in its Trace API that runs queries against AI traces and exposes per-trace fields including `trace.tool_call_count`, `trace.distinct_tool_count` and `trace.estimated_total_cost`. The query type uses the existing `POST https://<your-signoz-url>/api/v5/query_range` endpoint with `signal` set to `traces`, and accepts the same `filter`, `selectFields`, `aggregations`, `groupBy`, `order`, `limit` and `stepInterval` fields as a standard builder query. The API gives developers the data behind SigNoz's AI Observability Explorer without the UI, letting them filter, aggregate and order traces that contain at least one LLM, tool or agent span.

The builder ai query query type runs queries against your AI traces, which are traces with at least one LLM, tool, or agent span. It adds per-trace fields such as tool call count and estimated cost, so you can filter, aggregate, and order AI traces programmatically. Use it when you want the data behind the AI Observability Explorer https://signoz.io/docs/ai-observability-explorer/ without the UI. API endpoint builder ai query uses the same endpoint, authentication, and top-level payload as the rest of the Trace API: POST https://<your-signoz-url /api/v5/query range Replace <your-signoz-url with the URL that you use to open SigNoz in a browser, for example <your-workspace .<region .signoz.cloud for SigNoz Cloud. See the Trace API overview https://signoz.io/docs/traces-management/trace-api/overview/ for the API key and header setup, and the payload model https://signoz.io/docs/traces-management/trace-api/payload-model/ for the start , end , requestType , and compositeQuery fields. To use this query type, set type to builder ai query inside a query envelope and keep signal as traces : { "type": "builder ai query", "spec": { "name": "A", "signal": "traces" } } The spec accepts the same fields as a builder query https://signoz.io/docs/traces-management/trace-api/payload-model/ builder-query filter , selectFields , aggregations , groupBy , order , limit , stepInterval . The difference is that builder ai query also exposes the AI trace fields below. Fields for AI queries Two kinds of fields are available in AI queries. Trace fields hold a total for the whole trace. Use them in filter , in aggregations , and in order . With requestType: trace , SigNoz uses only the spans inside start and end to select traces. filter tests those spans, and order uses them to select the traces for the page. The returned totals include every span of the trace, even spans outside start and end . SigNoz sorts the page by these full totals. So a returned trace can show totals that differ from the values that filter tested. With raw , scalar , and time series , totals include only spans inside the query range. | Field | Type | Holds | |---|---|---| | trace.tool call count | int | Number of tool calls in the trace. | | trace.distinct tool count | int | Number of different tools called in the trace. | | trace.estimated total cost | float | Estimated cost of all LLM calls in the trace. See Model pricing https://signoz.io/docs/ai-observability-model-pricing/ for how cost is computed. | The Explorer lists the full set of trace fields, including trace.llm call count , trace.input tokens , trace.output tokens , trace.total tokens , and trace.max llm duration nano . See Run a query https://signoz.io/docs/ai-observability-explorer/ run-a-query for the complete table. Span attributes come from the GenAI semantic conventions. Reference them with fieldContext: attribute in selectFields and groupBy . | Attribute | Holds | |---|---| | gen ai.request.model | The model that the call requested, for example gpt-4o . | | gen ai.tool.name | The name of a tool called by the model. | | gen ai.usage.input tokens | The number of input tokens consumed. | | gen ai.usage.output tokens | The number of output tokens produced. | For every attribute that SigNoz reads from GenAI spans and the source keys it maps by default, see AI Observability Telemetry Requirements https://signoz.io/docs/ai-observability-telemetry-requirements/ . Examples All examples use type: builder ai query and signal: traces . The start and end values are epoch timestamps in milliseconds; replace them with your query range. stepInterval accepts a number of seconds 60 or a duration string "60s" . Raw spans from AI traces with many tool calls Return up to 100 spans from the ai-agent service whose trace has more than 10 tool calls, newest first. requestType: raw returns individual spans. { "schemaVersion": "v1", "start": 1640995200000, "end": 1640998800000, "requestType": "raw", "compositeQuery": { "queries": { "type": "builder ai query", "spec": { "name": "A", "signal": "traces", "filter": { "expression": "service.name = 'ai-agent' AND trace.tool call count 10" }, "order": { "key": { "name": "timestamp", "fieldContext": "span" }, "direction": "desc" } , "limit": 100 } } } } Raw spans with selected gen ai fields The same filter as above, with explicit output columns: the span name plus the requested model, tool name, and input and output token counts. { "schemaVersion": "v1", "start": 1640995200000, "end": 1640998800000, "requestType": "raw", "compositeQuery": { "queries": { "type": "builder ai query", "spec": { "name": "A", "signal": "traces", "filter": { "expression": "service.name = 'ai-agent' AND trace.tool call count 10" }, "selectFields": { "name": "name", "fieldContext": "span" }, { "name": "gen ai.request.model", "fieldContext": "attribute" }, { "name": "gen ai.tool.name", "fieldContext": "attribute" }, { "name": "gen ai.usage.input tokens", "fieldContext": "attribute" }, { "name": "gen ai.usage.output tokens", "fieldContext": "attribute" } , "order": { "key": { "name": "timestamp", "fieldContext": "span" }, "direction": "desc" } , "limit": 100 } } } } Average cost by model scalar Compute the average estimated per-trace cost for each model, most expensive first. requestType: scalar returns one summary value per group. When you group by model, each model gets only the cost of its own spans, so a trace that calls two models does not count its full cost under both. { "schemaVersion": "v1", "start": 1640995200000, "end": 1640998800000, "requestType": "scalar", "compositeQuery": { "queries": { "type": "builder ai query", "spec": { "name": "A", "signal": "traces", "aggregations": { "expression": "avg trace.estimated total cost " } , "groupBy": { "name": "gen ai.request.model", "fieldContext": "attribute" } , "order": { "key": { "name": "avg trace.estimated total cost " }, "direction": "desc" } , "limit": 10 } } } } Cost by model over time time series Sum the estimated per-trace cost for each model in 60-second buckets. requestType: time series with stepInterval is suited to trend charts. { "schemaVersion": "v1", "start": 1640995200000, "end": 1640998800000, "requestType": "time series", "compositeQuery": { "queries": { "type": "builder ai query", "spec": { "name": "A", "signal": "traces", "aggregations": { "expression": "sum trace.estimated total cost " } , "stepInterval": "60s", "groupBy": { "name": "gen ai.request.model", "fieldContext": "attribute" } , "order": { "key": { "name": "sum trace.estimated total cost " }, "direction": "desc" } , "limit": 10 } } } } Top traces by cost trace list Return the 10 most expensive traces that have more than 10 tool calls but fewer than 3 distinct tools. requestType: trace returns one row per trace. { "schemaVersion": "v1", "start": 1640995200000, "end": 1640998800000, "requestType": "trace", "compositeQuery": { "queries": { "type": "builder ai query", "spec": { "name": "A", "signal": "traces", "filter": { "expression": "trace.tool call count 10 AND trace.distinct tool count < 3" }, "order": { "key": { "name": "trace.estimated total cost" }, "direction": "desc" } , "limit": 10 } } } } Next steps - AI Observability Explorer https://signoz.io/docs/ai-observability-explorer/ : run the same queries in the UI. - AI Observability Telemetry Requirements https://signoz.io/docs/ai-observability-telemetry-requirements/ : the full list of GenAI attributes and their default source keys. - Trace API payload model https://signoz.io/docs/traces-management/trace-api/payload-model/ : the shared fields for every query type.