gen_ai.*
spans and calculating pricing already exists in the open-source version (v0.133.0+), there is a significant gap between running docker compose up
and actually seeing your AI data on a dashboard.## The "Silent Gates" to Deployment
Most users fail to reach the LLM observability features because three specific configuration hurdles fail silently—the backend reports success, but the frontend remains empty.
-
The Observability Flag: The feature is hidden behind an experimental flag. If disabled, the UI simply redirects you to the home page without any error message.
-
Dashboard Versioning: The LLM dashboards require v2 rendering. Without the correct flag, you'll see a "Welcome to your new dashboard" shell, even if the data is flowing perfectly in the background.
-
Empty Registries: The
llm_pricing_rules
and span_mapper_groups
registries ship empty. Without seeded prices or vendor mapping presets, the dashboards have nothing to display.
To bypass these gates and get the system operational, you must set both of these environment variables simultaneously:
SIGNOZ_FLAGGER_CONFIG_BOOLEAN_ENABLE__AI__OBSERVABILITY=true
SIGNOZ_FLAGGER_CONFIG_BOOLEAN_USE__DASHBOARD__V2=true
Improving the AI Workflow #
Beyond just getting the dashboard to load, the real challenge is the lack of a "measured answer quality" signal. Most observability tools tell you that a request happened and how much it cost, but they don't tell you if the answer was actually correct.
By treating SigNoz not as a black box but as a system under test, we were able to identify that following standard documentation (like using OpenInference) often results in zero gen_ai.*
attributes being emitted, leaving the mapping layer useless.
For anyone building an LLM agent or a complex AI workflow, the goal should be moving past simple "success/fail" metrics toward a deep dive into actual response quality. This requires a deliberate prompt engineering strategy to ensure your spans carry the metadata necessary for the observability layer to actually categorize and price the tokens correctly.
Next Claude Design Sync: A Deep Dive into Protocol Archaeology →