LLM Observability Overview SigNoz supports LLM observability through OpenTelemetry rather than a proprietary agent, with integrations covering 55 model providers, agent frameworks, and gateways. Applications emit gen_ai.* spans and metrics via standard OpenTelemetry libraries and export them over OTLP, letting SigNoz store and query LLM telemetry alongside other application telemetry. The approach captures traces and spans across the call chain, token usage, latency percentiles and time to first token, error classes such as HTTP 429 rate limits, cost attribution, model and prompt attribution, retrieval steps, and agent tool-call visibility. LLM observability is the practice of collecting traces, metrics, and logs from applications that call large language models, so you can see what each model call did, how long it took, how many tokens it consumed, and where it failed. A single user request in an LLM application often fans out into retrieval steps, several model calls, tool invocations, and retries. Without instrumentation, all of that collapses into one opaque block of latency and one line on a billing invoice. SigNoz approaches this with OpenTelemetry rather than a proprietary agent. Your application emits gen ai. spans and metrics through standard OpenTelemetry libraries, exports them over OTLP, and SigNoz stores and queries them next to the rest of your application telemetry. The integrations below cover 55 model providers, agent frameworks, and gateways. Supported Integrations What LLM Observability Covers An LLM call is a network request to a nondeterministic, metered, and comparatively slow service. That combination is what makes it worth instrumenting separately from an ordinary HTTP dependency. Traces and spans across the call chain. The unit of analysis is the full request, not the individual model call. A retrieval-augmented generation RAG request typically produces an embedding span, a vector search span, a chat completion span, and often a second completion span for reranking or summarization. Modeling these as a single trace shows which stage actually consumed the wall-clock time. In SigNoz these appear in the Traces explorer https://signoz.io/docs/userguide/traces/ alongside spans from your web framework and database, so an LLM call sits in the same waterfall as the SQL query that fed it. Token usage. Input and output tokens are recorded per call. Because tokens are the billing unit, token counts attributed by model and by code path are the closest thing to a live cost signal. Cached input tokens are counted separately by providers that support prompt caching, which matters because they are usually billed at a lower rate. Latency. LLM latency is bimodal and heavily influenced by output length, so averages are close to useless. Track percentiles, and for streaming responses track time to first token separately from total duration. A response that starts streaming in 300 ms and finishes in 8 seconds feels fast; one that returns nothing for 8 seconds and then dumps the whole answer does not, even though total duration is identical. Errors. Rate limits HTTP 429 , context-length overflows, content filter rejections, upstream 5xx responses, and client timeouts all fail differently and need different responses. Recording an error class per span lets you separate "we are being throttled" from "the model rejected this prompt." Cost attribution. Token counts alone do not tell you who spent them. Attributing spend to a model, a tenant, a feature, or a prompt version turns a single monthly figure into something actionable. Model and prompt attribution. Both the requested model and the model that actually served the response are worth capturing, since providers alias versions behind names like gpt-4 . Prompt template names and versions let you compare quality and cost across prompt revisions. Retrieval steps. For RAG, the retrieval stage is a common source of bad answers. Query latency, the number of documents returned, and empty result sets are the signals that separate a retrieval failure from a generation failure. Agent and tool-call visibility. Agent frameworks loop: the model picks a tool, the tool runs, the result is fed back, and the model decides again. Instrumenting each tool execution as its own span exposes runaway loops, tools that fail silently, and the iteration count per request. Instrumenting LLM Applications with OpenTelemetry Instrumentation works the same way it does for any other OpenTelemetry-instrumented service. A library patches your LLM client at runtime, wraps each call in a span, attaches attributes to it, and hands the finished spans to an exporter that ships them over OTLP to SigNoz. Automatic instrumentation Automatic instrumentation is the default path and covers most applications. You install an instrumentation package, and it patches the provider SDK so every call it makes produces a span with no changes to your application code. This works well because almost all LLM traffic flows through a small number of client libraries. Auto-instrumentation captures the mechanical facts: which model, how many tokens, how long, whether it failed. It cannot know your application's own semantics, such as which tenant made the request or which experiment arm the user is in. Manual instrumentation Manual instrumentation fills that gap. You create spans with the OpenTelemetry API for the stages that are specific to your application, such as a document chunking step or a business-level "answer a support ticket" operation, and you add your own attributes to spans that already exist. In practice most teams run both: auto-instrumentation for the provider calls, manual spans for the application logic wrapped around them. Instrumentation libraries Three community projects provide most of the LLM-specific instrumentation in use today. All three emit standard OpenTelemetry spans over OTLP, so all three work with SigNoz. - OpenLLMetry https://github.com/traceloop/openllmetry from Traceloop, a set of instrumentations covering provider SDKs and frameworks. See the Traceloop integration guide https://signoz.io/docs/traceloop/ . - OpenLIT https://github.com/openlit/openlit , which covers LLM SDKs along with vector databases and GPU metrics. See the OpenLIT integration guide https://signoz.io/docs/openlit/ . - OpenInference https://github.com/Arize-ai/openinference from Arize, which has strong coverage of agent and orchestration frameworks such as LangChain, LlamaIndex, and CrewAI. Some providers are also covered by instrumentation maintained in the OpenTelemetry contrib repositories https://github.com/open-telemetry/opentelemetry-python-contrib directly, such as opentelemetry-instrumentation-openai-v2 . Where an official instrumentation exists, the integration guides prefer it. No vendor SDK required SigNoz ingests OTLP. It does not ship an LLM-specific agent, and there is no SigNoz SDK to install in your application. Anything that speaks OTLP over HTTP or gRPC can send data, which has two consequences worth stating plainly. You can pick whichever instrumentation library best covers your stack, or mix them. And because the data is standard OpenTelemetry, moving it elsewhere later is a change of endpoint, not a re-instrumentation project. The exporter configuration is the same as any other OpenTelemetry service: OTEL EXPORTER OTLP ENDPOINT="https://ingest.