DSPy Observability & Monitoring with OpenTelemetry

SigNoz announced support for monitoring DSPy programs via OpenTelemetry, enabling real-time visibility into DSPy module calls and language model requests. The integration uses the OpenInference DSPy instrumentation to capture traces with DSPy-native attributes, allowing users to trace program runs, compare model usage, and set alerts on latency and errors. SigNoz offers both no-code auto-instrumentation and code-based instrumentation for connecting DSPy applications to its observability platform.

What is DSPy Observability? DSPy observability gives you real-time visibility into your DSPy programs by collecting traces using OpenTelemetry https://opentelemetry.io/ . The OpenInference DSPy instrumentation https://pypi.org/project/openinference-instrumentation-dspy/ automatically captures every DSPy module call Predict , ChainOfThought , ReAct , custom Module pipelines and the underlying language model request as spans, tagged with DSPy-native attributes such as openinference.span.kind CHAIN, LLM, TOOL , llm.model name , and llm.provider . With full DSPy monitoring in SigNoz, you can trace every program run end to end, break down module and tool activity, compare model usage and latency, set alerts on latency and errors, and continuously improve the reliability of your DSPy applications. Prerequisites - A SigNoz Cloud account https://signoz.io/teams/ with an active ingestion key or Self Hosted SigNoz instance https://signoz.io/docs/install/self-host/ - Python 3.9 or later - A DSPy program and a configured language model provider for example, an Anthropic or OpenAI API key Monitoring DSPy with OpenTelemetry DSPy does not ship with OpenTelemetry built in, so you add the OpenInference DSPy instrumentation https://pypi.org/project/openinference-instrumentation-dspy/ and export traces to SigNoz over OTLP/HTTP. You can wire this up two ways. For more details, refer to the DSPy documentation https://dspy.ai/ . No-code auto-instrumentation is recommended for quick setup with minimal code changes. It is ideal when you want observability up and running without modifying your application code, using the OpenInference DSPy instrumentor loaded automatically at startup. Step 1: Install the necessary packages in your Python environment. pip install \ dspy \ opentelemetry-distro \ opentelemetry-exporter-otlp \ openinference-instrumentation-dspy Step 2: Add automatic instrumentation. opentelemetry-bootstrap --action=install Step 3: Run an example. python import dspy dspy.configure lm=dspy.LM "anthropic/claude-sonnet-5" predict = dspy.Predict "question - answer" result = predict question="What is OpenTelemetry in one sentence?" print result.answer Step 4: Run your application with auto-instrumentation. OTEL RESOURCE ATTRIBUTES="service.name=<service name " \ OTEL EXPORTER OTLP ENDPOINT="https://ingest.<region .signoz.cloud:443" \ OTEL EXPORTER OTLP HEADERS="signoz-ingestion-key=<your-ingestion-key " \ OTEL EXPORTER OTLP PROTOCOL=http/protobuf \ OTEL TRACES EXPORTER=otlp \ opentelemetry-instrument <your run command Verify these values: <service name : The name your DSPy service appears under in SigNoz. <region : Your SigNoz Cloud region https://signoz.io/docs/ingestion/signoz-cloud/overview/ endpoint . <your-ingestion-key : Your SigNoz ingestion key https://signoz.io/docs/ingestion/signoz-cloud/keys/ . <your run command : The command you use to run your application, for example python main.py . Code-based instrumentation gives you fine-grained control over your telemetry configuration. Use this approach when you need to customize resource attributes, sampling strategies, or integrate with existing observability infrastructure. Step 1: Install the necessary packages in your Python environment. pip install \ dspy \ openinference-instrumentation-dspy \ opentelemetry-sdk \ opentelemetry-exporter-otlp-proto-http Step 2: Configure the OpenTelemetry exporter. Create a telemetry.py module that sets up the tracer provider, wires the OTLP/HTTP exporter to SigNoz, and turns on the DSPy instrumentation: python from openinference.instrumentation.dspy import DSPyInstrumentor from opentelemetry import trace as trace api from opentelemetry.exporter.otlp.proto.http.trace exporter import OTLPSpanExporter from opentelemetry.sdk import trace as trace sdk from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace.export import BatchSpanProcessor def setup tracing - None: resource = Resource.create {"service.name": "<service name "} tracer provider = trace sdk.TracerProvider resource=resource exporter = OTLPSpanExporter endpoint="https://ingest.<region .signoz.cloud:443/v1/traces", headers={"signoz-ingestion-key": "<your-ingestion-key "}, tracer provider.add span processor BatchSpanProcessor exporter trace api.set tracer provider tracer provider Turn on OpenInference DSPy auto-instrumentation. DSPyInstrumentor .instrument tracer provider=tracer provider Verify these values: <service name : The name your DSPy service appears under in SigNoz. <region : Your SigNoz Cloud region https://signoz.io/docs/ingestion/signoz-cloud/overview/ endpoint . <your-ingestion-key : Your SigNoz ingestion key https://signoz.io/docs/ingestion/signoz-cloud/keys/ . Step 3: Instrument your DSPy program. Call setup tracing once, before you configure DSPy and run your modules. Every DSPy module call and each underlying model request is then captured automatically: python import dspy from telemetry import setup tracing setup tracing dspy.configure lm=dspy.LM "anthropic/claude-sonnet-5" predict = dspy.Predict "question - answer" result = predict question="What is OpenTelemetry in one sentence?" print result.answer Each run emits a trace whose spans carry openinference.span.kind CHAIN, LLM, TOOL along with llm.model name and llm.provider on the model spans. OpenTelemetry batches spans before sending, so allow a few seconds after a run for data to appear in SigNoz. View DSPy Traces in SigNoz Once configured, your DSPy program automatically emits traces for every run. Traces are available in SigNoz under the Traces tab: When you click on a trace in SigNoz, you'll see a detailed view of the trace, including all associated spans, along with their events and attributes. CHAIN spans represent DSPy module and pipeline steps, LLM spans carry llm.model name and llm.provider , and TOOL spans capture ReAct tool calls. DSPy Observability Dashboard You can also check out our custom DSPy dashboard https://signoz.io/docs/dashboards/dashboard-templates/dspy-dashboard/ which provides specialized visualizations for monitoring your DSPy programs. The dashboard includes pre-built charts for module, chain, and tool activity, per-model usage, latency percentiles, and errors, along with import instructions to get started quickly. Troubleshooting DSPy Observability Troubleshooting DSPy Observability troubleshooting-dspy-observability No traces in SigNoz - Confirm your program actually ran a DSPy module. Setting up tracing alone emits nothing. - Verify setup tracing is called before dspy.configure ... and before any module runs. - Check that the region in the OTLP endpoint matches your SigNoz account. - OpenTelemetry batches data before sending, so wait 10-30 seconds after a run. Verify the instrumentor Make sure DSPyInstrumentor .instrument ... runs against the same tracer provider you registered with trace api.set tracer provider ... . If you build multiple providers, spans can be dropped silently. Auth errors 401 / 403 Re-check the ingestion key in the signoz-ingestion-key header. It must be the exact key from your SigNoz Ingestion Settings, with no extra spaces or quotes. Endpoint issues Keep the https:// scheme and :443 port on the endpoint, and the /v1/traces path for OTLP/HTTP. Do not send OTLP/HTTP traffic to the gRPC port. Setup OpenTelemetry Collector Optional Setup OpenTelemetry Collector Optional setup-opentelemetry-collector-optional What is the OpenTelemetry Collector? Think of the OTel Collector as a middleman between your app and SigNoz. Instead of your application sending data directly to SigNoz, it sends everything to the Collector first, which then forwards it along. Why use it? Cleaning up data - Filter out noisy traces you don't care about, or remove sensitive info before it leaves your servers. Keeping your app lightweight - Let the Collector handle batching, retries, and compression instead of your application code. Adding context automatically - The Collector can tag your data with useful info like which Kubernetes pod or cloud region it came from. Future flexibility - Want to send data to multiple backends later? The Collector makes that easy without changing your app. See Switch from direct export to Collector https://signoz.io/docs/opentelemetry-collection-agents/opentelemetry-collector/switch-to-collector/ for step-by-step instructions to convert your setup. For more details, see Why use the OpenTelemetry Collector? https://signoz.io/docs/opentelemetry-collection-agents/opentelemetry-collector/why-to-use-collector/ and the Collector configuration guide https://signoz.io/docs/opentelemetry-collection-agents/opentelemetry-collector/configuration/ . Additional resources: - Set up alerts https://signoz.io/docs/alerts/ for high latency or error rates - Learn more about querying traces https://signoz.io/docs/userguide/traces/ - Explore log correlation https://signoz.io/docs/userguide/logs query builder/