{"slug": "dspy-dashboard", "title": "DSPy Dashboard", "summary": "A new DSPy Dashboard monitors performance, cost, and reliability of DSPy programs using OpenTelemetry trace data, tracking span kinds CHAIN, LLM, and TOOL across model usage, latency percentiles, module behavior, tool usage, and error rates. The dashboard includes panels for summary metrics, activity distribution, model usage, latency percentiles, module breakdowns, tool usage, and recent errors, enabling users to understand program activity, track model usage, monitor latency, measure call fan-out, inspect module behavior, track tool usage, and catch errors early.", "body_md": "This dashboard provides a comprehensive view of your `DSPy`\n\nprograms using trace data. It is built on the DSPy-native span attributes emitted by `openinference-instrumentation-dspy`\n\n(`openinference.span.kind`\n\n, `llm.model_name`\n\n, `llm.provider`\n\n), so every panel keys off the span kinds DSPy produces: `CHAIN`\n\n(module and pipeline steps), `LLM`\n\n(model requests), and `TOOL`\n\n(ReAct tool calls). Use the `service_name`\n\npicker at the top to filter the panels to one or more DSPy services.\n\nDashboard Preview\n\nDashboards → + New dashboard → Import JSON\n\nWhat This Dashboard Monitors\n\nThis dashboard tracks the performance, cost, and reliability of your DSPy programs using OpenTelemetry trace data to help you:\n\n**Understand Program Activity**: See total DSPy span volume and how CHAIN, LLM, and TOOL work is distributed over time.** Track Model Usage**: Compare LLM call volume and average latency across every model and provider in use to guide model selection.** Monitor LLM Latency**: Watch p50, p90, p95, and p99 latency for model calls to surface slow responses and regressions.** Measure Call Fan-Out**: Track average LLM calls per program run to spot changes in how many model calls each run makes.** Inspect Module Behavior**: Break down CHAIN spans by module and operation to compare call counts and latency across a program.** Track Tool Usage**: Identify which tools your ReAct agents call most and which are slowest.** Catch Errors Early**: Watch the error rate and drill into recent failures the moment incidents begin.\n\nPanels Included\n\nSummary (Top Row)\n\n| Panel | Type | What It Shows |\n|---|---|---|\nDSPy operations | Value | Count of all instrumented DSPy spans (any span carrying `openinference.span.kind` ) in the selected window |\nLLM calls | Value | Count of `LLM` span-kind spans, representing the actual model requests DSPy makes |\nTool calls | Value | Count of `TOOL` span-kind spans, representing ReAct tool invocations |\nError rate | Value | Fraction of selected services' spans with an error status, shown as a percentage |\nAvg LLM calls / program run | Value | `LLM` spans divided by `dspy.program.run` root spans, the model-call fan-out per program run |\nLLM latency (p95) | Value | p95 duration of `LLM` span-kind spans |\n\nActivity & Distribution\n\n**DSPy operations by span kind (over time)**: Time-series graph of CHAIN, LLM, and TOOL span counts grouped by`openinference.span.kind`\n\n, revealing traffic patterns and which span kinds dominate.**Span kind distribution**: Pie chart showing each span kind's share of total activity, complementing the trend graph with a share-of-activity view.\n\nModel Usage & Latency\n\n**LLM calls by model (over time)**: Time-series graph of model-request volume grouped by`llm.model_name`\n\n, showing which models drive traffic and how it trends.**LLM latency percentiles (over time)**: Time-series graph of p50, p90, and p99 duration for`LLM`\n\nspans, surfacing both typical performance and tail latency over time.**Model usage**: Table of`LLM`\n\nspans grouped by`llm.model_name`\n\nand`llm.provider`\n\n, with call count and average latency, showing how usage is distributed across models and providers.\n\nModules & Tools\n\n**DSPy module breakdown**: Table of`CHAIN`\n\nspans grouped by module and operation name, with call count and average latency, for a side-by-side comparison across the modules in a program.**Tool usage**: Table of`TOOL`\n\nspans grouped by tool name, with call count and average latency, identifying which tools are used most and which are slowest.\n\nRecent Activity\n\n**Recent DSPy operations**: List of the latest instrumented DSPy spans ordered by timestamp, useful for drilling into individual operations when investigating a latency spike.**Errors**: List of recent errored spans across the selected DSPy services, for jumping straight to failures when the error rate climbs.", "url": "https://wpnews.pro/news/dspy-dashboard", "canonical_source": "https://signoz.io/docs/dashboards/dashboard-templates/dspy-dashboard", "published_at": "2026-07-21 00:00:00+00:00", "updated_at": "2026-07-22 04:25:06.201795+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "artificial-intelligence"], "entities": ["DSPy", "OpenTelemetry", "DSPy Dashboard"], "alternates": {"html": "https://wpnews.pro/news/dspy-dashboard", "markdown": "https://wpnews.pro/news/dspy-dashboard.md", "text": "https://wpnews.pro/news/dspy-dashboard.txt", "jsonld": "https://wpnews.pro/news/dspy-dashboard.jsonld"}}