DeepSeek Harness Dashboard

SigNoz released a new dashboard for monitoring DeepSeek Harness (dsh) usage, tracking token spend, cache efficiency, agent behavior, model latency, and tool activity via OpenTelemetry traces emitted by the @loongsuite/dsh-plugin. The dashboard, which requires SigNoz v0.135.0 or newer, provides panels for agent turns, sessions, model calls, tool calls, total tokens, cache hit rate, token usage over time, token distribution by model, activity over time, turn outcomes, model finish reasons, model call latency, time to first token, tool call distribution, and tool performance.

This dashboard provides comprehensive monitoring of DeepSeek Harness dsh usage patterns and performance, offering visibility into token spend and cache efficiency, turn and session volume, model latency, native and MCP tool activity, and where things fail. All panels read traces emitted by the @loongsuite/dsh-plugin plugin, which DeepSeek Harness needs because it ships no OpenTelemetry export in its core. Panels filter on gen ai.span.kind , the attribute that separates the five span kinds the plugin emits: ENTRY for a turn, AGENT for the agent loop, STEP for a react round, LLM for a model call, and TOOL for a tool execution. A service name variable scopes every panel, so one dashboard covers as many teams or machines as report under different service names. Dashboard Preview Recommended. Uses the V2 dashboard schema https://signoz.io/docs/dashboards/dashboards-v2-api/ and needs SigNoz v0.135.0 or newer. Import it in SigNoz with Dashboards → + New dashboard → Import JSON . Import guide https://signoz.io/docs/dashboards/import-dashboard/ What This Dashboard Monitors This dashboard tracks the traces the plugin emits to help you: Monitor Token Spend : Track input, output, and cached tokens to understand workload and cost Measure Cache Efficiency : Watch how much of each prompt is served from cache as sessions grow Understand Agent Behavior : See how many model calls and tool calls a single turn actually takes Compare Model Usage : See which models carry the workload and track migration between versions Track Reliability : Watch turn outcomes and error breakdown to catch failures early Find Slow Tools : Break tool latency down per tool to find the step that stalls a turn Panels Included | Panel | Description | |---|---| | Agent Turns | Every prompt-and-response cycle the harness completed | | Sessions | Distinct sessions, to tell one-shot invocations from long working sessions | | Model Calls | One count per model call, showing how many round trips a turn takes | | Tool Calls | Every tool executed, native and MCP together | | Total Tokens | Total token spend across every model call | | Cache Hit Rate | Share of input tokens served from cache | Token Usage Token Usage Over Time : Input, cached, and output tokens over time. Output stays low and flat on almost any real agent workload, because most model calls are short decisions about which tool to run next and only the final answer of a turn is long. Token Distribution by Model : Token spend per model, useful for tracking migration between model versions and spotting which model carries the workload. Agent Activity Activity Over Time : Turns, model calls, tool calls, and subagent turns on one axis. The subagent line is worth watching on its own, because those turns are also counted in the turn total. Turn Outcomes : How turns finished, from dsh.turn.end reason . A rising error share points at the model provider rather than at tooling, since a failing tool does not end a turn. Model Finish Reasons : Why each model call stopped. tool calls dominating over stop is the normal shape for an agent, because only the last call of a turn ends with a finished answer. Latency Model Call Latency : p95 and p99 duration of model calls. This tracks output length more than anything else, so a rising p99 usually means longer answers rather than a slower provider. Time to First Token : How long the model takes to start responding. Unlike total latency this is independent of answer length, so it is the honest measure of provider responsiveness. Tools and Errors Tool Call Distribution : Which tools the agent reaches for. Tools named mcp <server <tool come from an MCP server, and the name is the only thing that identifies them. Tool Performance : Call count and latency per tool. Slow tools matter even when they never fail, because the model sits idle while they run. Errors Over Time : Failed turns, model call errors, and reported tool errors together. Sessions Session Details : Token spend and model call count per session. Subagent sessions appear as their own rows, identifiable by a bare UUID where a top-level session id carries a session- prefix.