DeepSeek Harness Observability & Monitoring with OpenTelemetry

DeepSeek's open-source agent harness, DeepSeek Harness, now supports observability and monitoring through a new Apache-2.0 plugin, @loongsuite/dsh-plugin, which exports OpenTelemetry traces to SigNoz. The plugin, compatible with DeepSeek Harness 0.1.0-rc.6 to below 0.2.0, captures every turn as a trace, including reasoning rounds, model calls with token counts, and tool executions. By default, prompts and responses are not captured, but enabling captureContent increases payload size from 6,350 bytes to 73,078 bytes for a single-tool turn.

What is DeepSeek Harness Observability? DeepSeek Harness https://github.com/deepseek-ai/deepseek-harness dsh is DeepSeek's open-source agent harness. It has no OpenTelemetry export of its own, so instrumenting it means installing one plugin. Once installed, every turn becomes a trace: the agent loop, each reasoning round, each model call with its token counts and time to first token, and each tool execution. With full DeepSeek Harness observability in SigNoz, you can see how much your team is spending in tokens and on which models, how many round trips a single request actually takes, which tools the agent reaches for, and where turns fail. Prerequisites - SigNoz setup choose one : SigNoz Cloud account https://signoz.io/teams/ with an active ingestion key- Self-hosted SigNoz instance - DeepSeek Harness 0.1.0-rc.6 or later, below 0.2.0. See the DeepSeek Harness repository https://github.com/deepseek-ai/deepseek-harness - Node.js 22.19.0 or later - A DEEPSEEK API KEY , or another provider configured in your profile Monitor DeepSeek Harness with OpenTelemetry Instrumentation comes from @loongsuite/dsh-plugin https://github.com/loongsuite/dsh-plugin , an Apache-2.0 plugin that hooks the harness lifecycle and exports OTLP over HTTP. Step 1: Install the plugin into each profile you use dsh plugin --profile headless add @loongsuite/dsh-plugin dsh plugin --profile web add @loongsuite/dsh-plugin Plugins are installed per profile, so a plugin added to headless does nothing when you launch web . Confirm it loaded: dsh --profile headless --dump-config | grep loongsuite You should see id: loongsuite-observability . Step 2: Point the plugin at SigNoz export OTEL SERVICE NAME=dsh-agent export OTEL EXPORTER OTLP ENDPOINT="https://ingest.<region .signoz.cloud:443" export OTEL EXPORTER OTLP HEADERS="signoz-ingestion-key=<your-ingestion-key " Verify these values: <region : Your SigNoz Cloud region https://signoz.io/docs/ingestion/signoz-cloud/overview/ endpoint . <your-ingestion-key : Your SigNoz ingestion key https://signoz.io/docs/ingestion/signoz-cloud/keys/ . Step 3: Run the harness dsh --profile headless "list the files in this directory and summarise the project" Spans are batched, so allow a few seconds after the turn finishes before looking in SigNoz. View DeepSeek Harness Traces in SigNoz Open the Traces explorer and filter on gen ai.agent.system = 'deepseek-harness' . Each turn arrives as its own trace. Open any enter ai application system span to see the full turn. The waterfall shows each reasoning round, the model call inside it, and every tool the agent executed. The plugin tags every span with gen ai.span.kind , which is the attribute to filter and group by: gen ai.span.kind | Span name | What it covers | |---|---|---| ENTRY | enter ai application system | One turn, the root span | AGENT | invoke agent deepseek-harness | The agent loop, with turn-level token totals | STEP | react step | One reasoning round | LLM | chat <model | One model call, with tokens and time to first token | TOOL | execute tool <tool | One tool execution | Filter on gen ai.span.kind rather than on the span name, since names embed the model and tool name and change per call. Prompts and Responses Are Not Captured by Default captureContent is false , so spans carry structure, timings, and token counts but no prompt or completion text. Enable it with captureContent: true in the profile config, or OTEL INSTRUMENTATION GENAI CAPTURE MESSAGE CONTENT=SPAN ONLY . Two things to weigh before turning it on. Anything in the context window, including source code and file contents, is written to your backend. And the payload grows sharply: the same single-tool turn measured 6,350 bytes with capture off and 73,078 bytes with it on, mostly because gen ai.tool.definitions repeats the full schema of every registered tool on each model call. Note that dsh.session.cwd records the absolute working directory on every span even with content capture off. DeepSeek Harness Observability Dashboard The DeepSeek Harness dashboard https://signoz.io/docs/dashboards/dashboard-templates/deepseek-harness-dashboard/ gives you token spend and cache efficiency, turn and session volume, model latency, tool activity, and error breakdown out of the box. Troubleshooting DeepSeek Harness Observability No data in SigNoz An expired or wrong ingestion key fails silently. The harness prints a normal answer and exits successfully while the exporter receives a 401, so nothing in the terminal tells you anything is wrong. Test the key directly: curl -i -X POST "https://ingest.<region .signoz.cloud/v1/traces" \ -H "content-type: application/json" \ -H "signoz-ingestion-key: <your-ingestion-key " \ -d '{"resourceSpans": }' A working key returns 200 with {"partialSuccess":{}} . An expired one returns 401 with Expired key . The plugin does not appear in the config Plugins are per profile. Run dsh --profile <profile --dump-config | grep loongsuite for the profile you actually launch, and add the plugin to that profile if it is missing. Subagent work is missing from the parent trace A spawned subagent opens its own trace with its own root span, rather than nesting under the parent turn. There is no span link between them. Join them on attributes instead: the subagent's spans carry dsh.session.parent id , dsh.session.origin = 'subagent' , and dsh.session.delegation depth . A failing tool shows as a successful span Only tools that report failure explicitly, such as an MCP tool returning isError , set the span status to error and add error.type = 'TOOL ERROR' . A shell command exiting non-zero still records a successful span, so tool error counts based on span status will undercount. Do not confuse this with the built-in telemetry plugin DeepSeek Harness ships its own dsh-session-telemetry-otel plugin, disabled by default through DSH TELEMETRY MODE , which sends OTLP logs to DeepSeek's own endpoint. That is DeepSeek's product analytics and is unrelated to this setup. Related integrations Instrument the other AI coding agents your team runs, using the same OpenTelemetry pipeline: Monitor Claude Code with OpenTelemetry https://signoz.io/docs/claude-code-monitoring/ - track token usage, cost, session activity, and tool decisions OpenCode observability with OpenTelemetry https://signoz.io/docs/opencode-observability/ - another agent that instruments through a plugin rather than natively Monitor OpenAI Codex with OpenTelemetry https://signoz.io/docs/codex-monitoring/ - trace Codex runs from the IDE extension and the CLI Grok Build observability with OpenTelemetry https://signoz.io/docs/grok-build-observability/ - token usage, tool activity, and startup latency from its native exporter Monitor GitHub Copilot with OpenTelemetry https://signoz.io/docs/github-copilot-monitoring/ - trace Copilot chat requests, models, and token spend Browse all LLM observability integrations https://signoz.io/docs/llm-observability/ to instrument the rest of your stack.