What is DeepSeek Harness Observability? #
DeepSeek Harness (dsh
) is DeepSeek's open-source agent harness. It has no OpenTelemetry export of its own, so instrumenting it means installing one plugin. Once installed, every turn becomes a trace: the agent loop, each reasoning round, each model call with its token counts and time to first token, and each tool execution.
With full DeepSeek Harness observability in SigNoz, you can see how much your team is spending in tokens and on which models, how many round trips a single request actually takes, which tools the agent reaches for, and where turns fail.
Prerequisites #
-
SigNoz setup (choose one): SigNoz Cloud accountwith an active ingestion key- Self-hosted SigNoz instance
-
DeepSeek Harness 0.1.0-rc.6 or later, below 0.2.0. See the DeepSeek Harness repository - Node.js 22.19.0 or later
-
A
DEEPSEEK_API_KEY
, or another provider configured in your profile
Monitor DeepSeek Harness with OpenTelemetry #
Instrumentation comes from @loongsuite/dsh-plugin, an Apache-2.0 plugin that hooks the harness lifecycle and exports OTLP over HTTP.
Step 1: Install the plugin into each profile you use
dsh plugin --profile headless add @loongsuite/dsh-plugin
dsh plugin --profile web add @loongsuite/dsh-plugin
Plugins are installed per profile, so a plugin added to headless
does nothing when you launch web
. Confirm it loaded:
dsh --profile headless --dump-config | grep loongsuite
You should see id: loongsuite-observability
.
Step 2: Point the plugin at SigNoz
export OTEL_SERVICE_NAME=dsh-agent
export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
Verify these values:
<region>
: YourSigNoz Cloud region.<your-ingestion-key>
: Your SigNozingestion key.
Step 3: Run the harness
dsh --profile headless "list the files in this directory and summarise the project"
Spans are batched, so allow a few seconds after the turn finishes before looking in SigNoz.
View DeepSeek Harness Traces in SigNoz #
Open the Traces explorer and filter on gen_ai.agent.system = 'deepseek-harness'
. Each turn arrives as its own trace.
Open any enter_ai_application_system
span to see the full turn. The waterfall shows each reasoning round, the model call inside it, and every tool the agent executed.
The plugin tags every span with gen_ai.span.kind
, which is the attribute to filter and group by:
gen_ai.span.kind |
Span name | What it covers |
|---|---|---|
ENTRY |
enter_ai_application_system |
One turn, the root span |
AGENT |
invoke_agent deepseek-harness |
The agent loop, with turn-level token totals |
STEP |
react step |
One reasoning round |
LLM |
chat <model> |
One model call, with tokens and time to first token |
TOOL |
execute_tool <tool> |
One tool execution |
Filter on gen_ai.span.kind
rather than on the span name, since names embed the model and tool name and change per call.
Prompts and Responses Are Not Captured by Default #
captureContent
is false
, so spans carry structure, timings, and token counts but no prompt or completion text. Enable it with captureContent: true
in the profile config, or OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY
.
Two things to weigh before turning it on. Anything in the context window, including source code and file contents, is written to your backend. And the payload grows sharply: the same single-tool turn measured 6,350 bytes with capture off and 73,078 bytes with it on, mostly because gen_ai.tool.definitions
repeats the full schema of every registered tool on each model call.
Note that dsh.session.cwd
records the absolute working directory on every span even with content capture off.
DeepSeek Harness Observability Dashboard #
The DeepSeek Harness dashboard gives you token spend and cache efficiency, turn and session volume, model latency, tool activity, and error breakdown out of the box.
Troubleshooting DeepSeek Harness Observability #
No data in SigNoz
An expired or wrong ingestion key fails silently. The harness prints a normal answer and exits successfully while the exporter receives a 401, so nothing in the terminal tells you anything is wrong. Test the key directly:
curl -i -X POST "https://ingest.<region>.signoz.cloud/v1/traces" \
-H "content-type: application/json" \
-H "signoz-ingestion-key: <your-ingestion-key>" \
-d '{"resourceSpans":[]}'
A working key returns 200
with {"partialSuccess":{}}
. An expired one returns 401
with Expired key
.
The plugin does not appear in the config
Plugins are per profile. Run dsh --profile <profile> --dump-config | grep loongsuite
for the profile you actually launch, and add the plugin to that profile if it is missing.
Subagent work is missing from the parent trace
A spawned subagent opens its own trace with its own root span, rather than nesting under the parent turn. There is no span link between them. Join them on attributes instead: the subagent's spans carry dsh.session.parent_id
, dsh.session.origin = 'subagent'
, and dsh.session.delegation_depth
.
A failing tool shows as a successful span
Only tools that report failure explicitly, such as an MCP tool returning isError
, set the span status to error and add error.type = 'TOOL_ERROR'
. A shell command exiting non-zero still records a successful span, so tool error counts based on span status will undercount.
Do not confuse this with the built-in telemetry plugin
DeepSeek Harness ships its own dsh-session-telemetry-otel
plugin, disabled by default through DSH_TELEMETRY_MODE
, which sends OTLP logs to DeepSeek's own endpoint. That is DeepSeek's product analytics and is unrelated to this setup.
Related integrations #
Instrument the other AI coding agents your team runs, using the same OpenTelemetry pipeline:
Monitor Claude Code with OpenTelemetry- track token usage, cost, session activity, and tool decisionsOpenCode observability with OpenTelemetry- another agent that instruments through a plugin rather than nativelyMonitor OpenAI Codex with OpenTelemetry- trace Codex runs from the IDE extension and the CLIGrok Build observability with OpenTelemetry- token usage, tool activity, and startup latency from its native exporterMonitor GitHub Copilot with OpenTelemetry- trace Copilot chat requests, models, and token spend
Browse all LLM observability integrations to instrument the rest of your stack.