DeepSeek released Harness into developer preview yesterday — MIT license, source on GitHub, and a claim that "everything is a plugin." I spent some time reading through the docs and the Cordis kernel architecture to see what that means in practice vs. marketing.
The short version: it's more real than most.
The architecture in one pass
Harness is built on Cordis, a plugin kernel that handles mounting, unmounting, and dependency resolution. Every agent capability is a plugin: models, tools, skills, sessions, sandboxes, storage, loops, scheduling, and the UI. You compose them through configuration files, not forks.
That means you can swap out the model provider, replace the sandbox implementation, or change the loop strategy without touching the core. The plugins communicate through Cordis services and events.
Four runtime modes
Standard mode ships with the full toolset — file editing, shell, web search, skills, planning, goals, subagents, workflows.
Code mode exposes everything through a Code Mode SDK so the model writes TypeScript programs that orchestrate multi-step operations in a single turn. This is the same pattern Aider's architect mode and Claude Code's thinking tool use: let the model plan in code, then execute.
Minimal mode strips it to two tools: persistent bash and str_replace_editor. This is explicitly designed for benchmarking. If you're running SWE-bench or similar evals, you want minimal mode because it reduces the harness's influence on results.
Creator mode adds runtime inspection, in-memory plugin testing, and preset authoring. It's a development environment for building new agent configurations.
Why the traceability matters
Every run produces an append-only session log: system prompts, reasoning traces, tool calls and their results, subagent scheduling, context injections. The Trajectory view lets you inspect these by source. You can resume, fork, search, and replay from the same event stream.
This is the kind of instrumentation that makes empirical evaluation possible. If you're trying to measure whether model A produces better results than model B on a coding task, you need to see what each model actually saw and did — not just the final output. Harness logs it by design.
What I want to test
The plugin architecture is promising for reproducible benchmarking because you can lock every variable except the model. Same tools, same sandbox, same loop strategy — swap the model plugin, run the same session, compare trajectories.
But there's a gap I need to verify: how much does the Cordis kernel itself influence behavior? If plugins share events and services, the wiring between them could introduce subtle biases. A minimal mode test should control for this, but I haven't run it yet.
The repo is at github.com/deepseek-ai/deepseek-harness. npx @deepseek-ai/dsh web gets you running from source. I'll post benchmarks once I have reproducible numbers from minimal mode across a few models.