DeepSWE results: ten coding agents on one model (2026) AgentField's open-source coding agent CodeAF solved 62 of 113 DeepSWE tasks (54.9%) in senior-dev mode, placing first among ten harnesses that all ran the same model, deepseek/deepseek-v4-flash-0731, through OpenRouter. CodeAF also posted the lowest cost per solved issue at 39.9 cents (22 cents per task), ahead of mini-swe-agent at 56 of 113 (49.6%) and 1.9x that cost, while Claude Code and the DeepSeek Harness each solved 16 of 113. The same mode reached 77.9% on DeepSeek V4.1 Flash and 69.0% on Kimi K3, with per-task records published for all 113 tasks per harness. I work on codeaf https://github.com/Agent-Field/CodeAF at AgentField, one of the harnesses compared here. This post first appeared on agentfield.ai https://agentfield.ai/benchmarks/deepswe . CodeAF, the open-source coding agent by AgentField, solved 62 of 113 DeepSWE tasks 54.9% in senior-dev mode, first of ten harnesses in AgentField's DeepSWE comparison, where every harness ran the same model deepseek/deepseek-v4-flash-0731 through OpenRouter . It also had the lowest cost per solved issue: 39.9 cents, at 22 cents a task. Every number is in arms.csv https://github.com/Agent-Field/CodeAF/tree/d049b13406dc9179f2265608bafca3255f3c5f3a/docs/benchmarks/deepswe . | Result | CodeAF senior-dev mode solved 62 of 113 54.9% , first of ten harnesses in this comparison | | Model, for every harness | deepseek/deepseek-v4-flash-0731 through OpenRouter | | Cost | 22 cents a task, 39.9 cents per solved issue the lowest of the ten | | Next best | mini-swe-agent, 56 of 113 49.6% , 1.9x CodeAF's cost per solved issue | | With newer models | the same mode solved 77.9% on DeepSeek V4.1 Flash and 69.0% on Kimi K3 | Every row ran the full DeepSWE set on the same model. "Cost per solved issue" is each harness's spend divided by tasks solved, shown as a multiple of CodeAF's 39.9 cents. "Mean time" is mean agent wall time per task. | Harness | Solved of 113 | Cost per task | Cost per solved issue | Mean time | |---|---|---|---|---| | CodeAF senior-dev | 62, 54.9% | 22 cents | 1x | 54 min | | mini-swe-agent | 56, 49.6% | 38 cents | 1.9x | 44 min | | Codex CLI | 51, 45.1% | 37 cents | 2.1x | 46 min | | Pi | 42, 37.2% | 35 cents | 2.4x | 52 min | | Oh My Pi omp | 31, 27.4% | 50 cents | 4.5x | 49 min | | OpenCode | 30, 26.5% | 50 cents | 4.8x | 48 min | | Kilo Code | 30, 26.5% | 48 cents | 4.6x | 54 min | | Claude Code | 16, 14.2% | 19 cents | 3.4x | 32 min | | DeepSeek Harness | 16, 14.2% | 150 cents | 26.6x | 94 min | | Muse Code | 3, 2.7% | 12 cents | 11.3x | 16 min | 30 of 113 is 26.5%. The method page prints 26.6% for OpenCode and Kilo Code, rounded from the stored reward rate of 0.2655; the task count is the same. In arms.csv the CodeAF row is named senior-dev , because the binary had an earlier name on the day of the run. At a shell, the same mode is codeaf senior-dev ; in a conversation it is /senior-dev senior-dev docs https://agentfield.ai/docs/codeaf/senior-dev . The per-task records https://github.com/Agent-Field/CodeAF/tree/9b4344aaf56048c995cb9caa33369449cf137832/docs/benchmarks/deepswe/2026-09-12-harness-comparison-v4-flash/tasks , one CSV per harness with 113 rows each, show where the lead came from: | Benchmark | full DeepSWE set, 113 tasks | | Model | deepseek/deepseek-v4-flash-0731 through OpenRouter | | Verifiers | official DeepSWE at 0b9fabb | | Budget | 3 hours per task | | Isolation | four shards per harness, a dedicated OpenRouter key per harness | How the columns are derived, from the method page https://github.com/Agent-Field/CodeAF/tree/d049b13406dc9179f2265608bafca3255f3c5f3a/docs/benchmarks/deepswe : | Harness | Version tested | Release date UTC | |---|---|---| | CodeAF | first stable release containing senior-dev mode: v0.5.0 https://github.com/Agent-Field/CodeAF/releases/tag/v0.5.0 integration commit dd0fcc654 , 1488 https://github.com/Agent-Field/CodeAF/pull/1488 | 2026-09-30 | | mini-swe-agent | 2.4.6 https://github.com/SWE-agent/mini-swe-agent/releases/tag/v2.4.6 | 2026-07-23 | | Codex CLI | 0.154.0 https://github.com/openai/codex/releases/tag/rust-v0.154.0 | 2026-09-09 | | Pi | 0.85.1 https://github.com/earendil-works/pi/releases/tag/v0.85.1 | 2026-09-05 | | Oh My Pi | 18.1.17 https://github.com/can1357/oh-my-pi/releases/tag/v18.1.17 | 2026-09-10 | | OpenCode | 1.18.30 https://github.com/anomalyco/opencode/releases/tag/v1.18.30 | 2026-09-09 | | Kilo Code | 7.6.2 https://github.com/Kilo-Org/kilocode/releases/tag/v7.6.2 | 2026-09-10 | | Claude Code | 2.1.267 https://github.com/anthropics/claude-code/releases/tag/v2.1.267 | 2026-09-09 | | DeepSeek Harness @deepseek-ai/dsh | 0.1.5-rc.1 https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.5-rc.1 | 2026-09-10 | | Muse Code | 1.1.1-R2514.1 | not independently established | versions.csv https://github.com/Agent-Field/CodeAF/tree/d049b13406dc9179f2265608bafca3255f3c5f3a/docs/benchmarks/deepswe holds full source revisions and release timestamps. DeepSWE https://deepswe.datacurve.ai/ , published by Datacurve, is "a benchmark for measuring frontier coding agents on original, long-horizon software engineering tasks": 113 tasks across TypeScript, Go, Python, JavaScript and Rust, each graded by a verifier that "accepts any solution whose observable behavior is correct" DeepSWE README https://github.com/datacurve-ai/deep-swe . SWE-bench Verified https://www.swebench.com/verified.html is "a human-validated subset of 500 SWE-bench instances." AgentField's comparison holds the model fixed and changes the harness, so it measures what the agent around the model adds. Every harness was graded by the same verifiers on the same tasks. With newer models, the same mode solved 77.9% on DeepSeek V4.1 Flash and 69.0% on Kimi K3. That is 88 of 113 solved exact 95% CI 69.1% to 85.1% with DeepSeek V4.1 Flash and 78 of 113 59.6% to 77.4% with Kimi K3. The scripts that run and grade DeepSWE tasks are in bench/deepswe/ https://github.com/Agent-Field/CodeAF/tree/dev/bench/deepswe in the CodeAF repository. One task through CodeAF: bench/deepswe/run.sh