{"slug": "mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops", "title": "Mouse tops FrontierHarness at 24/30 with Kimi K3 using completion loops", "summary": "Mouse, an agent harness built on the OpenCode engine, scored 24/30 on the FrontierHarness benchmark, the highest pass rate among 13 harnesses tested with Kimi K3 on 30 coding tasks. Mouse passed 18/21 Terminal-Bench 2.1 tasks and 6/9 DeepSWE tasks, with a median time of 4:10 and cost per pass of $3.13. The harness's deterministic completion loop, which verifies task evidence, was credited for its performance, particularly on long-horizon DeepSWE tasks where OpenCode passed 0.", "body_md": "# Mouse on FrontierHarness\n\nMouse scored 24/30 with the highest pass rate.\n\n[FrontierHarness](https://frontierharness.org) is a benchmark that runs the same model, Kimi K3, through different coding agents on the same 30 tasks. Mouse passed 24 of 30, the highest pass rate and the fastest median time of any harness on the board.\n\n## The benchmark\n\nFrontierHarness runs 30 tasks. 21 come from Terminal-Bench 2.1 and are command line jobs, such as recovering a corrupted SQLite database or building a Cython extension. The other 9 are DeepSWE tasks, which are bug fixes and features in open source repos, graded by hidden tests.\n\nEvery harness gets the same model (Kimi K3), the same tasks, and the same runtime. The harness is the only thing that changes, so the results show what the prompt, config, tools, and loop are actually worth.\n\n## How Mouse works\n\nThe Mouse agent harness is built on top of the OpenCode engine. We designed our system prompt, configuration, rules, and set of skills to steer a custom loop that outperforms on coding tasks.\n\nThe loop is the biggest differentiator within Mouse. Most agents finish when the model decides a task is done. With Mouse, we use a deterministic approach that mandates the workspace inspects, verifies, and ensures the evidence of the completed task holds up. This loop iterates until the desired outcome.\n\nThis approach shows up when long-horizon jobs are required, which is why OpenCode passed 0 of the DeepSWE tasks and Mouse passed 6.\n\nThis is the same loop that powers [Night Shift](/blog/running-code-agents-overnight), how we run agents on long time horizon jobs and overnight.\n\n## Leaderboard\n\n| Harness | Pass | Terminal | DeepSWE | $ / pass | Median $ | Cache | Time |\n|---|---|---|---|---|---|---|---|\n| Mouse | 24/30 | 18/21 | 6/9 | $3.13 | $0.31 | 84% | 4:10 |\n| Codex | 20/30 | 15/21 | 5/9 | $3.47 | $0.12 | 88% | 6:43 |\n| DSH Creator | 19/30 | 15/21 | 4/9 | $3.28 | $0.12 | 84% | 6:43 |\n| Claude Code | 19/30 | 16/21 | 3/9 | $18.34 | $0.29 | 68% | 9:37 |\n| Pi | 18/30 | 16/21 | 2/9 | $2.43 | $0.07 | 79% | 7:33 |\n| DSH Standard | 18/30 | 16/21 | 2/9 | $3.46 | $0.12 | 86% | 6:16 |\n| DSH PTC | 18/30 | 16/21 | 2/9 | $4.58 | $0.14 | 87% | 7:43 |\n| Kimi Code | 17/30 | 14/21 | 3/9 | $3.65 | $0.18 | 88% | 7:55 |\n| DSH Minimal | 17/30 | 15/21 | 2/9 | $4.72 | $0.12 | 85% | 5:41 |\n| Oh My Pi | 17/30 | 16/21 | 1/9 | $4.75 | $0.14 | 82% | 6:46 |\n| Exo | 16/30 | 16/21 | 0/9 | $1.05 | $0.07 | 70% | 6:17 |\n| Hermes | 15/30 | 13/21 | 2/9 | $2.90 | $0.17 | 86% | 6:57 |\n| OpenCode | 15/30 | 15/21 | 0/9 | $3.24 | $0.06 | 78% | 6:27 |\n\nCost per pass is total spend on all 30 tasks divided by passes. Median cost, cache rate, and time cover passing tasks only. Mouse was the only harness to solve `kv-store-grpc`\n\nand `scc-bounded-memory-spilling`\n\n.\n\n## Cost\n\nMouse is fourth on cost per pass and last on median cost. Each verification round is another model call, which increases the harness's costs. Our next move is to improve cache and compaction to bring these costs down.\n\n## Terminal-Bench\n\n| Task | Mouse | Steps | Cost | Time | Field |\n|---|---|---|---|---|---|\n| regex-log | Pass | 21 | $0.51 | 7:50 | 11/12 |\n| openssl-selfsigned-cert | Pass | 13 | $0.20 | 2:44 | 12/12 |\n| polyglot-c-py | Pass | 14 | $0.15 | 2:35 | 12/12 |\n| sqlite-db-truncate | Pass | 21 | $0.32 | 4:17 | 12/12 |\n| git-leak-recovery | Pass | 15 | $0.14 | 2:06 | 12/12 |\n| log-summary-date-ranges | Pass | 21 | $0.25 | 3:32 | 12/12 |\n| constraints-scheduling | Pass | 15 | $0.31 | 4:49 | 12/12 |\n| gcode-to-text | Fail | 27 | $0.71 | 9:54 | 5/12 |\n| dna-insert | Pass | 18 | $0.71 | 11:05 | 2/12 |\n| largest-eigenval | Fail | 20 | $1.03 | – | 0/12 |\n| merge-diff-arc-agi-task | Pass | 19 | $0.30 | 4:03 | 12/12 |\n| vulnerable-secret | Pass | 17 | $0.18 | 2:15 | 12/12 |\n| extract-elf | Fail | 13 | $0.49 | 7:49 | 2/12 |\n| build-cython-ext | Pass | 47 | $0.93 | 13:08 | 12/12 |\n| kv-store-grpc | Pass | 17 | $0.15 | 2:32 | 0/12 |\n| chess-best-move | Pass | 18 | $0.59 | 6:13 | 5/12 |\n| db-wal-recovery | Pass | 13 | $0.15 | 2:23 | 12/12 |\n| code-from-image | Pass | 8 | $0.09 | 1:09 | 4/12 |\n| modernize-scientific-stack | Pass | 10 | $0.21 | 3:27 | 12/12 |\n| multi-source-data-merger | Pass | 10 | $0.19 | 3:07 | 12/12 |\n| sanitize-git-repo | Pass | 26 | $0.47 | 3:58 | 10/12 |\n\nField is how many of the 12 published harnesses passed the task.\n\n## DeepSWE\n\n| Task | Mouse | Steps | Cost | Time | Field |\n|---|---|---|---|---|---|\n| anko-typed-variable-bindings | Fail | 52 | $2.64 | 21:01 | 4/12 |\n| arktype-json-schema-refs-dependencies | Pass | 190 | $10.32 | 89:36 | 2/12 |\n| fastapi-deprecation-response-headers | Pass | 150 | $7.08 | 85:32 | 4/12 |\n| httpx-multipart-response-parsing | Pass | 141 | $6.06 | 70:48 | 5/12 |\n| expr-try-catch-errors | Pass | 134 | $8.59 | 44:43 | 2/12 |\n| python-statemachine-state-data-scoping | Pass | 201 | $13.49 | 73:21 | 7/12 |\n| katex-multicolumn-array-spans | Fail | 117 | $5.65 | 44:52 | 1/12 |\n| scc-bounded-memory-spilling | Pass | 84 | $4.65 | 52:44 | 0/12 |\n| meriyah-explicit-resource-declarations | Fail | 139 | $8.47 | 65:49 | 1/12 |\n\nTwo of the three misses passed nearly all of their hidden tests, 92 of 94 and 46 of 49.\n\n## Setup\n\n- Harbor 0.22, the benchmark's runner, on a Google Cloud VM with 4 tasks in parallel.\n- Kimi K3 through OpenRouter, pinned to the Fireworks servers the benchmark uses.\n- One trial per task. Cost at list price. Time is wall clock per task.\n\nMouse is built on top of [OpenCode](https://github.com/sst/opencode). Run date: September 3, 2026.", "url": "https://wpnews.pro/news/mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops", "canonical_source": "https://mouse.dev/blog/mouse-on-frontierharness/", "published_at": "2026-09-04 04:01:03+00:00", "updated_at": "2026-09-04 04:25:48.494571+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "ai-tools"], "entities": ["Mouse", "FrontierHarness", "Kimi K3", "OpenCode", "Terminal-Bench 2.1", "DeepSWE", "Codex", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops", "markdown": "https://wpnews.pro/news/mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops.md", "text": "https://wpnews.pro/news/mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops.txt", "jsonld": "https://wpnews.pro/news/mouse-tops-frontierharness-at-24-30-with-kimi-k3-using-completion-loops.jsonld"}}