# Atlarix vs opencode on Terminal-Bench 2.0 — same model, only the harness changes (k=1, receipts included)

> Source: <https://dev.to/amariahak/atlarix-vs-opencode-on-terminal-bench-20-same-model-only-the-harness-changes-k1-receipts-54nk>
> Published: 2026-06-29 19:22:30+00:00

I build [Atlarix](https://atlarix.dev), an agent workstation for open-weight models. The core claim behind it is that the harness — retrieval, tool surface, control loop — is what lets an open-weight model perform, not just the model's raw weights. This post is me trying to falsify that claim with a controlled run, and publishing every output file so you can check it.

Short version: on Terminal-Bench 2.0, single attempt, **Atlarix resolved 42/89 and opencode resolved 39/89** on the same model. That 3-task gap is **within k=1 noise** — I'm not claiming a win. What it shows is that the harness isn't bottlenecking the model. Details and caveats below; raw files at the end.

The only variable is the harness. Everything else is pinned identical across both agents.

`terminal-bench/terminal-bench-2`

— all 89 tasks, one isolated container each, automated verifiers.`minimax/minimax-m3`

, routed through OpenRouter, pinned to a single provider at `-e modal`

), one container per task.`-k 1`

.`--timeout-multiplier 1`

(same for both).`--max-retries 3`

(same for both).

```
# Atlarix harness
harbor run -d terminal-bench/terminal-bench-2 \
  -m openai/minimax/minimax-m3 \
  -n 24 -k 1 -y --timeout-multiplier 1 --max-retries 3 \
  -e modal --agent-import-path atlarix_tb:AtlarixAgent

# opencode harness (same model + provider + infra)
harbor run -d terminal-bench/terminal-bench-2 \
  -m bench/minimax/minimax-m3 \
  -n 24 -k 1 -y --timeout-multiplier 1 --max-retries 3 \
  -e modal --agent-import-path atlarix_tb.opencode_proxy:BenchOpenCodeAgent
```

(`-n 24`

is concurrency — how many containers run in parallel — not a task count. All 89 tasks run.)

| Harness | Resolved | Score |
|---|---|---|
| Atlarix | 42 / 89 | 47% |
| opencode | 39 / 89 | 44% |

**k=1 means one sample per task.** The official Terminal-Bench leaderboard requires **k=5** specifically to measure run-to-run variance. A 3-task difference at k=1 is inside that noise band. So this is **not** a leaderboard result and not a claim that Atlarix beats opencode. The honest takeaway: an open-weight model performs about as well under Atlarix as under a strong existing harness — the harness isn't holding it back.

**~25% of tasks timed out — for both harnesses.** At native timeout (×1), roughly a quarter of tasks hit `AgentTimeoutError`

on each side and count as unresolved. So the sub-50% absolute scores aren't all capability failures; a meaningful share are wall-clock on heavy tasks. The timeout ceiling is identical for both agents, so the comparison stays fair — but that's why neither number is higher.

Atlarix's desktop app asks for human approval before every file write and command — a core safety feature. Benchmarks run unattended, so I grant that approval once via an explicit operator flag (`ATLARIX_AUTONOMOUS_DANGER=1`

). Without it, any task needing an install or privileged command is blocked and fails.

This is **not** an advantage over opencode — every agent auto-approves to run an automated benchmark; it's inherent to running unattended. Stating it for full transparency. The flag is off by default; the interactive app always asks.

The exact Atlarix bundle I ran is a public, Electron-free headless build: `atlarix-headless-linux-amd64.tar.gz`

. The benchmark is the open-source Harbor framework. The raw Harbor result files — per-task pass/fail for both harnesses — are published unedited. Nothing is hand-typed.

Everything (raw `result.json`

for both sides, `summary.csv`

, exact bundle, full setup): [atlarix.dev/benchmark](https://atlarix.dev/benchmark)

If you spot something wrong in the result files, that's the point — tell me.

*Built in Nairobi.*