I build Empryo, an AI coding agent for desktop and terminal. In my benchmark published on 16 August 2026, Empryo and pi both fixed all four bugs on Opus 5. The published round totals were $2.27 for Empryo and $3.10 for pi, summing the per-task costs.
I'm focusing on Opus because both agents passed every task in that slice, so the cost comparison doesn't trade away a successful fix. This isn't a claim about today's versions or every model. The wider report covers six models, and explicitly notes that pi can still be cheaper on inexpensive models.
The bugs came from hono and opencode. Hidden regression tests from the merged fixes graded the changes.
| Opus 5 task | Empryo | pi |
|---|---|---|
| Cookie mangling, hono | $0.468 | $0.269 |
| TrieRouter regexp, hono | $0.570 | $0.634 |
| Symlinked grep root, opencode | $0.762 | $1.19 |
| Revert by ID, opencode | $0.472 | $1.01 |
| Round total, rounded | $2.27 | $3.10 |
pi was cheaper on the cookie bug. Most of Empryo's net savings came from the two opencode tasks.
The published report uses a small number of attempts: mostly two per result, with some single runs. Both repositories are TypeScript. Empryo ran with 37 tools and its Genome code map; the tested pi configuration used four tools. This compares those configurations, not every possible pi setup.
The timing protocol also prebuilt Empryo's code index. Cold indexing was reported separately at 96-100 seconds for opencode and eight seconds for hono. Don't mistake an agent-working timer for a fresh-install timer. I'm keeping this post's comparison to cost and correctness.