Ziva runs docked in the Unity editor now, in beta, so we reran our Godot benchmark there: build a playable Flappy Bird as an agent, three prompts, 20-minute cap each. Claude ran on a Pro/Max subscription.
| Model | Tool calls | Wall time | Build cost | Rubric | Stars |
|---|---|---|---|---|---|
| DeepSeek v4 Flash (free) | 158 | 45.6 min (2 capped) | $0.61 | 9/10 | 3/5 |
| DeepSeek v4 Pro | 413 | 42.9 min (1 capped) | $11.96* | 9/10 | 3/5 | Claude Sonnet 5 (BYO) | 384 | 28.2 min | $7.34* | 5/10 | 2/5 | Claude Opus 5 (BYO) | 213 | 33.6 min | $14.73* | 10/10 | 4/5 |
- List-price equivalents, not billed: Claude ran on a subscription and this account has a 100%-off promotion on Pro. Flash’s 61 cents is real spend. Sonnet 5 wrote the cleanest code and a game you cannot play. The scripts are correct; the scene is not. The camera sees y −5 to +5; Sonnet put the Ground at y = −250, so the bird dies before the first pipe arrives.
Opus 5 shipped a working game and, unprompted, a 306-line PlayMode suite — twelve tests, run sixteen times, ending at thirteen passing. One failure (“an in-flight keypress must not skip the overlay”) is why it added a restart delay.
DeepSeek v4 Flash, the free model, shipped a full game for 61 cents — built entirely in code — but capped on two turns. Same prompts cost $0.18 in Godot.
DeepSeek v4 Pro matched that 9/10 in 413 calls, but made zero cache reads to Flash’s 10.4 million — most of the twenty-fold cost gap.
Unity went from 12 tools to 21 this release, adding run_tests
, find_gameobjects
and
manage_asset
. Two of four models used run_tests
unasked.
Caveats: n=1 per model, one task type, wall times not comparable across engines. One Unity-beta defect cost three runs — a changed-API edit raises a modal dialog that blocks the
editor, and -accept-apiupdate
doesn’t stop it. The beta is in
Ziva.