So here’s the thing - every model is benchmaxxing, so how do you know which one is actually the best to use? The thing we’ve found most reliable is to benchmark it with something that’s relevant to you.
TL;DR #
Kimi actually beat Fable, but it’s unusably slow. Sol’s speed and affordability make it better for game development, even if the end result is slightly worse than the other two.
| Model | World | Conversation | Cost | Time |
|---|---|---|---|---|
| GPT 5.6 Sol | 6 | 9 | $2.94 | 9.7 min |
| Kimi 3 | 8 | 7 | $3.14 | 79 min |
| Claude Fable 5 | 7 | 6 | $13* | 32.5 min |
The Prompt #
Make a 3D parkour world where you compete against bots for time. There should be 3 races in the world, and you can start any of the races by walking over to their start point. Use the assets in Kenney_Platformer_Kit/ to build your world.
Created Worlds #
| GPT 5.6 Sol | Kimi 3 | Claude Fable 5 |
|---|---|---|
PlayPlay## Conversation Analysis I ran each conversation transcript through Opus 4.8 and had it come up with a grade:
GPT 5.6 Sol— tightest loop; headless logic checks, playtests only where they counted.** 9/10**·conversation** Kimi 3**— most thorough of the three (12 playtests), but redundant and slow.** 7/10**·conversation** Claude Fable 5**— sharpest debugging, but ships blind: it never sees its own playtest frames.** 6/10**·conversation
My Thoughts #
Kimi is lowkey very good here, but it’s painfully slow. Maybe it’s a reliability issue on their GPUs and it’ll be better once more people can self-host, but the truth is right now the time it takes to do things makes it borderline unusable.
Credits #
Assets used for the game generation are provided by Kenny.nl