CloudOpsBench: AI benchmark shows FAI models struggling with cloud infra

The CloudOpsBench v1.0 leaderboard shows the top AI agent configuration, claude-fable-5-1 (max) run through claude-code, resolving only 60.6% of real cloud operations tasks, at a cost of $404.3 and 38.6 hours across 97.9M tokens. The next closest entries were gpt-6-astra (max) via codex at 58.9% resolution ($310.9, 16.5h, 98.0M tokens) and gemini-4-argon (high) via gemini-cli at 57.5% ($100.5, 34.2h, 173.9M tokens). The results indicate that even the strongest agent harnesses fail roughly four in ten cloud infrastructure tasks, with grok-4.7 (high) on cursor-cli at 56.5% and claude-opus-5-5 (high) on terminus-2 at 47.8% rounding out the published rankings.

CloudOpsBench A benchmark for AI agents on real cloud operations | Model leaderboard. Select a row to view details. | | | | | | | | |---|---|---|---|---|---|---|---| | Rank | Model | Harness | Resolution rate https://cloudopsbench.ai/?version=v1.0&sort=asc&view=table&metric=cost | Release date https://cloudopsbench.ai/?version=v1.0&sort=release&view=table&metric=cost | Time https://cloudopsbench.ai/?version=v1.0&sort=time&view=table&metric=cost | Tokens https://cloudopsbench.ai/?version=v1.0&sort=tokens&view=table&metric=cost | Cost https://cloudopsbench.ai/?version=v1.0&sort=cost&view=table&metric=cost | |---|---|---|---|---|---|---|---| | 01 https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max | claude-fable-5-1 max https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max claude-code | claude-code https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max | 60.6% | | 38.6h https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max | 97.9M https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max | $404.3 https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-fable-5-1%2Beffort-max | | 02 https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max | gpt-6-astra max https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max codex | codex https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max | 58.9% | | 16.5h https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max | 98.0M https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max | $310.9 https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-max | | 03 https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high | gemini-4-argon high https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high gemini-cli | gemini-cli https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high | 57.5% | | 34.2h https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high | 173.9M https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high | $100.5 https://cloudopsbench.ai/results?version=v1.0&model=gemini-cli%2Bgemini-4-argon%2Beffort-high | | 04 https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high | grok-4.7 high https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high cursor-cli | cursor-cli https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high | 56.5% | | 33.2h https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high | 178.3M https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high | $138.8 https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.7%2Beffort-high | | 05 https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high | gpt-6-astra high https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high codex | codex https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high | 54.2% | | 16.1h https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high | 62.4M https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high | $188.4 https://cloudopsbench.ai/results?version=v1.0&model=codex%2Bgpt-6-astra%2Beffort-high | | 06 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high | grok-4.7 high https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high grok-build | grok-build https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high | 48.3% | | 25.9h https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high | 173.4M https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high | $127.3 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-high | | 07 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high | claude-opus-5-5 high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high | 47.8% | | 19.5h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high | 22.6M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high | $45.1 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-opus-5-5%2Beffort-high | | 08 https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high | claude-opus-5-5 high https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high claude-code | claude-code https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high | 46.7% | | 14.0h https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high | 37.4M https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high | $36.6 https://cloudopsbench.ai/results?version=v1.0&model=claude-code%2Bclaude-opus-5-5%2Beffort-high | | 09 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high | claude-fable-5-1 high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high | 43.8% | | 19.0h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high | 37.3M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high | $319.1 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bclaude-fable-5-1%2Beffort-high | | 10 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high | gemini-3.8-flash high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high | 42.8% | | 33.2h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high | 337.4M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high | $70.0 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgemini-3.8-flash%2Beffort-high | | 11 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high | gpt-6-astra high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high | 42.8% | | 12.9h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high | 19.2M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high | $94.6 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bgpt-6-astra%2Beffort-high | | 12 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium | grok-4.7 medium https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium grok-build | grok-build https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium | 40.0% | | 21.4h https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium | 121.6M https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium | $91.6 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.7%2Beffort-medium | | 13 https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high | grok-4.6 high https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high cursor-cli | cursor-cli https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high | 39.7% | | 19.7h https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high | 78.2M https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high | $63.8 https://cloudopsbench.ai/results?version=v1.0&model=cursor-cli%2Bgrok-4.6%2Beffort-high | | 14 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium | grok-4.6 medium https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium grok-build | grok-build https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium | 36.7% | | 16.2h https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium | 59.4M https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium | $51.2 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-medium | | 15 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high | grok-4.6 high https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high grok-build | grok-build https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high | 36.1% | | 17.7h https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high | 72.0M https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high | $61.7 https://cloudopsbench.ai/results?version=v1.0&model=grok-build%2Bgrok-4.6%2Beffort-high | | 16 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high | qwen3.8-max high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high | 33.7% | | 79.1h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high | 78.4M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high | $61.8 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bqwen3.8-max%2Beffort-high | | 17 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high | deepseek-v4.1-flash high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high | 21.8% | | 22.2h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high | 68.1M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high | $7.5 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bdeepseek-flash%2Beffort-high | | 18 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high | kimi-k3.1-pro high https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high terminus-2 | terminus-2 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high | 19.1% | | 14.8h https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high | 7.4M https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high | $10.7 https://cloudopsbench.ai/results?version=v1.0&model=terminus-2%2Bkimi-k3.1-pro%2Beffort-high |