xAI's latest model hits 51.2% Pass@1 accuracy but trails Anthropic's Fable 5 by a significant margin in real-world software engineering tests
The AI coding benchmark wars just got a new entry, and the scoreboard is already reshaping how enterprises think about which models to deploy. xAI’s Grok 4.5 has secured second place on the Mercor APEX-SWE leaderboard, posting a 51.2% Pass@1 accuracy score. That sounds impressive until you look one row up and see Anthropic’s Claude Fable 5 sitting at 65.5%.
What APEX-SWE actually measures #
APEX-SWE was launched in March 2026 by Mercor in collaboration with Cognition. The benchmark focuses specifically on real-world production software engineering tasks, with particular emphasis on integration work and observability challenges, testing whether a model can do the kind of messy, interconnected coding work that actual engineering teams deal with every day.
Fable 5, released in early June 2026 and optimized specifically for high-capability reasoning and agentic tasks, leads the board at 65.5% plus or minus 6.2%. Grok 4.5 sits at 51.2% plus or minus 6.0%.
Grok 4.5’s case for itself #
Grok 4.5, which launched in early July 2026, secured the top ranking on AutomationBench-AA with a 51% score, a separate benchmark that evaluates automation capabilities across agentic workflows.
xAI has positioned Grok 4.5 as delivering comparable results to competitors at a cost-per-task that is 4 to 17 times lower, depending on the workload.
The absence of blockchain-specific evaluation criteria in APEX-SWE also points to a gap that has not been filled. Real-world software engineering benchmarks have not yet incorporated the specific complexity of Solidity development, cross-chain integration testing, or on-chain state management as distinct evaluation categories.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our