Google is changing how it tests AI models for Android coding following new rankings led by Fable 5.
Google released the “Android Bench” earlier in the year as a glanceable ranking system for the best AI models. Those rankings are all based around the model’s ability to code for Android – a general ranking system it is not, but incredibly useful for developers.
Google says there are a couple of changes coming to how it ranks those AI models for Android development. The core benchmarking system is being changed to the standardized Harbor framework. As it stood, models like GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro Preview were ranked based on a mini-swe-agent v1 benchmark tool developed for general use.
Switching tracks and opening the system up to the Harbor framework allows Android developers to use the same tools to analyze AI models for individual use cases. In fact, Google says it’s opening up the Android Bench to users willing to submit Android development tasks. Those will be used to evaluate how models handle each scenario. Developers are also being invited to share their benchmark evaluations.
From the beginning, we’ve valued an open and transparent approach, which is why we made our original methodology and test harness publicly available on GitHub. You’ve asked for a way to provide feedback on our dataset, so now we’re taking collaboration a step further by giving you, the Android developer
community, a chance to shape Android Bench.
Android Bench adds Claude Fable 5 #
To top off the new approach, Google has refreshed the Android Bench list using the new framework for AI models. Each model between the closed-weight and open-weight variants has been reevaluated on the new testing bench.
Unsurprisingly, Claude Fable 5 sits at the top of the list with a rather comfortable lead. Google gave it a score of 84.5, a healthy 4 points ahead of GPT-5.5, ranked at 80.2. Claude Sonnet 5 is nearly 10 points lower than Fable 5. Anthropic’s power claims seem to be reasonable, and it’s worth noting that Fable 5 still has heavy restrictions put in place.
Most AI models that had a spot in the previous Android Bench rankings have received a new score, since the underlying rules have essentially changed. Here are the new rankings as Android Bench has them listed:
| Model | Score | Avg Latency | Avg Cost |
|---|---|---|---|
| Claude Fable 5 | 84.5 | 8.0 | $133.2 |
| GPT 5.5 | 80.2 | 15.7 | $138.3 |
| Claude Sonnet 5 | 76.2 | 12.3 | $99.9 |
| GPT 5.4 | 74.1 | 8.4 | $83.4 |
| Gemini 3.1 Pro Preview | 73.7 | 10.6 | $87.4 |
| Claude Opus 4.8 | 72.4 | 6.7 | $88.0 |
| GLM 5.2 | 72.2 | 38.9 | $117.0 |
| Gemini 3.5 Flash | 71.1 | 28.3 | $165.6 |
| Kimi K2.7 Code | 70.4 | 31.8 | $48.1 |
*FTC: We use income earning auto affiliate links.* [More.](https://9to5mac.com/about/#affiliate)
[our homepage](http://9to5google.com/)for all the latest news, and follow 9to5Google on
[exclusive stories](https://9to5google.com/feature/exclusive/),
[reviews](https://9to5google.com/guides/reviews/),
[how-tos](https://9to5google.com/guides/how-to/), and
[subscribe to our YouTube channel](https://www.youtube.com/9to5google)