Arena co-founders Anastasios Angelopoulos (@ml_angelopoulos), Wei-Lin Chiang (@infwinston) and Ion Stoica launched AutoEval on July 30th, giving new AI models a preliminary Arena Score before enough people have voted to produce a stable live ranking.
Outlined in a six-post thread on X and a technical blog post, AutoEval trains reward models on millions of pairwise preferences collected through Arena. Arena says the system can rank a model in under an hour, compared with the days required to accumulate human votes. The initial score carries an "AutoEval" label and is updated after sufficient live voting.
Angelopoulos and Chiang built the original Chatbot Arena while working in UC Berkeley's Sky Computing Lab. Both earned computer science doctorates at Berkeley and were postdoctoral researchers and housemates as the project became a business, according to a Berkeley News profile. The evaluation system emerged in 2023 after researchers working on the Vicuna open model needed a credible way to compare chatbot applications against models from larger labs.
AutoEval estimates votes before they arrive
AutoEval begins with prompts and responses sampled from Arena's live evaluations. Arena generates a response from the model being tested, then uses a pointwise reward model to score each prompt-response pair. Differences between those scores are converted into probability-weighted proxy votes and combined with existing human votes to calculate an Arena Score using the leaderboard's standard ranking method.
The system is trained to account for changing user preferences, ties, narrowing differences between frontier models and multi-turn conversations where a final vote may reflect decisions made much earlier in the exchange. Coding evaluations add another complication because correctness, technical quality and the resulting interface can all affect a user's choice.
Arena tested the approach using a temporal holdout designed to reduce the risk of grading models already represented in its training data. Arena trained the reward model on votes collected through the end of April 2026, then evaluated models released after that cutoff. The resulting AutoEval rankings had a correlation above 0.98 with the models' eventual live rankings, according to Arena.
Across more than 40 test models, Arena says AutoEval selected the stronger model in over 90% of comparisons when the final live scores were separated by at least 10 points. Its reported accuracy reached 100% for gaps above 15 points. Arena's test also defines where an early score becomes less useful: models within five points had overlapping confidence intervals in the live evaluation and were statistically indistinguishable.
Arena also says its text reward model predicted user preferences 8% to 10% more accurately than the Gemini 3 Flash, Gemini 3 Pro and GPT-5 judges it tested. Those figures come from Arena's own evaluation methodology. AutoEval currently covers text, vision, image generation and coding. Arena trained its text-to-image reward model on more than 3 million preference pairs and says the model led other pointwise reward models on Meta's MMRB2 benchmark.
Arena turns its voting history into a faster product
The release addresses a timing problem created by the pace of frontier-model launches. Public leaderboards become informative only after enough votes arrive, while model developers need comparative data as they choose checkpoints and prepare releases. AutoEval lets Arena use its accumulated voting history to estimate the eventual ordering, then hand the ranking back to live users for validation.
That approach also makes Arena's community data more commercially useful. In June, Angelopoulos said Arena had reached a $100 million annualized revenue run rate within eight months of introducing its enterprise offering. He described a platform with more than 10 million monthly visitors, 700 million conversations and 82 million human votes. The figures are self-reported, and the run rate reflects consumption-based revenue rather than conventional contracted annual recurring revenue.
Arena has raised $250 million across two announced rounds. Its $150 million Series A in January was led by Felicis and UC Investments, with participation from Andreessen Horowitz, The House Fund, LDVP, Kleiner Perkins, Lightspeed Venture Partners and Laude Ventures. Reuters reported that the financing valued Arena at $1.7 billion.
AutoEval applies Arena's central asset, its history of real user choices, to the period when human evaluation is slowest and commercial demand for an answer is highest. Its usefulness will depend on whether reward models continue tracking user preferences as model behavior changes. Arena's temporal testing and visible AutoEval labels give users a way to compare the early estimate with the live result that follows.