Via eachlabs.ai
The Y Combinator-backed startup has amassed over 5 million users who rank AI outputs, feeding critical human preference data back to frontier labs like OpenAI and Anthropic.
There’s a fundamental problem with judging whether an AI-generated website looks good: machines are terrible at it. Automated metrics can tell you if an image has the right number of pixels or if a layout loads fast enough, but they can’t tell you if something actually looks nice. That’s a human job, and a San Francisco startup called Design Arena just raised $8M to make sure humans keep doing it at scale.
Founded in 2025 by Grace Li, Design Arena emerged from the Y Combinator S25 cohort with a deceptively simple concept. Show users two AI-generated designs side by side, let them pick the better one, and aggregate millions of those votes into Elo-style rankings. Think of it as the old “Hot or Not” website, except instead of rating people, you’re rating whether GPT-5.6 or GLM-5.2 produces a prettier landing page.
How crowdsourced taste becomes AI training data #
The platform has already attracted more than 5 million users across 140-plus countries. Users engage in blind head-to-head comparisons of AI-generated content spanning web design, images, videos, audio, and UI/UX layouts. No labels, no branding cues. Just raw output versus raw output.
Those votes get processed into public leaderboards that function similarly to LMSYS Chatbot Arena, the widely referenced benchmark for large language models. But where Chatbot Arena focuses on text-based reasoning and conversation quality, Design Arena zeroes in on aesthetics and subjective visual preference.
Recent benchmarks on the platform have shown GPT-5.6 ranking at the top of web design evaluations, while GLM-5.2 has claimed leading positions in certain other design segments. Frontier AI labs including OpenAI and Anthropic are paying attention to Design Arena’s leaderboards as a real signal for how their models perform in creative and visual domains.
The AI evaluation infrastructure play #
The $8M raise positions Design Arena within a growing category of startups building evaluation infrastructure for AI, rather than the AI models themselves. Instead of paying armies of annotators to provide structured feedback, Design Arena turns evaluation into a casual activity. Users vote because it’s quick and mildly entertaining, not because they’re being compensated. That creates a dataset with genuine geographic and demographic diversity, drawn from those 140-plus countries.
Traditional benchmarks for visual AI tend to rely on metrics like FID scores or CLIP similarity, which measure technical fidelity but miss the subjective “does this actually look appealing” dimension entirely. Design Arena fills that gap with real human judgment at a scale that would be prohibitively expensive through conventional annotation pipelines.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our