ReactBench is an evaluation for coding agents on realistic React work. Today’s models can pass tests in benchmarks but still write React that fails in production. Tests verify behavior, but they miss React performance, accessibility, and quality issues.
Here’s what ReactBench does differently:
A higher bar than just passing tests
Every solution must pass both behavioral tests andReact Doctor, our open-source, deterministic verifier for React code. Its 400+ rules scan the agent's code for broken effects, unnecessary renders, accessibility problems, and maintainability issues.Built by React experts
Our team builtReact Doctor★ 13.8k,React Scan★ 21.6k, andMillion.js★ 17.7k, used by engineers at GitHub, PayPal, Rippling, and Airbnb. We spent hundreds of hours curating tasks to measure real gaps in model capabilities.Real work in real codebases
ReactBench spans open-source repositories and realistic changes grounded in existing projects (not synthetic puzzles). Agents must implement, preserve behavior, and meet the React-specific quality checks.
Results #
We evaluate two complementary capabilities: implementing new features and improving existing code.
Write React
Implement a real feature or fix, pass held-out behavioral tests, and validate no new React Doctor issues were introduced.See exampleFix React
Refactor React failures from existing code while preserving existing behavior without introducing new React Doctor issues.See example
We report the average of the pass@1 across 5 trials.
ReactBench score vs. cost / output tokens
Ranked by score. Cost is the average per rollout.
| Model | Score | Cost |
|---|---|---|
| GPT 5.6 Sol · MediumOpenAI | 43.1% | $1.35 |
| GPT 5.6 Sol · XHighOpenAI | 43.1% | $1.43 |
| GPT 5.6 Sol · LowOpenAI | 42.7% | $1.38 |
| GPT 5.6 Sol · MaxOpenAI | 42.4% | $1.35 |
| Fable 5 · XHighAnthropic | 41.2% | $9.05 |
| GPT 5.6 Sol · HighOpenAI | 40.4% | $1.36 |
| Fable 5 · MaxAnthropic | 40% | $13.50 |
| GPT 5.6 Terra · MediumOpenAI | 38.0% | $0.53 |
| Fable 5 · LowAnthropic | 37.3% | $3.15 |
| GPT 5.6 Terra · XHighOpenAI | 36.9% | $0.51 |
| Fable 5 · HighAnthropic | 35.7% | $6.17 |
| Opus 4.8 · MaxAnthropic | 34.1% | $7.18 |
| GPT 5.6 Terra · HighOpenAI | 33.7% | $0.52 |
| Opus 4.8 · XHighAnthropic | 33.3% | $5.49 |
| GPT 5.6 Terra · LowOpenAI | 32.9% | $0.51 |
| GLM 5.2 · HighZ.ai | 32.9% | $1.99 |
| GPT 5.6 Terra · MaxOpenAI | 32.5% | $0.52 |
| Opus 4.8 · MediumAnthropic | 30.6% | $3.86 |
| Sonnet 5 · XHighAnthropic | 30.6% | $3.23 |
| Sonnet 5 · MaxAnthropic | 29.8% | $5.91 |
| GLM 5.2 · LowZ.ai | 29.8% | $2.06 |
| GLM 5.2 · MaxZ.ai | 29.8% | $2.12 |
| Opus 4.8 · HighAnthropic | 29.4% | $4.48 |
| Sonnet 5 · HighAnthropic | 27.5% | $2.44 |
| GPT 5.6 Luna · LowOpenAI | 26.7% | $0.20 |
| GPT 5.6 Luna · HighOpenAI | 25.9% | $0.21 |
| GPT 5.6 Luna · MaxOpenAI | 25.9% | $0.20 |
| GPT 5.6 Luna · MediumOpenAI | 25.9% | $0.21 |
| Opus 4.8 · LowAnthropic | 24.7% | $1.72 |
| Sonnet 5 · MediumAnthropic | 24.3% | $1.08 |
| GPT 5.6 Luna · XHighOpenAI | 21.6% | $0.21 |
| Kimi K2.7 CodeKimi | 20.4% | $1.40 |
| Sonnet 5 · LowAnthropic | 19.6% | $0.58 |
Score: a weighted aggregate of the rubric items. Solutions that don’t pass blocking criteria receive 0.
Cost: the mean cost of a model rollout in US dollars.
ReactBench remains unsaturated, even on leading models. GPT-5.6 Sol at XHigh has the highest observed aggregate score at 43.1%, while Claude Fable 5 at XHigh reaches 41.2%. Across evaluated configurations, Fable costs $7.97 per trial on average versus $1.37 for Sol, or 5.8× as much. At XHigh, Fable costs 6.3× as much as Sol. Even the best configurations solve fewer than half of benchmark task attempts.
We also evaluated task discrimination across model configurations. In the 51-task panel, 36 tasks (70.6%) had a cross-configuration standard deviation above 0.10.
In terms of balancing quality and cost, GPT-5.6 Terra at Medium reaches 38.0%, 5.1 percentage points behind GPT-5.6 Sol at XHigh, while using 10.8% fewer output tokens and costing 63.2% less per task. It retains most of the leading configuration’s performance at roughly one-third the cost, making it the strongest practical tradeoff for high-volume workloads.
GPT-5.6 Sol at XHigh achieved the highest overall pass rate. However, the differences among the leading configurations are too small to identify a clear winner with confidence.
Bugs dominate newly introduced React issues
| Model | Bugs per 100 trials | Performance per 100 trials | Accessibility per 100 trials | Maintainability per 100 trials | Issues per 100 trials |
|---|---|---|---|---|---|
| GPT-5.6 Sol | 15.7 | 1.9 | 0.3 | 0.3 | 18.2 |
| Fable 5 | 15 | 2.0 | 0.9 | 0.7 | 18.7 |
| Opus 4.8 | 18.5 | 2.5 | 0.9 | 1.6 | 23.6 |
| GPT-5.6 Terra | 21.5 | 4.1 | 0.7 | 0.3 | 26.7 |
| Sonnet 5 | 18.2 | 2.8 | 3.0 | 3.0 | 27.0 |
| GPT-5.6 Luna | 26.4 | 4.7 | 0.4 | 0 | 31.6 |
| GLM 5.2 | 25.7 | 9.1 | 0.5 | 0.5 | 35.8 |
| Kimi K2.7 Code | 46.7 | 8.9 | 5.2 | 6.7 | 67.4 |
Across 4,455 Write React trials, the evaluated models introduced 1,194 graded React Doctor issues. Bugs, including security findings, accounted for 925, or 77.5%. The most common issues involved list rendering and Hook correctness.
The verifier-outcome taxonomy shows that Write and Fix tasks fail differently. Among 2,486 failed Write trials, 1,623 (65.3%) failed behavioral tests but passed React Doctor. Among 3,219 failed Fix trials, 1,956 (60.8%) passed behavior but failed React Doctor.
Write tasks fail primarily on requested behavior, while Fix tasks fail primarily on unresolved React problems.
Why we built ReactBench #
React is the dominant frontend framework and the most popular target for coding agents. Roughly 70% of websites built on a JavaScript framework choose React.
We have seen the risks firsthand. React Doctor is our open source tool for scanning React issues used by engineers at PayPal, Rippling, Polymarket, and the Centers for Disease Control and Prevention (CDC). Adoption is largely driven by the increase of model-generated code that makes it easier for subtle defects to reach production. As models write more React, small mistakes can propagate at enormous scale. In the worst cases, these defects lead to production failures:
Outages
IncorrectuseEffect
usage takes down production. Cloudflare traced itsSeptember 2025 dashboard and API outageto one effect with a faulty dependency. Despite human review and test, the bug still shipped to production.Lost revenue
Slow interfaces cost sales. Google and Deloitte found that a0.1s mobile speedup increased retail conversions by 8.4%. Rakuten 24 increasedrevenue per visitor by 53.4%after improving Core Web Vitals.Legal risk
Interfaces that are not accessible exclude customers and expose companies to lawsuits. WebAIM found automated accessibility failures on95.9% of the top one million home pagesand US federalweb accessibility lawsuits increased by 27% in 2025.
Building ReactBench #
Each task type uses a different construction and evaluation process.
Write React
Write React measures whether an agent can implement real work without introducing a React regression. Each task starts from a merged pull request in an open-source repository. The agent receives the base repository and an issue-style instruction. The reference patch and verifier are not provided.
Behavior
The verifier injects the hidden behavior tests and verifies the results in a separate container after the agent completes.React health
A pinned React Doctor scan compares the submission with the base commit. If a new issue is detected, it fails.
Fix React
Fix React measures whether an agent can recognize and refactor React problems from source code alone. We select a component with known React issues and ask the agent to improve it without naming those findings.
React health
The agent must remove every target without introducing another graded React issue. The grader ignores line shifts.Behavior
The task’s test suite must still pass to ensure refactoring the component did not regress it in functionality.
We strip React Doctor and other React-aware linters from the agent image. Only the verifier contains the pinned scanner and its clean baseline.
React Doctor as a verifier
We built React Doctor, an open-source, deterministic React verifier with 400+ rules to cover problems outside behavioral test assertions: Correctness
Catches conditional hooks, unstable list keys, hydration mismatches, and deprecated React APIs.State and effects
Flags derived or duplicate state, useEffect abuse, and infinite rerenders.Performance
Finds unnecessary renders, layout thrashing, sequential async work, and bundle-heavy imports.Accessibility and security
Detects unlabeled controls, keyboard-inaccessible interactions, unsafe HTML, and secrets exposed to client bundles.
ReactBench does not use every rule or an LLM-as-a-judge. We curate the specific React Doctor rules used for grading, then pin the scanner version and clean baseline.
From real pull requests to verified tasks
ReactBench converts merged changes from public repositories into evaluation tasks.
Mine candidates
We collect merged pull requests from open-source React projects.Filter candidates
We built automated filters to check if there were meaningful changes to product code. Then, reviewers assess exceptions that cover broader feature work or important projects, and have realistic behavioral tests.Author tasks
Each reviewer converts validated pull requests into a realistic issue-style instruction. If the candidate’s tests are too strict, reviewers will rebuild a custom test harness to validate behavior.Validate behavior
The tests must fail against the pinned base commit and pass with the reference solution. The unchanged repository must score 0, while the reference solution must score 1.Test the verifier
An adversarial agent attempts to earn full credit without implementing the requested behavior.Pin the release
For each task, we pin the source commit, environment, test suite, reference solution, and verifier configuration. We also have a versioned manifest that records a hash for each published artifact.
Filter criteria
| Filter | Requirement |
|---|---|
| Changed code | At least 50 changed lines |
| Additions | At least 40 added lines |
| React signal | Changes React product code |
| Recency | Merged on or after February 1, 2026 |
| Repository size | Fewer than 20,000 GitHub stars during our audit |
92% percent of Write React tasks meet both the recency and repository-size criteria. These thresholds are implemented to reduce likely training exposure.
Behavioral coverage
Every task defines deterministic behavioral checks before release. We use the repository’s existing Vitest, Jest, or Playwright tests when they fully specify the requested behavior.
When existing tests leave gaps, we build a separate harness around observable behavior. Reviewers confirm that the verifier accepts valid implementations beyond the reference solution.
Clean-room grading
The agent and verifier run in separate containers. Before grading, the verifier restores its own Git metadata, dependencies, protected configuration, hidden tests, and pinned React Doctor binary.
Continuous integration (CI) confirms that both containers use the same source commit. The verifier runs offline and doesn’t make external network requests. The agent cannot access hidden tests or the reference solution during its run.
Separating model failures from benchmark failures
A zero can reflect a model failure, unclear instructions, a brittle test, a verifier error, an infrastructure failure, or an invalid run.
Reviewers inspect the trajectory and final patch before including a result in model-performance conclusions. They classify each reviewed rollout as a legitimate solve, genuine model failure, verifier false positive, verifier false negative, or invalid run.
Testing ReactBench against reward hacking
The adversarial agent probes the test infrastructure, reward files, Git history, and React Doctor inputs. Its goal is to earn full credit without implementing the requested behavior.
We fix or remove any task that exposes a cheat, then rerun every control. This final check tests whether ReactBench measures the requested work rather than an agent’s ability to exploit the grader.
Limitations #
React Bench evaluates agents, not models in isolation. Differences among Codex CLI, Claude Code, Cursor, Gemini CLI, and other harnesses can affect results.
Every task combines behavioral tests with React Doctor as a verifier. These checks catch important React problems, but they cannot guarantee visual correctness and other important attributes.
Also, the benchmark primarily covers open-source React projects. Results may not generalize to proprietary codebases, different architectures, or other frontend frameworks or setups.
React Bench will publish versioned tasks, solutions, manifests, and release records so others can inspect and reproduce each evaluated task set.
Future work #
We plan to compare more models, on more effort levels. Additionally, we plan to run not only on the commercial harnesses, but also on mini-swe-agent to get a more solid baseline.
We also plan to expand into visual design and other frontend frameworks and diversify the repositories and task types represented.
We’re constantly improving ReactBench. Report benchmark issues on GitHub. Model labs and coding-agent teams can reach out to evaluate models on our held-out task set, get early access to new benchmarks, or collaborate with us.
Acknowledgments #
ReactBench exists because engineers and open-source maintainers contributed their time and expertise. We thank everyone who reviewed its tasks, drafts, and evaluation infrastructure.
Research
Design
External contributors
Michał Pierzchała(React Native team)Jovi De Croock(Preact core team)Dev Agrawal(SolidJS core team)ryoppippi(creator of ccusage)Rahim Alwer(creator of Vidstack, Video.js v10 team)Tiger Abrodi(early Lovable team)Isabelle Reksopuro
Data Advisory
Parth Patel(Founding Engineer at AfterQuery) Special thanks to the engineers who built, verified, and reviewed ReactBench’s tasks and evaluation infrastructure, and to the maintainers of the open-source repositories represented in the current task set.