The benchmark tests agents building 17 products from scratch, adding features in sequence and checking the finished apps in a browser.
By [Ryan Merket](https://runtimewire.com/author/ryan-merket)
· Published
Primary source: [replit.com](https://replit.com/blog/vibench-2-long-horizon-software-benchmark)
Why it matters #
Whole-product tests expose failures that feature-by-feature coding benchmarks can miss. ViBench's public harness gives teams a way to reproduce Replit's results, while its vendor-built test suite and partial holdout set are important limits on how broadly to apply the scores.
Replit has released ViBench 2.0, a benchmark that measures whether AI agents can build complete software products feature by feature. In Fireworks AI's runs, the top three models passed every test plan in 49% to 65% of builds, according to Replit's announcement and its methodology post.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.