Show HN: Compute:Arena – Community submitted local AI benchmarks

Compute:Arena launched a public leaderboard for community-submitted local AI benchmarks, open sourcing the internal testing harness used to track model and chip performance across AMD, NVIDIA, Apple Silicon, Intel, and Qualcomm hardware. The platform is live with a few hundred submissions already, allowing anyone to run open source models on their own hardware and submit results. The tool aims to simplify benchmarking open source models and quantization variants across the major chip vendors.

Benchmarking AI models on real hardware is way harder than it looks. Between AMD, NVIDIA, Apple Silicon, Intel, and Qualcomm, plus hundreds of open source models and quants, getting the test bench right is a challenge. So we're making it dead simple. We're open sourcing our internal testing harness. Anyone can run open source models on their own hardware and submit results to the public leaderboard. It's live now, with a few hundred submissions already. If you want to know how a specific model performs on a given hardware, chances are the data is already there. This is the same tool we use internally to track model and chip performance. Try it out and tell us what to improve Comments URL: https://news.ycombinator.com/item?id=49737278 https://news.ycombinator.com/item?id=49737278 Points: 2 Comments: 1