{"slug": "benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and", "title": "Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation", "summary": "Researchers released Benchmark Radar, a living database and search engine for AI benchmarks that catalogs 1,283 source records drawn from 4 benchmark catalogs and 12,916 numeric observations on 790 records. The system draws on 37 sources for daily discovery, comprising 13 direct connectors and 24 first-party research and engineering feeds, and covers LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. The release includes a web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface for offline queries, and reproducible analysis.", "body_md": "arXiv:2609.11115v1 Announce Type: new \nAbstract: Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores. We present Benchmark Radar, a living database and search engine for retrieval and discovery of AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. The system combines daily discovery of benchmark papers, repositories, datasets, and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. It retains source identities and citations so readers can inspect candidate benchmarks and their evaluation evidence. Daily discovery draws on 37 sources: 13 direct connectors and 24 first-party research and engineering feeds. The catalog contains 1,283 source records drawn from 4 benchmark catalogs and 12,916 numeric observations on 790 records. We describe collection and retrieval, audit the full catalog, and examine benchmark saturation, adoption trends, and the limits of score comparisons. A worked example walks through a complete prior-art search, showing how to query the catalog and inspect benchmark evidence when designing a new evaluation. We release the web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface (CLI) for offline queries, and reproducible analysis.", "url": "https://wpnews.pro/news/benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and", "canonical_source": "https://www.machinebrief.com/news/benchmark-radar-a-living-database-and-search-engine-for-ai-b-31g4", "published_at": "2026-09-12 04:00:00+00:00", "updated_at": "2026-09-12 05:27:41.884162+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools", "ai-agents"], "entities": ["Benchmark Radar", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and", "markdown": "https://wpnews.pro/news/benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and.md", "text": "https://wpnews.pro/news/benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and.txt", "jsonld": "https://wpnews.pro/news/benchmark-radar-a-living-database-and-search-engine-for-ai-benchmarks-and.jsonld"}}