{"slug": "ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth", "title": "OVEarth-Bench: Evaluating Category Breadth and Query Diversity for Open-Vocabulary Earth Observation", "summary": "Researchers introduced OVEarth-Bench, a benchmark for open-vocabulary Earth observation that expands category breadth and query diversity, finding that current methods perform limitedly, with MLLM-based methods achieving the strongest overall performance and EO-specific methods generally underperforming general models. The benchmark supports mask and box localization under a unified zero-shot protocol and is released at https://earth-insights.github.io/OVEarth-bench.", "body_md": "arXiv:2607.27278v1 Announce Type: new\nAbstract: Open-vocabulary Earth observation (EO) aims to localize geospatial concepts specified in natural language rather than a fixed label set. Existing benchmarks, however, usually cover narrow category vocabularies or limited query forms. To fill this gap, we introduce OVEarth-Bench, which extends existing evaluation in two directions: category breadth, through broad hierarchical category coverage with positive and negative expressions, and query diversity, through vocabulary, referring, and reasoning queries. The benchmark supports mask and box localization under a unified zero-shot protocol. We evaluate a broad set of general and EO-specific methods. The evaluation reveals that: (1) the performance of current methods remains limited, while broader category coverage yields more stable model rankings; (2) MLLM-based methods achieve the strongest overall performance; and (3) EO-specific methods generally underperform general models and rarely match the strongest methods. These findings provide guidance for future open-vocabulary EO method design and highlight the importance of developing more realistic, diverse, high-quality, and large-scale benchmarks for reliable evaluation. Our data and evaluation package are released at https://earth-insights.github.io/OVEarth-bench.", "url": "https://wpnews.pro/news/ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth", "canonical_source": "https://arxiv.org/abs/2607.27278", "published_at": "2026-07-31 04:00:00+00:00", "updated_at": "2026-07-31 04:38:44.558758+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "natural-language-processing", "ai-research"], "entities": ["OVEarth-Bench", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth", "markdown": "https://wpnews.pro/news/ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth.md", "text": "https://wpnews.pro/news/ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth.txt", "jsonld": "https://wpnews.pro/news/ovearth-bench-evaluating-category-breadth-and-query-diversity-for-open-earth.jsonld"}}