{"slug": "pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world", "title": "Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments", "summary": "Researchers introduced Pro-Bench, a prompt-conditioned benchmark for open-vocabulary visual grounding in heterogeneous real-world environments, comprising 13k+ RGB frames from subterranean, industrial, indoor, outdoor and urban robotic domains, 74.5k manual instance annotations and 515 target queries. Benchmarking 16 open-vocabulary model configurations in strict zero-shot inference, the team found prompt-robustness is strongly architecture-dependent: 10 of 16 configurations perform best with short category labels, while free-form queries yield the highest accuracy for only one, and similar aggregate mAP can conceal substantial differences in consistent target recovery across reformulations.", "body_md": "arXiv:2609.27076v1 Announce Type: new \nAbstract: Open-vocabulary visual grounding enables robots to localise task-relevant entities from natural-language queries without dependence on predefined perceptual taxonomies. However, existing benchmarks largely rely on short category labels and web-scraped imagery, leaving it unclear whether open-vocabulary models can robustly ground diverse queries and visual conditions under real deployments. We introduce \\textbf{Pro-Bench}, a prompt-conditioned benchmark for open-vocabulary visual grounding in heterogeneous, real-world environments. Pro-Bench includes $13k+$ RGB frames from independent robotic domains (subterranean, industrial, indoor, outdoor, urban), with $74.5k$ manual instance annotations and $515$ target queries covering categorical, attributive, relational, affordance, state, part-whole, negative, and compositional semantics. We benchmarked $16$ open-vocabulary model configurations in strict zero-shot inference, measuring localisation accuracy across IoU thresholds, end-to-end inference latency, prompt-induced performance variation, and target recovery consistency. Our results show that prompt-robustness is strongly architecture-dependent. Most model configurations ($10/16$) perform best with short category labels, whereas free-form queries yield the highest accuracy for only one. Moreover, similar aggregate mAP can conceal substantial differences in consistent target recovery across reformulations. Pro-Bench enables systematic evaluation of these gaps and supports prompt-robust visual grounding. Pro-Bench: https://pro-bench.github.io/.", "url": "https://wpnews.pro/news/pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world", "canonical_source": "https://arxiv.org/abs/2609.27076", "published_at": "2026-09-24 04:00:00+00:00", "updated_at": "2026-09-24 04:01:39.855952+00:00", "lang": "en", "topics": ["computer-vision", "robotics", "ai-research", "machine-learning"], "entities": ["Pro-Bench", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world", "markdown": "https://wpnews.pro/news/pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world.md", "text": "https://wpnews.pro/news/pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world.txt", "jsonld": "https://wpnews.pro/news/pro-bench-prompt-robust-open-vocabulary-visual-grounding-across-real-world.jsonld"}}