Where Do Vision Language Models Fail at 3D Reasoning?

Researchers introduced BedBench, a controlled benchmark of thirteen procedurally generated tasks, to characterize how vision-language models can get individual judgments right while failing to recover and consistently use the exact properties of a 3D object. The benchmark spans individual judgments and composite tasks, targeting the gap between correct single answers and consistent 3D reasoning.

Vision-language models can get individual judgments right while struggling to recover and consistently use the exact properties of a 3D object. To characterize this gap, we introduce BedBench, a controlled benchmark of thirteen procedurally generated tasks spanning individual judgments and composite