TempCloze: Can Video-LLMs Identify the Missing Middle?

Researchers introduced TempCloze, a video cloze benchmark designed to evaluate visual temporal reasoning in Video-LLMs while reducing linguistic shortcuts from option wording, answer correlations, or language priors. The benchmark targets the gap left by existing temporal reasoning evaluations that are often mediated by language.

Temporal reasoning benchmarks for Video-LLMs are often mediated by language, leaving room for linguistic shortcuts from option wording, answer correlations, or language priors. To reduce such shortcuts, we introduce TempCloze, a video cloze benchmark for evaluating visual temporal reasoning in Video