StateBridge: Phase- and Track-Grounded Reasoning for Synthetic-to-Real Traffic Video Captioning and VQA

A research team's StateBridge framework scored an official S2 score of 55.4679 on AI City Challenge 2026 Track 2 while training only on synthetic data, according to the arXiv paper 2610.31898v1. StateBridge combines phase-aware clip construction, pedestrian and vehicle track summaries, and structured prompts to fine-tune public vision-language models on synthetic captioning and visual question answering, then merges outputs through fact-locked caption fusion and evidence-diverse VQA consensus. The result indicates structured evidence transfer can help traffic safety captioning and VQA models generalize from simulated scenes to real-world traffic.

arXiv:2610.31898v1 Announce Type: new Abstract: Traffic safety videos provide rich evidence for understanding interactions between pedestrians and vehicles, but synthetic-to-real evaluation makes this setting challenging because models must learn from simulated scenes and generalize to real-world traffic. In this work, we present StateBridge, a traffic video-language framework designed to operate under this synthetic-only training protocol. Our approach uses phase-aware clip construction, pedestrian and vehicle track summaries, and structured prompts to ground vision-language models in stable traffic evidence rather than appearance alone. We fine-tune public vision-language models on synthetic captioning and visual question answering examples, then combine their outputs through fact-locked caption fusion and evidence-diverse VQA consensus to improve consistency across actor roles, behavioral phases, and question types. StateBridge achieves an official S2 score of 55.4679 on AI City Challenge 2026 Track 2 using only synthetic training data. These results demonstrate the effectiveness of structured evidence transfer for traffic safety captioning and visual question answering.