{"slug": "vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models", "title": "VSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Models", "summary": "Researchers have introduced VSAS-Bench, a new benchmark for evaluating visual streaming assistant models that process continuous video frames in real time. The framework provides over 18,000 temporally dense annotations and standardized protocols to measure accuracy, latency, proactiveness, and consistency. Large-scale evaluations revealed that conventional vision-language models adapted to streaming settings without additional training outperformed specialized streaming models, with Qwen3-VL-4B surpassing the leading streaming VLM by 3% under the asynchronous protocol.", "body_md": "[content type paper](/research/)published May 2026\n\nVSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Models\n\nAuthorsPavan Kumar Anasosalu Vasu*, Cem Koc*, Fartash Faghri*, Chun-Liang Li, Bo Feng, Zhengfeng Lai, Meng Cao, Oncel Tuzel, Hadi Pouransari*\n\nVSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Models\n\nAuthorsPavan Kumar Anasosalu Vasu*, Cem Koc*, Fartash Faghri*, Chun-Liang Li, Bo Feng, Zhengfeng Lai, Meng Cao, Oncel Tuzel, Hadi Pouransari*\n\nStreaming vision-language models (VLMs) continuously generate responses given an instruction prompt and an online stream of input frames. This is a core mechanism for real-time visual assistants. Existing VLM frameworks predominantly assess models in offline settings. In contrast, the performance of a streaming VLM depends on additional metrics beyond pure video understanding, including proactiveness, which reflects the timeliness of the model’s responses, and consistency, which captures the robustness of its responses over time. To address this limitation, we propose VSAS-Bench, a new framework and benchmark for Visual Streaming Assistants. In contrast to prior benchmarks that primarily employ single-turn question answering on video inputs, VSAS-Bench features temporally dense annotations with over 18,000 annotations across diverse input domains and task types. We introduce standardized synchronous and asynchronous evaluation protocols, along with metrics that isolate and measure distinct capabilities of streaming VLMs. Using this framework, we conduct large-scale evaluations of recent video and streaming VLMs, analyzing the accuracy–latency trade-off under key design factors such as memory buffer length, memory access policy, and input resolution, yielding several practical insights. Finally, we show empirically that conventional VLMs can be adapted to streaming settings without additional training, and demonstrate that these adapted models outperform recent streaming VLMs. For example, Qwen3-VL-4B surpasses Dispider, the best streaming VLM on our benchmark by 3% under asynchronous protocol.\n\nFastVLM: Efficient Vision Encoding for Vision Language Models\n\nJuly 23, 2025[research area Computer Vision](/highlights?domain=Computer%20Vision)\n\nVision Language Models (VLMs) enable visual understanding alongside textual inputs. They are typically built by passing visual tokens from a pretrained vision encoder to a pretrained Large Language Model (LLM) through a projection layer. By leveraging the rich visual representations of the vision encoder and the world knowledge and reasoning capabilities of the LLM, VLMs can be useful for a wide range of applications, including accessibility…\n\nHow Far Are We from Intelligent Visual Deductive Reasoning?\n\nMay 1, 2024[research area Computer Vision](/research/?domain=Computer%20Vision), [research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[How Far Are We from AGI?](/research/?event=How%20Far%20Are%20We%20from%20AGI%3F)\n\nThis paper was accepted at the How Far Are We from AGI? workshop at ICLR 2024.\n\nVision-Language Models (VLMs) such as GPT-4V have recently demonstrated incredible strides on diverse vision language tasks. We dig into vision-based deductive reasoning, a more sophisticated but less explored realm, and find previously unexposed blindspots in the current SOTA VLMs. Specifically, we leverage Raven’s Progressive Matrices (RPMs), to assess VLMs’…", "url": "https://wpnews.pro/news/vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models", "canonical_source": "https://machinelearning.apple.com/research/vsas-bench-streaming-assistant", "published_at": "2026-05-22 00:00:00+00:00", "updated_at": "2026-05-29 08:04:16.568082+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "large-language-models", "ai-research"], "entities": ["Pavan Kumar Anasosalu Vasu", "Cem Koc", "Fartash Faghri", "Chun-Liang Li", "Bo Feng", "Zhengfeng Lai", "Meng Cao", "Oncel Tuzel"], "alternates": {"html": "https://wpnews.pro/news/vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models", "markdown": "https://wpnews.pro/news/vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models.md", "text": "https://wpnews.pro/news/vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models.txt", "jsonld": "https://wpnews.pro/news/vsas-bench-real-time-evaluation-of-visual-streaming-assistant-models.jsonld"}}