Training-Free Inference Acceleration for Video Large Language Models: Methods, Trade-offs, and Evaluation

A new arXiv survey (2610.36327v1) organizes training-free inference acceleration methods for Video Large Language Models by where they intervene in the inference pipeline: input, visual encoding, projector, prefill, and decoding, with post-training quantization (PTQ) treated separately as a cross-stage extension with varying calibration requirements. The survey analyzes how intervention position affects preservation of task-relevant spatiotemporal evidence and computational savings, and why local efficiency gains may not produce end-to-end acceleration, backing the analysis with quantitative results from prior studies and the authors' own experiments under a shared evaluation setting to assess accuracy-latency trade-offs.

arXiv:2610.36327v1 Announce Type: new Abstract: Video Large Language Models VLMs have demonstrated strong capabilities in video understanding and reasoning, but processing long visual sequences incurs substantial computational, latency, and memory overhead. This survey examines training-free inference acceleration, focusing on reducing inference costs while preserving task-relevant spatiotemporal evidence. We organize existing methods by their intervention stages in the inference pipeline, covering input, visual encoding, projector, prefill, and decoding. Post-training quantization PTQ is discussed separately as a cross-stage extension with varying calibration requirements. We analyze how intervention position affects evidence preservation and computational savings, and why local efficiency improvements may not translate into end-to-end acceleration. We complement this analysis with quantitative evidence from prior studies and our own experiments under a shared evaluation setting to assess accuracy-latency trade-offs.