Spool: Low-Latency Streaming Inference for Hybrid Vision-Language Models A paper submitted to arXiv on 17 Nov 2026 presents Spool, a training-free framework that caches per-frame state transformations in hybrid vision-language models so expired frames can be removed from recurrent layers without replaying the whole sliding window. Across five open-source hybrid VLMs and four video reasoning benchmarks, Spool posted the lowest p95 time-to-first-token in all 20 model-benchmark tests, cutting it by a geometric mean of 6.6x versus the faster of two recent vLLM baselines while improving accuracy by 2.3 percentage points on average. The work targets latency-sensitive applications such as visual accessibility agents and robotics that must answer questions about recent video over a bounded window of past frames. Computer Science Computer Vision and Pattern Recognition Submitted on 17 Nov 2026 Title:Spool: Low-Latency Streaming Inference for Hybrid Vision-Language Models View PDF http://arxiv.org/pdf/2611.23228v1 HTML experimental https://arxiv.org/html/2611.23228v1 Abstract:Applications such as visual accessibility agents and robotics require answering questions about recent video content promptly, over a bounded window of past frames e.g., the last two minutes . Hybrid vision-language models VLMs , which interleave attention with linear recurrent layers such as Gated DeltaNet GDN , are attractive for this setting, but maintaining a sliding window is expensive. The reason is that recurrent layers combine all past frames into a shared state, so expiring old frames normally requires re-running the entire window through the model. We observe that the recurrent update is affine in the state with input-dependent coefficients, so the effect of each frame can be captured as a transformation that composes associatively. We present Spool, a training-free framework that caches per-frame state transformations and composes them efficiently as the window advances, removing expired frames from each recurrent layer's state without replaying frames through the backbone. Removal is exact within each layer; as with key-value eviction in attention, expired frames can retain residual influence through the cached inputs computed by earlier layers. We design a dedicated data structure and GPU kernels to achieve low caching and composition overhead. Across five open-source hybrid VLMs and four video reasoning benchmarks, Spool achieves the lowest p95 time-to-first-token in all 20 model-benchmark tests, reducing it by a geometric mean of $6.6\times$ over the faster of two recent vLLM baselines, while improving accuracy by 2.3 percentage points on average. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .