Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization

Researchers propose CoverPruner, a training-free visual token pruner for vision-language models (VLMs) that frames pruning as Representational Coverage Maximization (RCM), ensuring each removed token is represented by a surviving original token. In tests across multiple VLM architectures and compression rates, CoverPruner achieves the best average accuracy among compared methods, with largest gains under aggressive compression.

arXiv:2609.03158v1 Announce Type: new Abstract: Visual token pruning reduces the inference cost of vision-language models VLMs , but most methods only ask which tokens to keep. This retained-token view can keep redundant high-scoring tokens while leaving discarded evidence without a close representative. We propose CoverPruner, a training-free pruner that asks the complementary demand-side question: after a token is removed, which surviving original token represents it for the target VLM? CoverPruner formulates pruning as Representational Coverage Maximization RCM , covering the full projected visual-token set with query-weighted demand. It instantiates RCM with projector-space coverage and a lightweight first-layer attention probe. Across multiple VLM architectures and compression rates, CoverPruner achieves the best average accuracy among all compared methods, with the largest gains usually appearing under aggressive compression.