{"slug": "post-hoc-sparse-coding-of-latent-communication-between-vision-language-model", "title": "Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents", "summary": "Researchers found that latent-space communication between vision-language model agents can be compressed 128x with minimal accuracy loss. Fitting a post-hoc sparse autoencoder to frozen Vision Wormhole activations, a uint16-index/float16-value sparse payload with k=4 active coefficients per token reduced transmitted bytes by 128x, while mean accuracy on seven non-AIME reasoning benchmarks changed from 49.85% to 49.77%. The fitted 4096-element dictionary used only 50 features, and task-level active sets had a mean pairwise Jaccard similarity of 0.906, indicating strong post-hoc compressibility.", "body_md": "arXiv:2608.10198v1 Announce Type: new\nAbstract: Latent-space communication allows heterogeneous vision-language model agents to exchange continuous representations without serializing visual and reasoning states into text. Vision Wormhole realizes this approach by translating visual features into a universal latent representation that can be consumed by another model, but every message is transported as a dense tensor of the same size regardless of its content. A fixed-capacity dense tensor therefore need not have a fixed effective information density: some messages may use only a small fraction of the available representational degrees of freedom. This observation suggests that the communication channel may be substantially compressible. We study its redundancy by fitting a post-hoc sparse autoencoder to frozen Vision Wormhole activations and measuring reconstruction, downstream utility, feature reuse, and token-level interventions across nine reasoning benchmarks. Relative to the original float32 transport, a uint16-index/float16-value sparse payload with k=4 active coefficients per token reduces the transmitted bytes by 128x. In a single-run evaluation, the seven-task non-AIME mean accuracy changes from 49.85% to 49.77%. The fitted 4096-element dictionary uses only 50 features, and task-level active sets have a mean pairwise Jaccard similarity of 0.906. These measurements establish strong post-hoc compressibility relative to the original transport, but do not yet isolate the incremental contribution of sparse coding from position selection, reduced precision, low-rank structure, or SAE optimization effects. The results motivate matched-payload comparisons and communication mechanisms whose payload adapts to the information used by each message.", "url": "https://wpnews.pro/news/post-hoc-sparse-coding-of-latent-communication-between-vision-language-model", "canonical_source": "https://arxiv.org/abs/2608.10198", "published_at": "2026-08-12 04:00:00+00:00", "updated_at": "2026-08-12 04:18:28.632641+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "neural-networks"], "entities": ["Vision Wormhole"], "alternates": {"html": "https://wpnews.pro/news/post-hoc-sparse-coding-of-latent-communication-between-vision-language-model", "markdown": "https://wpnews.pro/news/post-hoc-sparse-coding-of-latent-communication-between-vision-language-model.md", "text": "https://wpnews.pro/news/post-hoc-sparse-coding-of-latent-communication-between-vision-language-model.txt", "jsonld": "https://wpnews.pro/news/post-hoc-sparse-coding-of-latent-communication-between-vision-language-model.jsonld"}}