C2C Links Models Through Their KV-Caches

A system called Cache-to-Cache (C2C), published on GitHub by thu-nics, lets large language models communicate directly through their KV-Caches instead of generating text, achieving 8.5-10.5% higher accuracy than individual models and 3.0-5.0% better performance than text-based communication with a 2.0x latency speedup. C2C uses a lightweight Neural Fuser to project and fuse Model A's attention memory into Model B's KV-Cache via high-dimensional spatial rotation and alignment, with millisecond-level adaptive balancing that lets models sense which layers gain most from external caches. The approach removes intermediate tokens in favor of direct "thought projection" between models of differing layer counts and structures.

Cache-to-Cache C2C https://github.com/thu-nics/C2C enables large language models to communicate directly through their KV-Caches, bypassing text generation. By projecting and fusing KV-Caches between models, C2C achieves 8.5-10.5% higher accuracy than individual models and 3.0-5.0% better performance than text-based communication, with a 2.0x speedup in latency. This is an interesting experiment. The C2C approach removes the intermediate tokens and goes straight for “thought projection.” After Model A computes, it doesn’t generate any text at all. The system uses a lightweight neural network the Neural Fuser to splice and fuse Model A’s attention memory KV-Cache directly into Model B’s internal KV-Cache, through high-dimensional spatial rotation and alignment. The challenge is that different models have varying numbers of layers and structures. In the paper, the model dynamically senses on its own which key layers absorb the highest gains from external caches, and which layers should stay independent in thought, with millisecond-level adaptive balancing.