Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Researchers Tianyu Fu and colleagues posted Cache-to-Cache (C2C) to arXiv on 3 Oct 2025, revised 2 Mar 2026, a method that lets large language models communicate directly through fused KV-cache representations instead of text. C2C uses a neural network to project and fuse the source model's KV-cache into the target model's, with a learnable gating mechanism selecting which target layers receive the transfer. The authors report C2C achieves 6.4-14.2% higher average accuracy than individual models, beats text-based communication by roughly 3.1-5.4%, and delivers an average 2.5x latency speedup, with code at github.com/thu-nics/C2C.

Computer Science Computation and Language Submitted on 3 Oct 2025 v1 https://arxiv.org/abs/2510.03215v1 , last revised 2 Mar 2026 this version, v2 Title:Cache-to-Cache: Direct Semantic Communication Between Large Language Models View PDF https://arxiv.org/pdf/2510.03215 HTML experimental https://arxiv.org/html/2510.03215v2 Abstract:Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache C2C , a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at this https URL https://github.com/thu-nics/C2C . Submission history From: Tianyu Fu view email https://arxiv.org/show-email/5bc6748f/2510.03215 Fri, 3 Oct 2025 17:52:32 UTC 484 KB \ v1\ https://arxiv.org/abs/2510.03215v1 v2 Mon, 2 Mar 2026 19:24:02 UTC 546 KB References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .