X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding
Researchers proposed X-CoSD, a cross-vocabulary collaborative speculative decoding framework for distributed LLM inference that works with heterogeneous small language model and server LLM vocabularies, according to an a…