Speculative Decoding Grows Up, and Prefix Caches Learn to Collapse
A weekly inference-and-serving digest highlights new arXiv research and engineering posts on LLM serving, including ForeSpark, a joint draft-and-verification budget scheduler for parallel speculative decoding that report…