BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Researchers propose BlockServe, a continuous batching framework for diffusion large language models that uses block-grained scheduling to eliminate compute bubbles and tail latency caused by convergen…