VLLM's disaggregated serving using AMD's MORI-IO connector achieved 2.5x higher goodput on the same 8-GPU node compared to standard collocated serving, according to an April 2026 blog post. The benchm…
A vLLM RFC from the llm-d team warns that disaggregated inference deployments, where prefill and decode run on separate schedulers, can trigger recomputation-based preemption inside the decode instanc…
Disaggregated LLM serving, which runs prefill and decode on separate GPU pools and transfers KV caches over the network, should always be used in practice under sufficient load, according to a technic…
Prefill/decode disaggregation, which runs LLM prefill and decode phases on separate hardware pools, cut p99 inter-token latency by 66% (from 88ms to 30ms) in a Go simulation, at the cost of higher tim…
A new arXiv paper (arXiv:2607.28633v1) proposes a topology-aware transfer orchestrator for disaggregated GPU inference, claiming existing systems like DistServe, Splitwise, and Mooncake ignore that ba…