aiperf

VLLM's Decode Context Parallelism (DCP) enables higher concurrency and throughput for long-context agentic workloads by splitting the KV cache across GPUs, according to a new blog post from the vLLM t…

The open-source LLM benchmarking tool llmperf, originally from the Ray/Anyscale team, has been archived, prompting developers to seek alternatives. Developer Wheynelau built llmperf-rs, a single Rust …