Netflix Details Its In-House LLM Serving Platform with Triton and vLLM
Netflix detailed its in-house LLM serving platform built on Triton and vLLM, revealing how it handles real-time and batch inference across CPUs and GPUs while managing version compatibility and constr…