18:07
2026-07-17
netflixtechblog.medium.com
large-language-models
In-House LLM Serving at Netflix
Netflix's AI Platform team built an in-house LLM serving stack using vLLM as the paved-path inference engine, integrated with NVIDIA Triton Inference Server and a Java control plane. The platform handβ¦