In news that may be overshadowed by certain other events at the company, Nvidia announced its membership in an initiative focused around open-source AI inferencing.
The chip giant is an inaugural member of SGLang Consortium, a project from nonprofit LMSYS Org that also includes AMD, Intel, Google, Meta, and Cloudflare.
According to LMSYS, the venture centers around its full-stack SGLang inference engine designed for frontier AI models. Members will explore emerging inference AI trends with SGLang’s maintainers and "turn shared priorities into joint engineering work, focused on frontier model support, reasoning and multimodal workloads, distributed inference, and hardware optimization."
“The teams running SGLang at the largest scale see what inference needs next before anyone else does,” Ying Sheng, co-author of the original SGLang paper and co-founder and CEO of RadixArk, explained. “With the challenges in building new data centers, software approaches to improve AI efficiency will be incredibly important. These lessons will benefit every SGLang user and advance the AI ecosystem.”
"Better software helps developers get more performance from Nvidia compute. Open models enable them to build new applications and businesses. And lower inference costs make AI more accessible to organizations everywhere,” Ian Buck, VP of hyperscale and high-performance computing at Nvidia, added.
Prior to his departure from Nvidia, now ex-CEO Jensen Huang described inference as the "growth engine of AI." The comments came as the chip vendor unveiled its LPX racks for AI inference accelerators in September, with Huang claiming its Vera Rubin platform offered "workload-optimized AI factory configurations designed for the era of agentic AI, advancing the performance frontier with LPX for ultrafast token generation.”