Shepherd Model Gateway

mentions 2 type Person feed RSS

// recent coverage 2 mentions

22:36

2026-07-26

github.com

large-language-models

Shepherd Model Gateway – Model-routing gateway for LLM deployments

LightSeek Org released Shepherd Model Gateway (SMG), an engine-agnostic, high-performance model-routing gateway for large-scale LLM deployments that centralizes worker lifecycle management and balance…

18:56

2026-04-30

pytorch.org

large-language-models

SMG: The Case for Disaggregating CPU from GPU in LLM Serving

Shepherd Model Gateway (SMG) has disaggregated all CPU-bound workloads from GPU inference in large language model serving, moving tokenization, detokenization, and parsing into a dedicated Rust gatewa…

// co-occurs with top 8 entities

SGLang 2 vLLM 2 Python 1 Rust 1 GIL 1 GPU 1 CPU 1 LightSeek Org 1

// topics top 6 topics

large language models 2 ai infrastructure 2 ai tools 2 ai products 1 mlops 1 developer tools 1