{"slug": "sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference", "title": "SGLang: the AI Tool You’ve Never Heard of Just killed Hugging Face’s Own Inference Engine", "summary": "Hugging Face retired its own Text Generation Inference (TGI) server in December 2025, directing users to SGLang, a tool that began as a UC Berkeley research paper in 2023 and by 2026 was running on over 400,000 GPUs, generating trillions of tokens daily. The move effectively replaced Hugging Face's flagship inference engine with an open-source project from academia.", "body_md": "Member-only story\n\n# SGLang: the AI Tool You’ve Never Heard of Just killed Hugging Face’s Own Inference Engine\n\nSGLang started as a UC Berkeley research paper in 2023. By 2026 it was running on 400,000+ GPUs, generating trillions of tokens a day, and had become credible enough that Hugging Face offered it as an alternative once it retired its own inference server. Here’s the actual story.\n\nIn December 2025, Hugging Face put its own inference server into maintenance mode. Text Generation Inference, TGI, had been the company’s flagship serving engine since before most people had heard the phrase “LLM inference,” the tool it built to run models on its own Inference Endpoints. It stopped taking new features. Bug fixes only. And Hugging Face’s own documentation started pointing new users somewhere else: to vLLM, or to a tool most people outside machine learning infrastructure have never heard of, called SGLang.\n\nThat’s a strange thing for a company to do to its own product. It’s an even stranger thing when the replacement wasn’t built by a competitor with a marketing budget, it came out of a university research lab, published as an academic paper, with a name few people can pronounce with confidence on the first try (it’s “S-G-Lang,” short for Structured Generation Language, not one word). By the time Hugging Face quietly retired TGI, that research project was running on more than…", "url": "https://wpnews.pro/news/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference", "canonical_source": "https://pub.towardsai.net/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-face-s-own-inference-engine-d6ac204b0146?source=rss----98111c9905da---4", "published_at": "2026-07-22 03:44:34+00:00", "updated_at": "2026-07-22 03:59:48.386807+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-startups", "ai-tools"], "entities": ["Hugging Face", "SGLang", "Text Generation Inference", "UC Berkeley", "vLLM"], "alternates": {"html": "https://wpnews.pro/news/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference", "markdown": "https://wpnews.pro/news/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference.md", "text": "https://wpnews.pro/news/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference.txt", "jsonld": "https://wpnews.pro/news/sglang-the-ai-tool-youve-never-heard-of-just-killed-hugging-faces-own-inference.jsonld"}}