20:02
2026-08-14
dev.to
ai-infrastructure
Building Sluice: QoS-Aware Capacity Governance for Self-Hosted LLM Inference
A developer has released Sluice, an open-source QoS-aware capacity governance proxy for self-hosted LLM inference. Sluice sits in front of vLLM deployments to enforce tenant-based service tiers, prese…