sglang

Wafer served GLM5.2 on AMD MI355X GPUs at 2626 tokens per second per node with over 2x lower cost than NVIDIA Blackwell, achieving 213 tok/s single stream. The company used MXFP4 quantization via AMD …

A developer launched an unlimited LLM provider for $6/month using 4x RTX 3090s after a failed launch on AMD MI300x hardware. Despite initial bugs and hardware issues, the service gained traction with …

A developer has released LLMhop, a stateless HTTP router that directs OpenAI-compatible API requests to the appropriate LLM inference backend based on the model name in the request body. The single-bi…