A practical guide argues that developers should stop relying on generic leaderboards like LMSYS to choose large language models, and instead create a custom 'golden dataset' of difficult edge cases to…
Nvidia shipped NeMo Switchyard, an Apache 2.0 Rust proxy that routes LLM traffic between models mid-workflow, alongside the Nemotron 3.5 Lightning model on August 11. The escalation router cut costs 7…
Manifest, an open-source development platform, shipped an LLM router in March 2024, saw 7,000 cloud users use it for four months, and deprecated it in June, with full shutdown on September 1. The comp…
Spectro Cloud released a TCO calculator for its PaletteAI Inference Launchpad, a turnkey appliance that runs open models on local GPUs with intelligent routing, claiming it can cut inference costs by …
KTransformers, a Tsinghua-born framework now with 19k GitHub stars, has restructured from a standalone inference system into CPU kernels and placement strategies embedded within SGLang and LLaMA-Facto…
A new book, 'Build a Reasoning Model (From Scratch)' (ISBN-13 9781633434677), teaches readers how to add reasoning capabilities to a pre-trained large language model through hands-on code examples, co…
LMSYS FastChat is the most comprehensive open-source platform for building, serving, and evaluating LLM-powered chatbots in 2026, offering tools for training instruction-tuned models like Vicuna and A…
LMSYS Org launched a new leaderboard ranking AI models by a composite of human preference and factuality, with a default 25% weight on factuality. The system labeled over 2 million claims from 170,000…
Arena, the UC Berkeley spinout behind the AI model evaluation platform LMArena, reached a $100 million annual revenue run rate eight months after launching its first commercial product, AI Evaluations…
An open-source benchmark for prompt-injection detectors has been released, measuring both attack catch rate and false positives on real traffic in a threshold-agnostic way. The benchmark evaluates ten…
Jian Chen, Yesheng Liang, and Zhijian Liu integrated Z Lab's DFlash block diffusion speculative decoding method into SGLang, collaborating with Modal and LMSYS. The team reports up to 4.31x higher thr…