{"slug": "tokenrace-compare-local-llm-servers-on-your-own-hardware", "title": "TokenRace: compare local LLM servers on your own hardware", "summary": "A developer released TokenRace, an open-source app that benchmarks local LLM inference servers on a user's own hardware by sending the same streaming request to each server and timing everything client-side. TokenRace reports time to first token (TTFT), tokens per second, total time and RAM live, then summarizes median, min and max results with a chart and CSV export, and supports 15 servers out of the box including Ollama, LM Studio, llama.cpp, vLLM, SGLang and MLX, plus any OpenAI-compatible server. It runs locally with the command uvx tokenrace, and the developer is soliciting results from NVIDIA, AMD and Linux users.", "body_md": "**Note: The Show and Tell category is for sharing and discussing projects, showcasing your Spaces, Models, Datasets and more. We value open-source and technical details over promotional content, so focus on sharing the intricate aspects of your work**.Which local inference server is fastest for your model? On your machine, it depends, and benchmark posts rarely transfer.\n\nI kept failing to reproduce other people’s tok/s numbers, so I built TokenRace: a small open-source app that sends the same streaming request to whichever local servers you run and times everything on the client, with the same code for all of them.\n\nYou pick a server and any model you have, start the test and get TTFT, tok/s, total time and RAM live, then a summary with median, min and max, a chart and a CSV.\n\nIt works with Ollama, LM Studio, llama.cpp, vLLM, SGLang, MLX servers and others (15 out of the box, any OpenAI-compatible server can be added), and runs locally with one command: uvx tokenrace\n\nRepo: [GitHub - ernext/tokenrace: Compare local LLM inference servers (Ollama, LM Studio, MLX, llama.cpp, vLLM…) on your own hardware, with the same client-side timing for all of them. · GitHub](https://github.com/ernext/tokenrace)\n\nI also wrote up the method here: [Why your local LLM tok/s numbers don't match mine - DEV Community](https://dev.to/ernext/why-your-local-llm-toks-numbers-dont-match-mine-5dn6)\n\nIf you measure a model on NVIDIA, AMD or Linux, I’d love to see the numbers.", "url": "https://wpnews.pro/news/tokenrace-compare-local-llm-servers-on-your-own-hardware", "canonical_source": "https://discuss.huggingface.co/t/tokenrace-compare-local-llm-servers-on-your-own-hardware/191775#post_1", "published_at": "2026-11-12 10:57:09+00:00", "updated_at": "2026-11-12 11:22:14.636325+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "developer-tools", "mlops"], "entities": ["TokenRace", "Ollama", "LM Studio", "llama.cpp", "vLLM", "SGLang", "MLX", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tokenrace-compare-local-llm-servers-on-your-own-hardware", "markdown": "https://wpnews.pro/news/tokenrace-compare-local-llm-servers-on-your-own-hardware.md", "text": "https://wpnews.pro/news/tokenrace-compare-local-llm-servers-on-your-own-hardware.txt", "jsonld": "https://wpnews.pro/news/tokenrace-compare-local-llm-servers-on-your-own-hardware.jsonld"}}