{"slug": "beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline", "title": "Beta testing Giraffe: test your LLM serving setup against a saved baseline", "summary": "A developer is building Giraffe, an open-source tool that tests existing OpenAI-compatible LLM endpoints and compares results against a saved baseline, and is seeking beta testers. Giraffe is aimed at catching regressions after changes such as a batching setting adjustment, an inference server upgrade, or a model switch, and ships with a CLI and a local browser UI. The tool is available at github.com/0dust/giraffe, with feedback collected through a Discord server.", "body_md": "You change a batching setting, upgrade your inference server, or switch a model. The endpoint still answers a prompt, but the question is: does it still handle your workload properly?\n\nI’m building **Giraffe**, an open-source tool that tests existing OpenAI-compatible LLM endpoints and compares results with a saved baseline.\n\nIt checks things like:\n\nThere’s a CLI and a local browser UI.\n\n**I’m looking for beta testers. If you run local or self-hosted LLMs, try Giraffe and let me know what works, what breaks, and what’s confusing.**\n\nGet started: [github.com/0dust/giraffe](https://github.com/0dust/giraffe)\n\nFeedback and questions: [Join the Discord](https://discord.gg/GBefPbWebr)", "url": "https://wpnews.pro/news/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline", "canonical_source": "https://discuss.huggingface.co/t/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline/191032#post_1", "published_at": "2026-10-26 11:49:14+00:00", "updated_at": "2026-10-26 12:14:49.050368+00:00", "lang": "en", "topics": ["ai-tools", "mlops", "large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["Giraffe", "OpenAI", "github.com/0dust/giraffe", "Discord"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline", "markdown": "https://wpnews.pro/news/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline.md", "text": "https://wpnews.pro/news/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline.txt", "jsonld": "https://wpnews.pro/news/beta-testing-giraffe-test-your-llm-serving-setup-against-a-saved-baseline.jsonld"}}