Beta testing Giraffe: test your LLM serving setup against a saved baseline

A developer is building Giraffe, an open-source tool that tests existing OpenAI-compatible LLM endpoints and compares results against a saved baseline, and is seeking beta testers. Giraffe is aimed at catching regressions after changes such as a batching setting adjustment, an inference server upgrade, or a model switch, and ships with a CLI and a local browser UI. The tool is available at github.com/0dust/giraffe, with feedback collected through a Discord server.

You change a batching setting, upgrade your inference server, or switch a model. The endpoint still answers a prompt, but the question is: does it still handle your workload properly? I’m building Giraffe , an open-source tool that tests existing OpenAI-compatible LLM endpoints and compares results with a saved baseline. It checks things like: There’s a CLI and a local browser UI. I’m looking for beta testers. If you run local or self-hosted LLMs, try Giraffe and let me know what works, what breaks, and what’s confusing. Get started: github.com/0dust/giraffe https://github.com/0dust/giraffe Feedback and questions: Join the Discord https://discord.gg/GBefPbWebr