Beta testing Giraffe: test your LLM serving setup against a saved baseline

You change a batching setting, upgrade your inference server, or switch a model. The endpoint still answers a prompt, but the question is: does it still handle your workload properly?

I’m building Giraffe, an open-source tool that tests existing OpenAI-compatible LLM endpoints and compares results with a saved baseline.

It checks things like:

There’s a CLI and a local browser UI.

I’m looking for beta testers. If you run local or self-hosted LLMs, try Giraffe and let me know what works, what breaks, and what’s confusing.

Get started: [github.com/0dust/giraffe](https://github.com/0dust/giraffe)

Feedback and questions: [Join the Discord](https://discord.gg/GBefPbWebr)