Show HN: Revalvo – local-first LLM eval bench (multi-model, BYOK)

Revalvo, a new local-first LLM evaluation workbench, launched on Product Hunt today, enabling users to run prompts against multiple models in parallel with per-response latency, tokens, and cost tracking, while keeping API keys in the browser and data in IndexedDB. The tool offers 40 built-in evaluators—25 rule-based and 14 LLM-judge-based—and supports versioning, batch eval, and GitHub sync for prompt recipes. Founder seeks feedback on evaluator coverage and rule-vs-judge labeling.

Hi HN — I built Revalvo: a local-first workbench for prompt engineering and LLM evaluation. The problem I kept hitting: chat playgrounds are fast but leave no receipt, and hosted eval platforms are rigorous but slow and server-side. Revalvo sits in the middle — sub-minute setup, your keys stay in your browser, no Revalvo account. What it does today: • Run the same prompt against multiple models in parallel OpenRouter, Groq, direct APIs, Ollama/LM Studio locally • Per-response latency, tokens, and cost • Version prompts like code saves, diffs, rollback • Batch eval on datasets with 40 built-in evaluators • GitHub sync for prompt recipes YAML push/pull On evaluators: ~25 are deterministic rules exact match, regex, JSON schema, length, PII patterns, etc. — no extra API spend . ~14 need a model LLM judge, rubric, faithfulness-style scorers plus embedding similarity. Each is labeled Rule-based vs LLM judge in the UI — I stack rules first, judges when rules aren’t enough. Privacy / cost: BYOK only. We don’t markup API spend. App data lives in IndexedDB in your browser; keys aren’t sent to our servers CORS-blocked providers go through a same-origin dev proxy when you use the hosted app . We launched on Product Hunt today; early feedback is pushing on exactly the evaluator split above, which matches how I think about the product too. Try it: paste an OpenRouter/Groq key or run fully offline with Ollama. Playground is the fastest path to “does this actually work for me?” I’d love HN feedback on: 1. Evaluator coverage — what’s missing for your workflows? 2. Whether rule vs judge labeling is clear enough before you attach evals to a dataset 3. GitHub sync for prompt YAML — useful or noise? Happy to answer technical questions architecture, storage, provider adapters . Comments URL: https://news.ycombinator.com/item?id=49477347 https://news.ycombinator.com/item?id=49477347 Points: 1 Comments: 0