# Feedback on LLM-as-a-Judge design for open-source library

> Source: <https://discuss.huggingface.co/t/feedback-on-llm-as-a-judge-design-for-open-source-library/166224#post_2>
> Published: 2026-07-24 20:44:43+00:00

Nice work shipping an open LLM-as-a-Judge API.

One design pressure we keep hitting on science-critique evals: if the *task itself* is about whether a model overclaims, putting another LLM judge in the scoring loop can re-import the same failure mode.

REFUTE is a judge-free board for that split on recent science paper summaries (critique skill vs calibration). Useful as a foil when deciding where LLM judges help vs where objective axes are safer.

Dataset: [BGPT-OFFICIAL/refute · Datasets at Hugging Face](https://huggingface.co/datasets/BGPT-OFFICIAL/refute)

Leaderboard: [REFUTE Leaderboard - a Hugging Face Space by BGPT-OFFICIAL](https://huggingface.co/spaces/BGPT-OFFICIAL/refute-leaderboard)