{"slug": "llm-as-a-judge-what-it-is-and-how-to-build-one-yourself", "title": "LLM-as-a-Judge: What It Is and How to Build One Yourself", "summary": "LLM-as-a-Judge is a technique that uses a large language model to evaluate the output of another model, offering a scalable and explainable proxy for human preference. The approach was formalized in a paper by Zheng and colleagues from UC Berkeley, which introduced MT Bench and Chatbot Arena, and showed that GPT-4 agreed with human judgments about as often as two humans agreed with each other. Traditional metrics like BLEU and ROUGE show low correlation with human judgments, while LLM-as-a-Judge can reach correlations as high as 0.85 in legal question answering, compared to 0.32 for ROUGE-L and 0.52 for BLEU.", "body_md": "If you have trained or fine-tuned a language model recently, you already know the real bottleneck is rarely building the model. It is evaluating it. How do you know if your chatbot’s answer is actually good? How do you know if your RAG system is hallucinating? Human evaluation is accurate but slow and expensive. Classic metrics like BLEU or ROUGE are fast but they barely correlate with what a human actually thinks of the output. This is the gap that LLM as Judge was built to fill.\n\nLLM as Judge is a technique where you use a large language model to evaluate the output of another model, or sometimes the same model, instead of relying only on humans or on rule based metrics. You give the judge model the original task, the response to be evaluated, and a set of instructions describing what “good” looks like. The judge then returns a score, a ranking between two responses, or a written verdict.\n\nThink of it like hiring an experienced editor to review a stack of essays. The editor is not the one grading with a rigid answer key, and they are not the student who wrote the essay either. They read the work, apply a rubric, and give you a judgment along with reasoning. LLM as Judge tries to replicate that editor role, except the editor is itself a language model, which means you can run it thousands of times a day for a fraction of the cost of a human reviewer.\n\nThe idea was formalized in a paper by Zheng and colleagues from UC Berkeley and collaborators, who introduced two benchmarks built around this concept. They established LLM as judge, where a strong model such as GPT 4 grades open ended chat responses, as a scalable and explainable proxy for human preference, showing that GPT 4 agreed with human judgments about as often as two humans agreed with each other. That paper also gave the field two lasting artifacts, MT Bench, a set of eighty multi turn questions, and Chatbot Arena, a crowdsourced platform where humans vote on pairwise battles between models to produce an Elo style leaderboard.\n\nSince then, LLM as Judge has become one of the default tools in the evaluation toolbox, sitting alongside human review and traditional metrics rather than replacing them entirely.\n\nBefore diving deeper, it helps to understand why this technique exists at all.\n\nTraditional metrics like BLEU and ROUGE compare generated text against a fixed reference answer using word overlap. They work reasonably well for tasks with a single correct answer, like translation, but they fall apart for open ended generation. A summarization technical deep dive noted that conventional reference based metrics such as BLEU and ROUGE have relatively low correlation with human judgments, especially for tasks that require creativity and diversity. In one legal question answering study, ROUGE L reached a correlation with human scores of only 0.32 and BLEU reached 0.52, while a model based evaluation method reached 0.85. That is a massive gap.\n\nHuman evaluation, on the other hand, is the gold standard for quality but it does not scale. You cannot ask a person to read ten thousand chatbot transcripts every week, and even if you could, human raters are inconsistent with each other and expensive to coordinate.\n\nLLM as Judge sits in the middle. It is far cheaper and faster than humans, and it is far more nuanced than word overlap metrics, because it can actually read and reason about the response the way a person would.\n\nThere are three common evaluation formats you will run into.\n\n**Pointwise or single answer grading.** The judge sees one response at a time along with a rubric, and outputs a score, often on a scale like one to five or zero to one. This is useful when you want an absolute quality measurement rather than a comparison, for example, “rate this customer support reply for helpfulness on a scale of one to four.”\n\n**Pairwise comparison.** The judge sees two responses to the same prompt, usually labeled Response A and Response B, and picks a winner or declares a tie. This is what powers leaderboards like Chatbot Arena and is generally considered easier for the judge to get right than absolute scoring, because comparing two things side by side is a simpler cognitive task than producing a calibrated number out of thin air.\n\n**Reference guided grading.** The judge is also given a gold answer or a rubric with specific criteria, and it checks the candidate response against that reference rather than judging in a vacuum. This tends to produce more consistent results because the judge has something concrete to anchor to.\n\nA major refinement on the basic idea came from a paper called G Eval. It presented a framework that uses large language models with chain of thought and a form filling paradigm to assess the quality of natural language generation outputs, tested on summarization and dialogue generation. Instead of just asking the model to “rate this from one to five,” G Eval first has the model generate its own step by step evaluation criteria, then uses those self generated steps to score the response field by field, almost like filling out a structured form.\n\nG Eval using GPT 4 as the backbone reached a Spearman correlation of 0.514 with human judgments on a summarization benchmark, outperforming all previous automatic evaluation methods by a large margin. One clever detail is how the final score is computed. Rather than taking the single most likely score token, G Eval uses a probability weighted summation across the possible score tokens, which produces a smoother, more fine grained score instead of the model just always outputting a round number like 3 or 4.\n\nThe G Eval authors were also honest about the technique’s biggest weakness, noting the potential concern that LLM based evaluators may be biased toward text generated by LLMs in general, a theme we will come back to.\n\nThis is the part most tutorials skip, and it is the part that matters most if you actually plan to trust these scores in production.\n\n**Position bias.** When comparing two responses side by side, judges tend to favor whichever one is placed in a particular slot, regardless of quality. The original MT Bench paper documented this directly, and the effect has been replicated widely. One 2026 guide summarized it plainly, GPT 4 shows roughly 40 percent inconsistency due to position, meaning it may flip its decision purely because the order of the two answers was swapped.\n\n**Verbosity bias.** Judges tend to reward longer answers even when the extra length adds nothing of value. The same guide estimated verbosity bias causes around a 15 percent inflation in scores for longer responses.\n\n**Self preference or self enhancement bias.** A model used as a judge tends to rate outputs that resemble its own writing style more favorably, which becomes a real problem if you are using, say, GPT 4 both to generate candidate answers and to judge them. Estimates for the size of this effect vary, with one source putting self enhancement bias at around a 5 to 7 percent boost, though newer research complicates the picture. A 2026 study found self preference bias is not strongly correlated with how capable the judge model is, meaning a stronger judge is not automatically a fairer one.\n\nThere are others worth knowing. Authority bias, where the judge is swayed by confident sounding but false claims. Domain gaps, where agreement with human experts drops by ten to fifteen percent in specialized fields. And judge drift, where the exact same prompt produces different scores after the underlying API model gets silently updated.\n\nThe good news is that most of these biases have known, practical countermeasures.\n\n**Swap and average for position bias.** Run every pairwise comparison twice, once with Response A first and once with Response B first, then only trust the verdict if the judge agrees with itself both times. This is often called swap consistency, and industry writeups describe it as running each comparison with the response order reversed and averaging the results, referred to as position consistency checking. A related metric, swap consistency rate, measures how often the judge gives the same relative verdict when the order of two responses is reversed, with a perfectly unbiased judge scoring 100 percent.\n\n**Explicit rubrics for verbosity bias.** Instead of a vague “which answer is better” prompt, use tighter scales and instructions that explicitly reward conciseness, which several practitioners recommend as a direct fix.\n\n**Different model families for self preference bias.** Never let a model judge its own outputs if you can avoid it. A common recommendation is to judge candidate outputs using a model from a different family than the one that generated them.\n\n**Panels of judges for high stakes decisions.** For the most important calls, such as deciding whether to ship a new model version, use multiple judges from different providers and combine their verdicts. One 2026 field guide suggests that a defensible ensemble setup uses three judges from three different model families, aggregated by majority or weighted vote, at roughly three times the cost of a single judge, but with family specific biases largely canceling out. This mirrors older academic work on using a panel of diverse models as a jury rather than relying on a single judge.\n\n**Version pinning.** Treat your judge configuration, model id, rubric version, and prompt template, as something you version control just like code, and recalibrate against human ratings on a regular schedule rather than assuming last month’s numbers still hold.\n\nYou do not need to call a paid API to get an LLM as Judge pipeline running. There is a healthy ecosystem of open models built specifically for evaluation.\n\n**Prometheus and Prometheus 2** are probably the best known. Prometheus 2 is an open source evaluator language model built to closely mirror human and proprietary model judgments, supporting both direct scoring and pairwise ranking with fully customizable criteria. It is built on top of Mistral Instruct and trained on roughly 100,000 feedback examples from a Feedback Collection dataset plus 200,000 from a Preference Collection dataset. On benchmark comparisons, it reaches 72 to 85 percent agreement with human judgments across pairwise ranking benchmarks including MT Bench Human Judgment, and helpfully, the smaller 7B version requires only 16 GB of VRAM, making it usable on a single consumer GPU.\n\nBeyond Prometheus, there are fine tuned judge models like JudgeLM, and general purpose instruct models like Llama and Qwen that work well as judges when paired with a good G Eval style prompt, even without special fine tuning.\n\nFor RAG specific evaluation, the popular open source library **RAGAS** ships a faithfulness metric that checks whether every claim in a generated answer is actually supported by the retrieved context, and Vectara’s more recent **FaithJudge** framework extends this idea using a pool of diverse, human annotated hallucination examples to substantially improve automated hallucination evaluation compared to earlier zero shot methods.\n\nLet’s put all of this together and solve an actual problem. Retrieval augmented generation, or RAG, is supposed to ground a model’s answers in retrieved documents so it does not make things up. In practice it still does. The model can misread a retrieved chunk, blend two chunks incorrectly, or just add a plausible sounding detail that was never in the context at all.\n\nWe are going to build a faithfulness judge, similar in spirit to what RAGAS calls its faithfulness metric, where the score reflects the fraction of claims in the answer that are actually supported by the retrieved context. The approach we will follow, breaking the answer into individual claims and checking each one, has consistently been found to correlate well with human judgment, since the underlying G Eval technique was shown to correlate strongly with human judgment across several benchmark datasets. Our judge model will be Qwen2.5 7B Instruct, a strong open source instruction tuned model that runs comfortably on a single consumer GPU with 16 to 24 GB of memory, or on a smaller GPU using 4 bit quantization.\n\n```\npip install transformers accelerate torch bitsandbytes\n```\n\nIf you have a GPU with less than 16 GB of memory, install bitsandbytes as shown above so we can load the model in 4 bit precision. If you are on CPU only, swap the model for a smaller one such as Qwen/Qwen2.5-1.5B-Instruct, the same code will work, it will just run slower.\n\n``` python\nimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigMODEL_ID = \"Qwen/Qwen2.5-7B-Instruct\"quant_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type=\"nf4\",    bnb_4bit_compute_dtype=torch.bfloat16,)tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(    MODEL_ID,    quantization_config=quant_config,    device_map=\"auto\",)def judge_generate(prompt: str, max_new_tokens: int = 700) -> str:    messages = [        {\"role\": \"system\", \"content\": \"You are a strict, careful evaluator. You never invent facts and you follow the requested output format exactly.\"},        {\"role\": \"user\", \"content\": prompt},    ]    input_ids = tokenizer.apply_chat_template(        messages, add_generation_prompt=True, return_tensors=\"pt\"    ).to(model.device)    output = model.generate(        input_ids,        max_new_tokens=max_new_tokens,        do_sample=False,       # temperature 0 equivalent, for reproducibility        temperature=None,        top_p=None,    )    response = output[0][input_ids.shape[-1]:]    return tokenizer.decode(response, skip_special_tokens=True)\n```\n\nNotice do_sample=False. Judges should be deterministic wherever possible. If your judge gives a different score every time you run it on the exact same input, your evaluation pipeline becomes noise you cannot trust or reproduce.\n\nRather than asking the model to blindly output one overall score, we follow the G Eval pattern, we ask it to reason first, in this case by decomposing the answer into individual factual claims and checking each one against the context, then produce a structured verdict.\n\n```\nFAITHFULNESS_PROMPT = \"\"\"You are evaluating whether an AI generated answer is faithful to a retrieved context. Faithful means every factual claim in the answer is directly supported by the context, with no invented, contradicted, or unsupported information.CONTEXT:{context}QUESTION:{question}ANSWER TO EVALUATE:{answer}Follow these steps:1. Break the answer down into a list of individual, atomic factual claims.2. For each claim, decide if it is SUPPORTED, CONTRADICTED, or UNSUPPORTED by the context.3. Compute a faithfulness score between 0.0 and 1.0 as the fraction of claims that are SUPPORTED.Respond with only a JSON object in exactly this format, no extra commentary:{{  \"claims\": [    {{\"claim\": \"text of claim\", \"verdict\": \"SUPPORTED\" | \"CONTRADICTED\" | \"UNSUPPORTED\", \"reason\": \"short explanation\"}}  ],  \"faithfulness_score\": 0.0}}\"\"\"\n```\n\nLanguage models occasionally wrap JSON in stray text or markdown fences, even when told not to, so we defensively extract the JSON block before parsing.\n\n``` php\nimport jsonimport redef extract_json(text: str) -> dict:    match = re.search(r\"\\{.*\\}\", text, re.DOTALL)    if not match:        raise ValueError(f\"No JSON object found in judge output: {text}\")    return json.loads(match.group(0))def evaluate_faithfulness(context: str, question: str, answer: str) -> dict:    prompt = FAITHFULNESS_PROMPT.format(context=context, question=question, answer=answer)    raw_output = judge_generate(prompt)    try:        result = extract_json(raw_output)    except (ValueError, json.JSONDecodeError) as error:        return {\"error\": str(error), \"raw_output\": raw_output}    return result\ncontext = (    \"The Eiffel Tower was completed in 1889 as the entrance arch for the \"    \"1889 World's Fair in Paris. It was designed by the engineer Gustave \"    \"Eiffel's company. It stands 330 meters tall including antennas and \"    \"was the tallest man made structure in the world until 1930, when the \"    \"Chrysler Building in New York surpassed it.\")question = \"When was the Eiffel Tower built and how tall is it?\"grounded_answer = (    \"The Eiffel Tower was completed in 1889 for the World's Fair in Paris \"    \"and stands 330 meters tall including its antennas.\")hallucinated_answer = (    \"The Eiffel Tower was completed in 1887 for the World's Fair in Paris, \"    \"stands 350 meters tall, and was designed by Alexandre Gustave Eiffel \"    \"himself who personally supervised every rivet placed during construction.\")for label, ans in [(\"grounded\", grounded_answer), (\"hallucinated\", hallucinated_answer)]:    result = evaluate_faithfulness(context, question, ans)    print(f\"\\n--- {label} answer ---\")    print(json.dumps(result, indent=2))\n```\n\nRunning this, you would expect the grounded answer to score close to 1.0, since both claims trace directly back to the context, while the hallucinated answer should score much lower. The judge should flag the wrong year, the wrong height, and the invented detail about personally supervising every rivet, none of which appear anywhere in the context.\n\nIn production you rarely evaluate one answer at a time, you run this over a batch of logged RAG responses and flag the low scoring ones for human review.\n\n``` python\nfrom dataclasses import dataclass@dataclassclass RAGSample:    sample_id: str    context: str    question: str    answer: strdef run_faithfulness_audit(samples: list[RAGSample], threshold: float = 0.8) -> list[dict]:    flagged = []    for sample in samples:        result = evaluate_faithfulness(sample.context, sample.question, sample.answer)        score = result.get(\"faithfulness_score\")        if score is None or score < threshold:            flagged.append({                \"sample_id\": sample.sample_id,                \"score\": score,                \"claims\": result.get(\"claims\"),            })    return flagged\n```\n\nEvery flagged sample below the threshold gets routed to a human reviewer, or logged into a dashboard so your team can spot patterns, for example, discovering that hallucinations spike specifically on multi hop questions where the retriever pulled only a partial context.\n\nEven in this single answer setup, a few of the earlier lessons still apply directly.\n\n**Do not use the same model family for generation and judging.** If your RAG system’s answer generator is also Qwen2.5, consider swapping the judge for a model from a different family, such as Llama 3.1, so that self preference bias does not quietly inflate your faithfulness numbers.\n\n**Sample a subset for expensive judging, not everything.** Running a 7B model over every single production response is costly at scale. A common pattern is to judge a random 5 to 10 percent of traffic continuously for trend monitoring, and to judge 100 percent only for a specific incident investigation or a pre launch regression test.\n\n**Calibrate against humans periodically.** Take a sample of maybe fifty flagged and fifty unflagged answers each month, have a human quickly confirm or reject the judge’s verdict, and track your agreement rate over time. If agreement drops, your judge prompt or model may need updating.\n\nPulling everything from this post into a single checklist you can actually follow:\n\nLLM as Judge did not appear out of nowhere, it grew directly out of a real gap between metrics that are fast but shallow and human review that is deep but slow. Used carefully, with position swapping, cross family judges, explicit rubrics, and regular human calibration, it becomes one of the most practical tools available for evaluating generative AI systems at scale. Used carelessly, as a single unchecked model grading everything with a vague prompt, it becomes a confident sounding number that quietly misleads your whole team.\n\nThe code above gives you a working starting point you can extend today, whether that means swapping in your own retrieval pipeline, adding a pairwise comparison mode to A/B test two RAG configurations, or scaling the batch audit into a nightly job that watches your production traffic for hallucinations before your users ever notice them.\n\n**Is LLM as Judge the same thing as a reward model?**\n\nNot quite. A reward model is typically a smaller model trained specifically to output a scalar score, often used inside reinforcement learning from human feedback. An LLM as Judge is usually a general purpose instruction tuned model being prompted to evaluate, without any extra training. Prometheus 2 sits somewhere in between, since it is a general model that has been fine tuned specifically for evaluation.\n\n**Can I use LLM as Judge to replace human evaluation entirely?**\n\nYou could, but you probably should not for anything high stakes. Treat it as a scalable first pass. Run the judge on everything, then route the low scoring or borderline cases to a human reviewer. Periodically sample a batch of judge verdicts and have a human confirm or reject them, so you catch drift before it becomes a real problem.\n\n**Which bias should I worry about most?**\n\nPosition bias tends to be the largest and most consistently documented, especially in pairwise comparisons. If you only fix one thing, add the swap and average step, since it directly targets the biggest effect size and is cheap to implement.\n\n**Do I need a GPU to run an open source judge model?**\n\nFor something like Qwen2.5 7B Instruct or Prometheus 2 7B, yes, ideally a GPU with at least 16 GB of memory, or a smaller GPU using 4 bit quantization. If you do not have a GPU available, drop down to a smaller model such as Qwen2.5 1.5B Instruct, which runs on CPU, just expect slower generation and somewhat noisier scores.\n\n**How do I know if my judge is actually any good?**\n\nCalibrate it against humans. Take a sample of outputs, get human ratings on that same sample, and compare. Track this agreement rate over time, not just once at setup, since model updates and prompt drift can quietly degrade it.\n\n**Should the judge model be the same as the model being evaluated?**\n\nAvoid this where possible. Using the same model, or even the same family, to both generate and judge responses risks self preference bias, where the judge favors outputs that resemble its own style regardless of actual quality.\n\n**What is the difference between LLM as Judge and RAGAS?**\n\nRAGAS is a library, not a technique. It uses LLM as Judge internally, along with some rule based checks, to compute specific RAG metrics like faithfulness, context precision, and answer relevance. Think of RAGAS as a packaged, RAG specific application of the broader LLM as Judge idea.\n\n**Is a higher faithfulness score always better?**\n\nMostly, but watch for false confidence. A model that abstains and says “I don’t know based on the provided documents” when the context truly does not answer the question is behaving correctly, even though it produced no supported claims to score. Build a separate check for appropriate abstention rather than only optimizing for a raw faithfulness number.\n\n[LLM-as-a-Judge: What It Is and How to Build One Yourself](https://pub.towardsai.net/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself-db5f1283e58c) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself", "canonical_source": "https://pub.towardsai.net/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself-db5f1283e58c?source=rss----98111c9905da---4", "published_at": "2026-08-05 12:05:58+00:00", "updated_at": "2026-08-05 12:22:18.912744+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools"], "entities": ["LLM-as-a-Judge", "GPT-4", "MT Bench", "Chatbot Arena", "UC Berkeley", "Zheng", "BLEU", "ROUGE"], "alternates": {"html": "https://wpnews.pro/news/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself", "markdown": "https://wpnews.pro/news/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself.md", "text": "https://wpnews.pro/news/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself.txt", "jsonld": "https://wpnews.pro/news/llm-as-a-judge-what-it-is-and-how-to-build-one-yourself.jsonld"}}