LLM-as-a-Judge: Building LLM-Based Evaluation Pipelines for AI Applications

LLM-as-a-Judge uses a second large language model to evaluate AI-generated responses against predefined criteria, enabling scalable automated evaluation for AI applications. The approach, demonstrated with the Wayfinder flight search agent, assesses dimensions such as helpfulness, relevance, completeness, groundedness, and instruction following, and is supported by evaluation platforms like LangSmith. The complete implementation is available in the Wayfinder repository on GitHub under the v0.4.0 release.

AI Engineering Fundamentals AI Evaluation · Part 5 ← Part 4 https://pub.towardsai.net/human-evaluation-building-reusable-evaluation-datasets-for-ai-applications-54f6d93fd2db?sharedUserId=divakar.ungatla 📦 Complete Code The complete implementation used in this article is available in the companionWayfinderrepository on GitHub. To follow along with the exact code and examples from this article, check out thev0.4.0 release. View the complete code on GitHub In the previous article, we explored Human Evaluation and how human reviewers can assess AI application quality using structured evaluation criteria. Human evaluation solves an important problem - it allows us to measure qualities that are difficult to express as deterministic rules, such as helpfulness, relevance, completeness and groundedness. However human evaluation has a practical limitation — it does not scale. As AI application grows, the number of interactions increases rapidly. Reviewing every response manually becomes expensive, time-consuming, and difficult to maintain. This raises an important question: Can we automate the evaluation process while maintaining the quality of human judgment ? This is where LLM-as-a-Judge comes in. Instead of relying entirely on human reviewers, we can use another large language model to evaluate AI-generated responses against predefined evaluation criteria. At a high level, LLM-as-a-Judge introduces another LLM into the evaluation pipeline. The application LLM generates the response, while the judge LLM evaluates that response against predefined criteria. In this article, we will explore how LLM-as-a-Judge works, build an evaluation pipeline for an AI application, and see how evaluation platforms like LangSmith help run and track these evaluations at scale. The key idea behind LLM-as-a-Judge is simple — instead of asking a language model only to generate responses, we introduce another language model whose responsibility is to evaluate those responses. In an AI application, there are now two distinct roles: The application LLM tries to solve the user’s request. The judge LLM does not solve the request again; it reviews whether the response produced by the application meets the expected quality standards. Let’s understand this using Wayfinder https://github.com/DivakarUngatla/wayfinder/tree/v0.4.0 , our flight search AI application. A user asks: Find the cheapest flight from Bangalore to Tokyo tomorrow. The Wayfinder https://github.com/DivakarUngatla/wayfinder/tree/v0.4.0 agent processes the request, searches available flight data, and generates a response: The cheapest flight is Air India AI302 at ₹410. Now the judge LLM evaluates this response. The judge receives: User Query:Find the cheapest flight from Bangalore to Tokyo tomorrow.Application Response:The cheapest flight is Air India AI302 at ₹410.Retrieved Context:Available flight information returned by the search tool.Evaluation Criteria:- Helpfulness- Relevance- Completeness- Groundedness- Instruction Following The judge is not checking whether the response matches a hardcoded expected answer. Instead, it evaluates whether the response is correct, useful, and supported by the available context. The output is an evaluation result: Helpfulness: 5/5Groundedness: 5/5Instruction Following: 5/5Explanation:The response correctly identifies the cheapest flightand the information is supported by the retrieved flight data. An LLM judge is only as effective as the criteria it uses to evaluate responses. Just like human reviewers need a clear rubric to evaluate AI outputs consistently, an LLM judge also needs predefined evaluation criteria that describe what a good response looks like. For Wayfinder, we use the same evaluation dimensions introduced in human evaluation: These criteria form the evaluation rubric that defines how the judge LLM should assess response quality. For example, instead of asking: Is this response good? the judge receives specific criteria: Evaluate this response for:- Helpfulness- Groundedness- Instruction Following This makes evaluations more consistent and repeatable across different responses. In the next section, we will see how these criteria are translated into an LLM-as-a-Judge implementation and how the evaluator produces structured scores and explanations. Now that we have defined our evaluation criteria, the next step is to build an evaluator that can apply this rubric to AI-generated responses. An LLM-as-a-Judge pipeline starts with an evaluation dataset. The complete code is here — Wayfinder https://github.com/DivakarUngatla/wayfinder/tree/v0.4.0 . An evaluation dataset contains representative user scenarios that we want our AI application to handle. Each evaluation sample contains: For Wayfinder, our dataset contains scenarios such as: { "inputs": { "query": "Find the cheapest flight from Bangalore to Tokyo tomorrow." }, "outputs": { "expected behavior": "Recommend the retrieved flight with the lowest price without inventing information." }, "metadata": { "category": "cheapest" }} The important thing is that the dataset does not contain a fixed expected answer. Instead, it describes the expected behavior of the application. During evaluation, each dataset sample is executed against the AI application. The quality of an LLM-as-a-Judge system depends heavily on how the judge prompt is designed. A good judge prompt should clearly define: For Wayfinder, the judge prompt follows this pattern: You are an expert evaluator assessing the quality of an AI assistant response.You will be given:- The user's original query.- The expected behavior: a description of what a good assistant response should do for this query.- The assistant's actual response.- The flight data retrieved by the search tool before the response was generated.Your task is to evaluate the assistant's response against each of the following criteria.Evaluation criteria and rubric:{criteria block}Instructions:- "Expected behavior" describes what the application was expected to do for this sample. It is not a model answer. Use it to understand the intent of the query.- Evaluate the response against each criterion independently.- When evaluating Groundedness, cross-reference the response against the retrieved flight data.- Do not invent facts that are not present in the supplied context.- Score every criterion from 1 to 5 using the rubric above.- Provide a concise explanation for every criterion score.- Provide an overall score from 1 to 5 that reflects your holistic judgment. Do not calculate the overall score as an average of the criteria scores.- Provide a concise overall explanation."""User Query:{query}Application Response:{response}Retrieved Context:{context}Provide a score from 1-5 for each criterionand explain your reasoning. The judge is not asked to generate a better response. Its only responsibility is to evaluate the response it receives. Free-form evaluation responses from the judge are difficult to analyze automatically. Instead, the evaluator asks the LLM Judge to return structured output. In our implementation, we enforce a predefined response schema so every evaluation produces a consistent format. The schema contains: Example: { "criteria scores": { "criterion": "Helpfulness", "score": 5, "explanation": "The response directly answers the user's request." }, { "criterion": "Groundedness", "score": 5, "explanation": "The response is supported by the provided context." }, { "criterion": "Instruction Following", "score": 5, "explanation": "The response satisfies the user's requirements." } , "overall score": 5} With the evaluation dataset, criteria, judge prompt, and structured output format in place, we can now implement the LLM-as-a-Judge evaluator. The evaluator acts as a reusable component that takes an AI application’s output and evaluates it using the predefined rubric. At a high level, the judge performs four steps: The evaluator receives: User QueryApplication ResponseExpected BehaviorRetrieved ContextEvaluation Criteria and produces: Criterion Scores+Explanations+Overall Assessment The application code exposes a simple interface: result = judge.evaluate query=query, expected behavior=expected behavior, response=response, retrieved flights=retrieved context Internally, the evaluator builds the judge prompt: messages = { "role": "system", "content": judge instructions }, { "role": "user", "content": evaluation input } The system prompt defines the judge’s role, evaluation criteria and scoring guidelines. The user message provides the specific evaluation sample, including the query, response and supporting context. The LLM response is then parsed into the structured evaluation model: JudgeResult criteria scores= CriterionScore criterion="Groundedness", score=5, explanation="The response is supported by the retrieved context." , overall score=5, overall explanation="The response satisfies the user's request." Keeping the judge as a separate component makes it reusable. The same evaluator can be applied to different versions of an AI application using the same evaluation dataset and criteria. This allows teams to measure whether changes actually improve the system: Instead of manually reviewing responses after every change, teams can run repeatable evaluations and compare results across versions. The complete implementation used throughout this article is available in the companion GitHub repository .To follow along with exactly the same code shown in this article, check out thev0.4.0release.This ensures the code, commands, and screenshots remain consistent over time. To run the examples locally, clone the repository and checkout the release containing the LLM-as-a-Judge implementation. git clone https://github.com/DivakarUngatla/wayfinder.gitcd wayfindergit checkout v0.4.0-llm-judge Install the project dependencies: uv sync Before running the evaluation, configure the required environment variables. Create a .env file: cp .env.example .env Add your OpenAI API key: OPENAI API KEY=<your-api-key For LangSmith experiments later in this article, also configure: LANGSMITH API KEY=<your-api-key LANGSMITH TRACING=true The LLM-as-a-Judge evaluator uses the evaluation dataset created earlier. The complete dataset containing evaluation scenarios, expected behavior and metadata is available in the companion repository: examples/llm judge evaluation/llm judge evaluation dataset.jsonl https://github.com/DivakarUngatla/wayfinder/blob/v0.4.0/examples/llm as a judge/wayfinder llm judge evaluation v1.jsonl The evaluator loads each sample from this dataset, runs the application, and passes the generated response to the LLM judge for evaluation. uv run examples/llm judge evaluation/local evaluation.py For each evaluation sample, the judge evaluates the generated response against the predefined criteria and returns structured scores along with explanations. For example, consider the arrival-time constraint scenario: User Query:Which flight from Bangalore to Tokyo arrives before 8 PM? The judge evaluates the response across criteria that is provided to it. After evaluating all samples in the dataset, the pipeline generates an overall evaluation summary. While this run completed successfully, evaluation becomes especially valuable when it detects regressions after application changes. After establishing the baseline evaluation, we can introduce a controlled change to simulate a regression. In a real AI system, changes to prompts, models or retrieval logic can unintentionally affect response quality. For this example, let us temporarily modify the response-generation prompt. Change the below line in the If the user asks for cheapest, select the flight with the lowest price among flights satisfying all constraints. to If the user asks for cheapest, provide the two cheapest available options. The evaluation dataset remains unchanged. Only the application behavior changes. Running the evaluation again produces something as shown below The LLM judge identifies that the response is technically correct but does not fully satisfy the user’s intent. The response identifies the cheapest flight correctly, but it provides an additional flight even though the user requested only the cheapest option. The evaluator detects this through criteria such as: This demonstrates an important advantage of LLM-as-a-Judge: it can evaluate whether a response satisfies the user’s intent, not just whether the output contains valid data. Once we start running evaluations repeatedly across different application versions, we need a way to track experiments and compare results. Evaluation platforms like LangSmith provide the required infrastructure. The same evaluation dataset used for local evaluation can be uploaded to LangSmith. The evaluation dataset contains: The dataset used in this article is available in the companion repository. wayfinder llm judge evaluation v1.jsonl Create a new dataset and upload the JSONL file. LangSmith automatically detects the fields from the dataset. For Wayfinder: The LangSmith evaluation runner uses the same LLM-as-a-Judge evaluator that we built earlier. Run: uv run examples/llm as a judge/langsmith evaluation.py The evaluator executes each dataset example against Wayfinder and sends the generated response to the judge. Once the evaluation completes, click on the langsmith link displayed to view the results For each example, we can inspect: The important point is that the evaluator itself does not change. The same LLM-as-a-Judge component can run locally or through LangSmith. In this article, we explored LLM-as-a-Judge , an evaluation approach where an LLM acts as an evaluator to assess AI-generated responses against predefined criteria. We built an end-to-end LLM-as-a-Judge evaluation pipeline for an AI-powered flight search application, covering evaluation criteria, judge prompts, structured evaluation results, and running evaluations locally and through LangSmith. Unlike traditional rule-based checks, LLM-as-a-Judge can evaluate whether AI responses satisfy user intent while considering the context available to the application. It provides both scores and explanations, helping teams understand and improve AI application quality. LLM-as-a-Judge enables powerful offline evaluation using curated scenarios. However, offline evaluation only covers predefined cases. Production AI systems also need evaluation approaches that learn from real user interactions and live application behavior. In the next article, we’ll explore Online Evaluation , where we’ll learn how to evaluate AI applications using real-world usage data and feedback signals. Follow along as we build a complete AI evaluation toolkit — from deterministic rule-based checks to human evaluation, LLM-as-a-Judge, evaluation datasets, and production-scale evaluation workflows. LLM-as-a-Judge: Building LLM-Based Evaluation Pipelines for AI Applications https://pub.towardsai.net/llm-as-a-judge-building-automated-evaluation-pipelines-for-ai-applications-8680a412a1bd was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.