{"slug": "language-model-hallucination-evaluation-with-grapheval", "title": "Language Model Hallucination Evaluation with GraphEval", "summary": "Amazon researchers have developed GraphEval, a framework that uses knowledge graphs to detect hallucinations in large language model (LLM) outputs. The method constructs semantic triples from a model's response and evaluates each triple against a ground-truth context using a natural language inference (NLI) model, flagging any triple that cannot be entailed as a hallucination. GraphEval emphasizes explainability by identifying exactly where factual inconsistencies occur.", "body_md": "# Language Model Hallucination Evaluation with GraphEval\n\nTurning the key principles and methodological stages of GraphEval into a simulated practical scenario to better understand its usefulness and key implications in understanding and combating LLM hallucinations.\n\n## # Introduction\n\n**Hallucinations** are one of the best-known problems that **large language models (LLMs)** may experience when generating responses. They occur when a model produces a response that is factually incorrect, nonsensical, or simply made up, typically due to the model's lack of internal knowledge on the matter.\n\nWhile many solutions have arisen in recent years to tackle the problem of model hallucinations, methodological evaluation frameworks for internally diagnosing them have been comparatively less studied. One ** recent study** by Amazon researchers proposes using knowledge graphs as a means to analyze and detect hallucinations occurring in LLMs. The framework presented in the study is named\n\n**GraphEval**.\n\nIn this article, we will take a gentle, practical approach to illustrate the conceptual building blocks of GraphEval through a simulation-based, lightweight code example that you can easily try on your machine.\n\n## # GraphEval in a Nutshell\n\nGraphEval leverages knowledge graphs to identify and signal hallucinations in LLM-generated outputs. Unlike classical performance metrics that provide single scores to evaluate aspects like accuracy, certainty, and so on, GraphEval applies a two-stage evaluation process that emphasizes explainability, namely, providing insights into where exactly the hallucination took place.\n\nTo do this, GraphEval considers two stages:\n\n- Constructing a knowledge graph from the generated model response. The graph consists of semantic triples of the form\n*(Subject, Relationship, Object)*, where subjects and objects correspond to nodes, and relationships correspond to the edges connecting those nodes. - Evaluating each triple in the constructed knowledge graph against a source context (a ground-truth body of knowledge) through a natural language inference (NLI) model. Any triple that cannot be entailed by the context according to the NLI engine — because it is contradictory or neutral — is flagged as a hallucination.\n\n## # Illustrating GraphEval Through a Code Example\n\nBefore starting the code that simulates the application of the GraphEval framework, let's make sure we have the necessary libraries installed:\n\n```\n!pip install -q transformers networkx matplotlib torch\n```\n\nThe purpose of the code example we are about to walk through is to demystify how the GraphEval methodology works, so we will replace the stages that would demand a heavy computational burden in a real-world setting with simulated, lightweight alternatives.\n\nAccordingly, we will simulate a ground-truth knowledge base (context) assumed to contain factual information. In a production setting, this ground-truth knowledge would stem, for instance, from retrieving relevant documents from the [vector database of a retrieval-augmented generation (RAG) system](https://machinelearningmastery.com/understanding-rag-part-vii-vector-databases-indexing-strategies/). For simplicity, here we directly create a ground-truth context and store it in `source_context`\n\n.\n\n```\n# The ground-truth context provided to the LLM\nsource_context = (\n    \"GraphEval is a hallucination evaluation framework based on representing information \"\n    \"in Knowledge Graph (KG) structures. It acts as a pre-processing step and utilizes \"\n    \"out-of-the-box NLI models to detect factual inconsistencies.\"\n)\n```\n\nNow, let's suppose the following is the original LLM response to a user prompt like \"explain succinctly what GraphEval is\". To initiate the first stage of the evaluation process, we would ask an auxiliary LLM to build the knowledge graph from that response. Both the response and the follow-up prompt used to obtain the knowledge graph are shown below:\n\n```\n# The generated response we want to evaluate (contains a hallucination)\nllm_output = (\n    \"GraphEval is an evaluation framework that uses Knowledge Graphs. \"\n    \"It requires a highly expensive, enterprise-level server farm to operate.\"\n)\n\n# Prompt template that would theoretically be passed to a local/free LLM (e.g. Mistral-7B)\nKG_EXTRACTION_PROMPT = f\"\"\"\nYou are an expert information extractor. Extract the core information from the following text as a Knowledge Graph.\nReturn the output strictly as a Python list of tuples in the format: (Subject, Relationship, Object).\n\nText: {llm_output}\n\"\"\"\n```\n\nOnce again, for the sake of simplicity and to bypass the otherwise heavy computational load of running a massive LLM locally, let's suppose the following graph triples are obtained:\n\n```\n# Simulated extraction to bypass the heavy computational load of running a massive LLM locally\nextracted_triples = [\n    (\"GraphEval\", \"is\", \"evaluation framework\"),\n    (\"GraphEval\", \"uses\", \"Knowledge Graphs\"),\n    (\"GraphEval\", \"requires\", \"expensive enterprise server farm\")\n]\n\nprint(\"Extracted Triples:\")\nfor t in extracted_triples:\n    print(t)\n```\n\nOutput:\n\n```\nExtracted Triples:\n('GraphEval', 'is', 'evaluation framework')\n('GraphEval', 'uses', 'Knowledge Graphs')\n('GraphEval', 'requires', 'expensive enterprise server farm')\n```\n\nWe deliberately added a triple that is fundamentally a hallucination (no enterprise server farm needed whatsoever!), so we can demonstrate how the subsequent NLI process applied to the knowledge graph reveals it.\n\nEnough simulated steps for today. Let's get into the real action for the next stage: the NLI process. The next piece of code is fundamental to leveraging the ideas behind GraphEval. It uses a pre-trained NLI model from ** Hugging Face** — the model is publicly available, so no access token is needed to download it — to compare each triple against the ground-truth context. If no entailment is \"predicted\" by the NLI model for a given triple, it is labeled as a hallucination.\n\n``` python\nfrom transformers import pipeline\n\n# Loading the open-source NLI model\nprint(\"Loading DeBERTa NLI model...\")\nnli_evaluator = pipeline(\"text-classification\", model=\"cross-encoder/nli-deberta-v3-small\")\n\ndef evaluate_triple(context, triple):\n    subject, relation, obj = triple\n    hypothesis = f\"{subject} {relation} {obj}\"\n\n    # Checking if the context entails the hypothesis\n    result = nli_evaluator({\"text\": context, \"text_pair\": hypothesis})\n\n    # NLI models normally output: 'entailment', 'neutral', or 'contradiction'\n    label = result['label'].lower()\n\n    # In GraphEval, anything other than 'entailment' is flagged as a hallucination\n    is_hallucinated = label != 'entailment'\n\n    return is_hallucinated, label, hypothesis\n\n# Running the evaluation pipeline\nevaluation_results = []\n\nprint(\"\\n--- GraphEval Results ---\")\nfor t in extracted_triples:\n    is_hallucinated, nli_label, hypothesis = evaluate_triple(source_context, t)\n    evaluation_results.append((is_hallucinated, nli_label))\n\n    status = \"🚨 HALLUCINATION\" if is_hallucinated else \"✅ GROUNDED\"\n    print(f\"{status} | Triple: {t} | NLI Output: {nli_label}\")\n```\n\nOutput:\n\n```\n--- GraphEval Results ---\n✅ GROUNDED | Triple: ('GraphEval', 'is', 'evaluation framework') | NLI Output: entailment\n✅ GROUNDED | Triple: ('GraphEval', 'uses', 'Knowledge Graphs') | NLI Output: entailment\n🚨 HALLUCINATION | Triple: ('GraphEval', 'requires', 'expensive enterprise server farm') | NLI Output: neutral\n```\n\nAs we expected, the last triple in the knowledge graph is detected as a hallucination.\n\nTo finish with a visual touch, we can also display the knowledge graph of the original LLM response alongside the detection results:\n\n``` python\nimport networkx as nx\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches\n\ndef visualize_grapheval(triples, eval_results):\n    G = nx.DiGraph()\n    edge_colors = []\n\n    for (triple, res) in zip(triples, eval_results):\n        sub, rel, obj = triple\n        is_hallucinated = res[0]\n\n        G.add_node(sub)\n        G.add_node(obj)\n        G.add_edge(sub, obj, label=rel)\n\n        # Color-code the edges based on the NLI evaluation\n        edge_colors.append('red' if is_hallucinated else 'green')\n\n    # Set up the plot\n    plt.figure(figsize=(10, 6))\n    pos = nx.spring_layout(G, seed=42)\n\n    # Draw nodes\n    nx.draw_networkx_nodes(G, pos, node_color='lightblue', node_size=2500)\n    nx.draw_networkx_labels(G, pos, font_size=10, font_weight='bold')\n\n    # Draw edges and labels\n    nx.draw_networkx_edges(G, pos, edge_color=edge_colors, width=2.5, arrowsize=20)\n    edge_labels = nx.get_edge_attributes(G, 'label')\n    nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, font_color='black')\n\n    # Add legend\n    green_patch = mpatches.Patch(color='green', label='Grounded (Entailment)')\n    red_patch = mpatches.Patch(color='red', label='Hallucination (Neutral/Contradiction)')\n    plt.legend(handles=[green_patch, red_patch], loc='lower right')\n\n    plt.title(\"GraphEval Hallucination Map\", fontsize=14, fontweight='bold')\n    plt.axis('off')\n    plt.tight_layout()\n    plt.show()\n\n# Render the knowledge graph\nvisualize_grapheval(extracted_triples, evaluation_results)\n```\n\nResulting visualization:\n\n## # Closing Remarks\n\nGraphEval is an evaluation methodology proposed to help detect and localize the root cause of hallucinations in LLM outputs. This article turned its key principles and methodological stages into a simulated practical scenario to better understand its usefulness and its key implications for potential implementation in production systems.\n\nis a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.\n\n[Iván Palomares Carrascosa](https://www.linkedin.com/in/ivanpc/)", "url": "https://wpnews.pro/news/language-model-hallucination-evaluation-with-grapheval", "canonical_source": "https://www.kdnuggets.com/language-model-hallucination-evaluation-with-grapheval", "published_at": "2026-07-24 13:02:40+00:00", "updated_at": "2026-07-24 13:35:08.537400+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools", "natural-language-processing", "ai-safety"], "entities": ["Amazon", "GraphEval"], "alternates": {"html": "https://wpnews.pro/news/language-model-hallucination-evaluation-with-grapheval", "markdown": "https://wpnews.pro/news/language-model-hallucination-evaluation-with-grapheval.md", "text": "https://wpnews.pro/news/language-model-hallucination-evaluation-with-grapheval.txt", "jsonld": "https://wpnews.pro/news/language-model-hallucination-evaluation-with-grapheval.jsonld"}}