Knowledge Graphs Quality Assessment in the Era of LLMs: A Conceptual Framework

A paper posted to arXiv (2610.31684v1) proposes a general-purpose, domain- and task-customizable quality assessment framework for knowledge graphs, extending prior state-of-the-art frameworks to a catalog of 24 quality dimensions plus associated quality metrics. The authors reviewed literature on quality assessment of data, information, linked data, knowledge graphs, and the intersection of knowledge graphs and large language models, motivated by the growing use of KGs to ground LLM answers that must be concise, complete, accurate, faithful, fresh and traceable. The framework is intended to help determine when a knowledge graph should be used and whether it can be trusted to ground an LLM.

arXiv:2610.31684v1 Announce Type: new Abstract: Knowledge Graphs KGs have been popularized during the last few decades and are increasingly constructed and consumed by Large Language Models LLMs . Various applications therefore rely on KGs to provide concise, complete, accurate, faithful, fresh, and traceable answers. However, what is the quality of those KGs? In which cases should a KG be used, and can it be trusted to ground an LLM? How might it be evaluated? We reviewed the literature on quality assessment QA of data, information, linked data, KGs, and the intersection of KGs and LLMs. We extended the current state-of-the-art frameworks to a catalog of 24 quality dimensions QDs and various quality metrics QMs . Furthermore, we propose a general-purpose, customizable to a domain or task, and practical QA framework for assessing the quality of KGs.