PatentScore: Multi-Dimensional Evaluation of LLM-Generated Patent Claims

Researchers at the 2025 Conference on Empirical Methods in Natural Language Processing introduced PatentScore, a multi-dimensional evaluation framework for LLM-generated patent claims. PatentScore integrates hierarchical decomposition, legal validation patterns, and scoring across structural, semantic, and legal dimensions, achieving a correlation of 0.819 with expert annotations and outperforming standard NLG metrics. The framework addresses the need for reliable evaluation of high-stakes documents like patent claims.

Abstract High-stakes texts such as patent claims, medical records, and technical reports are structurally complex and demand a high degree of reliability and precision. While large language models LLMs have recently been applied to automate their generation in high-stakes domains, reliably evaluating such outputs remains a major challenge. Conventional natural language generation NLG metrics are effective for generic documents but fail to capture the structural and legal characteristics essential to evaluating complex high-stakes documents. To address this gap, we propose PatentScore, a multi-dimensional evaluation framework specifically designed for one of the most intricate and rigorous domains, patent claims. PatentScore integrates hierarchical decomposition of claim elements, validation patterns grounded in legal and technical standards, and scoring across structural, semantic, and legal dimensions. In experiments on our dataset which consists of 400 Claim1, PatentScore achieved the highest correlation with expert annotations r = 0.819 , significantly outperforming widely used NLG metrics. This work establishes a new standard for evaluating LLM-generated patent claims, providing a solid foundation for research on patent generation and validation.- Anthology ID: - 2025.emnlp-main.1564 - Volume: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing /volumes/2025.emnlp-main/ - Month: - November - Year: - 2025 - Address: - Suzhou, China - Editors: Christos Christodoulopoulos /people/christos-christodoulopoulos/ , Tanmoy Chakraborty /people/tanmoy-chakraborty/ , Carolyn Rose /people/carolyn-rose/ , Violet Peng /people/violet-peng/unverified/ - Venue: EMNLP /venues/emnlp/ - SIG: - Publisher: - Association for Computational Linguistics - Note: - Pages: - 30727–30746 - Language: - URL: https://aclanthology.org/2025.emnlp-main.1564/ https://aclanthology.org/2025.emnlp-main.1564/ - DOI: 10.18653/v1/2025.emnlp-main.1564 https://doi.org/10.18653/v1/2025.emnlp-main.1564 - Cite ACL : - Yongmin Yoo, Qiongkai Xu, and Longbing Cao. 2025. PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims https://aclanthology.org/2025.emnlp-main.1564/ . In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages 30727–30746, Suzhou, China. Association for Computational Linguistics. - Cite Informal : PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims https://aclanthology.org/2025.emnlp-main.1564/ Yoo et al., EMNLP 2025 - PDF: https://aclanthology.org/2025.emnlp-main.1564.pdf https://aclanthology.org/2025.emnlp-main.1564.pdf