Abstract
This paper presents Patent-CR, the first dataset created for the patent claim revision task in English. It includes both initial patent applications rejected by patent examiners and the final granted versions. Unlike normal text revision tasks that predominantly focus on enhancing sentence quality, such as grammar correction and coherence improvement, patent claim revision aims at ensuring the claims meet stringent legal criteria. These criteria are beyond novelty and inventiveness, including clarity of scope, technical accuracy, language precision, and legal robustness. We assess various large language models (LLMs) through professional human evaluation, including general LLMs with different sizes and architectures, text revision models, and domain-specific models. Our results indicate that LLMs often bring ineffective edits that deviate from the target revisions. In addition, domain-specific models and the method of fine-tuning show promising results. Notably, GPT-4 outperforms other tested LLMs, but further revisions are still necessary to reach the examination standard. Furthermore, we demonstrate the inconsistency between automated and human evaluation results, suggesting that GPT-4-based automated evaluation has the highest correlation with human judgment. This dataset, along with our preliminary empirical research, offers invaluable insights for further exploration in patent claim revision.- Anthology ID:
- 2025.naacl-long.116
- Volume:
- April
- Year:
- 2025
- Address:
- Albuquerque, New Mexico
- Editors:
[Luis Chiruzzo](/people/luis-chiruzzo/),[Alan Ritter](/people/alan-ritter/unverified/),[Lu Wang](/people/lu-wang/unverified/)- Venue:
[NAACL](/venues/naacl/)- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2300–2314
- Language:
- URL:
[https://aclanthology.org/2025.naacl-long.116/](https://aclanthology.org/2025.naacl-long.116/)- DOI:
[10.18653/v1/2025.naacl-long.116](https://doi.org/10.18653/v1/2025.naacl-long.116)- Cite (ACL):
- Lekang Jiang, Pascal A. Scherz, and Stefan Goetz. 2025. Patent-CR: A Dataset for Patent Claim Revision. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2300–2314, Albuquerque, New Mexico. Association for Computational Linguistics. - Cite (Informal):
[Patent-CR: A Dataset for Patent Claim Revision](https://aclanthology.org/2025.naacl-long.116/)(Jiang et al., NAACL 2025)- PDF:
[https://aclanthology.org/2025.naacl-long.116.pdf](https://aclanthology.org/2025.naacl-long.116.pdf)