{"slug": "a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage", "title": "A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage", "summary": "Researchers from multiple institutions published a comprehensive survey on Process Reward Models (PRMs) at ACL 2026, covering data generation, model construction, and usage for step-level reasoning evaluation in large language models. The survey highlights PRMs' advantages over outcome reward models in math, code, multimodal reasoning, robotics, and agents, and identifies open challenges for fine-grained reasoning alignment.", "body_md": "[A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage](https://aclanthology.org/2026.acl-long.163.pdf)\n\n[Congmin Zheng](/people/congmin-zheng/unverified/),\n[Jiachen Zhu](/people/jiachen-zhu/),\n[Zhuoying Ou](/people/zhuoying-ou/unverified/),\n[Yuxiang Chen](/people/yuxiang-chen/unverified/),\n[Kangning Zhang](/people/kangning-zhang/),\n[Rong Shan](/people/rong-shan/),\n[Zeyu Zheng](/people/zeyu-zheng/),\n[Mengyue Yang](/people/mengyue-yang/unverified/),\n[Jianghao Lin](/people/jianghao-lin/),\n[Yong Yu](/people/yong-yu/),\n[Weinan Zhang](/people/weinan-zhang-ucl/)\n\n##### Abstract\n\nLarge Language Models (LLMs) have advanced reasoning ability, yet conventional alignment remains dominated by outcome reward models (ORMs) that judge only final answers. Process Reward Models(PRMs) address this gap by evaluating and guiding reasoning at the step or trajectory level. This survey provides a systematic overview of PRMs through the full loop: how to generate process data, build PRMs, and use PRMs for test-time scaling and reinforcement learning. We summarize applications across math, code, text, multimodal reasoning, robotics, and agents, and review emerging benchmarks. Our goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.- Anthology ID:\n- 2026.acl-long.163\n- Volume:\n[Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](/volumes/2026.acl-long/)- Month:\n- July\n- Year:\n- 2026\n- Address:\n- San Diego, California, United States\n- Editors:\n[Maria Liakata](/people/maria-liakata/),[Viviane P. Moreira](/people/viviane-p-moreira/unverified/),[Jiajun Zhang](/people/jiajun-zhang/unverified/),[David Jurgens](/people/david-jurgens/)- Venue:\n[ACL](/venues/acl/)- SIG:\n- Publisher:\n- Association for Computational Linguistics\n- Note:\n- Pages:\n- 3591–3607\n- Language:\n- URL:\n[https://aclanthology.org/2026.acl-long.163/](https://aclanthology.org/2026.acl-long.163/)- DOI:\n- Cite (ACL):\n- Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, and Weinan Zhang. 2026.\n[A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage](https://aclanthology.org/2026.acl-long.163/). In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 3591–3607, San Diego, California, United States. Association for Computational Linguistics. - Cite (Informal):\n[A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage](https://aclanthology.org/2026.acl-long.163/)(Zheng et al., ACL 2026)- PDF:\n[https://aclanthology.org/2026.acl-long.163.pdf](https://aclanthology.org/2026.acl-long.163.pdf)", "url": "https://wpnews.pro/news/a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage", "canonical_source": "https://aclanthology.org/2026.acl-long.163/", "published_at": "2026-06-22 00:00:00+00:00", "updated_at": "2026-06-26 08:17:00.385557+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "natural-language-processing", "ai-safety"], "entities": ["Congmin Zheng", "Jiachen Zhu", "Weinan Zhang", "Association for Computational Linguistics", "ACL"], "alternates": {"html": "https://wpnews.pro/news/a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage", "markdown": "https://wpnews.pro/news/a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage.md", "text": "https://wpnews.pro/news/a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage.txt", "jsonld": "https://wpnews.pro/news/a-comprehensive-survey-of-process-reward-models-data-generation-model-and-usage.jsonld"}}