TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models

Researchers proposed TagPR, a framework that adds semantic tags to a large language model's reasoning process for step-by-step guidance on personalization tasks, according to an arXiv paper (2509.23140v2). TagPR generates a structured, tagged dataset for supervised fine-tuning and then applies multi-stage reinforcement learning with a composite reward signal combining tag-based process supervision and a Personalization Reward Model with User Embeddings. Experiments on LaMP, LongLaMP, PGraphRAG and a self-constructed dataset produced state-of-the-art results, with an average improvement of 32.65% over the base model across all LaMP benchmark tasks.

arXiv:2509.23140v2 Announce Type: replace Abstract: Recent advancements have endowed Large Language Models with impressive general reasoning capabilities. However, these reasoning models often perform worse than non-reasoning models on personalization tasks. While some methods use outcome-based RL to improve personalization reasoning, they fail to supervise the reasoning process. As a result, models may reach correct answers through flawed reasoning chains, limiting further improvement. To address this, we propose TagPR, a novel framework that adds semantic tags to the reasoning process for step-by-step guidance. TagPR first automatically generates a structured, tagged dataset for Supervised Fine-Tuning. It then employs a multi-stage RL process guided by a composite reward signal, which integrates tag-based process supervision with a novel Personalization Reward Model with User Embeddings to achieve fine-grained alignment with user-specific logic. Extensive experiments on public LaMP, LongLaMP, PGraphRAG, and a self-constructed dataset demonstrate that our approach achieves state-of-the-art results, delivering an average improvement of 32.65% over the base model across all LaMP benchmark tasks. Our work demonstrates that tag-guided process supervision is an effective approach for personalization reasoning.