Self-Play Search Distillation for Large Language Model Reasoning

Researchers introduced Self-Play Search Distillation, a method that improves reasoning in large language models by generating high-quality training data through self-play search, addressing data scarcity caused by low-quality synthetic data and the cost of human labeling. The work targets the difficult decisions and competing alternatives that expose model reasoning gaps.

Improving reasoning abilities in Large Language Models LLMs requires high-quality data that exposes difficult decisions, competing alternatives, and their consequences. Data scarcity is driven by the low quality of synthetic data and the cost of human labeling. We introduce Self-Play Search Distil