Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

Researchers investigated how specialist models trained solely on question-answer pairs, without explicit reasoning supervision, generate reasoning trajectories for domain expert distillation. The work examines what governs the trajectories these specialists produce when teacher-generated reasoning paths are unavailable during training.

Specialist distillation effectively transfers domain expertise to student models via teacher-generated reasoning trajectories. However, when these specialists are trained solely on question--answer pairs without explicit reasoning supervision, what governs the trajectories they generate? In this wor