DecisionRank: On-Policy Distillation for Decision-Model Rerankers

A method called DecisionRank applies on-policy distillation to transfer a generative LLM's ranking knowledge into a decision-model reranker, addressing the difficulty of teaching decision models that predict choices or scores directly rather than decoding autoregressively. The approach targets the mismatch in which the teacher expresses preferences as a ranked list of document ID sequences, per the article. The work matters for builders seeking LLM-quality ranking without autoregressive decoding latency.

Decision models target structured tasks by directly predicting choices or scores, avoiding autoregressive decoding latency. However, transferring a generative LLM's ranking knowledge to a decision model remains challenging: the teacher expresses preferences through a ranked list of document ID seque