Metadata-Free Meta-Reweighted Direct Preference Optimization under Noisy Preference Labels
A new study proposes Metadata-Free Meta-Reweighted Direct Preference Optimization (DPO) to improve alignment of large language models under noisy preference labels. The method uses a bilevel optimization framework and a …