iADD: Improving Alignment and Diversity in Diffusion Policy Optimization

Researchers propose iADD, a method for reinforcement-learning post-training of diffusion models that aims to improve alignment with reward functions without sacrificing output diversity, addressing a limitation of Denoising Diffusion Policy Optimization (DDPO). DDPO optimizes a reverse diffusion process under a reward function, but the paper states current reward-optimization approaches achieve this at the cost of diversity and quality. The work targets the trade-off between reward alignment and sample diversity in diffusion policy optimization.

Reinforcement learning based post training of diffusion models, such as Denoising Diffusion Policy Optimization DDPO , optimizes a reverse diffusion process under a reward function. However, current approaches to reward optimizations do so at the cost of diversity and quality. In this paper, we pro