FastMix: Fast Data Mixture Optimization via Gradient Descent
Researchers introduced FASTMIX, a framework that automates data mixture optimization for training large models by reformulating mixture selection as a bilevel optimization problem and using gradient descent to jointly op…