Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL
Hugging Face researchers released a new method called delta weight sync that reduces the per-step weight transfer in asynchronous reinforcement learning from 1.2 GB to as little as 20 MB for a 0.6B parameter model. The t…