Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL
Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL Async RL has a dirty secret: every step, the trainer has to ship the whole model to the inference engine. For a 7B in bf16 that is 14 GB. For a frontier 1T model...
Scan with your phone
Scan with your phone