DeepSpeed

DeepSpeed

Extreme Speed and Scale for DL Training and Inference

Microsoft Research blog