PipeDream: A more effective way to train deep neural networks using pipeline parallelism
October 28, 2019 | Amar Phanishayee
Deep Neural Networks (DNNs) have facilitated tremendous progress across a range of applications, including image classification, translation, language modeling, and video captioning. DNN training is extremely time-consuming, needing efficient multi-accelerator parallelization. In “PipeDream: Generalized Pipeline Parallelism for DNN Training,” published at the 27th ACM Symposium…