Microsoft Research Blog

Research Blog

  1. DeepSpeed: Extreme-scale model training for everyone 

    September 10, 2020 | DeepSpeed Team, Rangan Majumder, and Junhua Wang

    In February, we announced DeepSpeed, an open-source deep learning training optimization library, and ZeRO (Zero Redundancy Optimizer), a novel memory optimization technology in the library, which vastly advances large model training by improving scale, speed, cost, and usability. DeepSpeed has enabled researchers to create Turing…

  2. Adversarial robustness as a prior for better transfer learning 

    August 11, 2020 | Hadi Salman

    Editor’s note: This post and its research are the collaborative efforts of our team, which includes Andrew Ilyas (opens in new tab) (PhD Student, MIT), Logan Engstrom (opens in new tab) (PhD Student, MIT), Aleksander Mądry (opens in new tab) (Professor at MIT), Ashish Kapoor…