DeepSpeed

DeepSpeed

Extreme Speed and Scale for DL Training and Inference

微软研究院博客

background pattern
微软研究院博客

New Research | FLASH: Workflow automatio…

微软研究院博客

AI saw unparalleled growth in 2023, reac…

DeepSpeed4Science Initiative - graphic with 6 icons
微软研究院博客

| Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang, Conglong Li, Martin Cai, 和 Yuxiong He

Editor’s note, Sept. 28, 2023 – The foun…

DeepSpeed ZeRO++ blog hero
微软研究院博客

| DeepSpeed Team 和 Andrey Proskurin

Large AI models are transforming the dig…

Microsoft Research Focus 03: Week of November 7th, 2022
微软研究院博客

Welcome to Research Focus, a new series …

Three bar plots. The first plot shows that the model size of XTC-BERT is 32 times smaller than that of BERT, and two dots show the accuracy of BERT and XTC-BERT, which are 83.95 and 83.44, respectively. The second one shows that INT8 using ZeroQuant can be 2.6 times faster than Baseline with FP16 using PyTorch and ZeoQuant can reduce the number of GPUs for inference from 2 to 1, which in total provides 5.2 times efficiency. It also shows that ZeroQuant has 50.4 accuracy compared to 50.5 using Baseline PyTorch. The third plot shows that ZeroQuant is more than 5000 times cheaper than baseline to compress a model, and the accuracy of ZeroQuant is 42.26 compared to 42.35 of baseline.
微软研究院博客

| DeepSpeed Team 和 Andrey Proskurin

Large-scale models are revolutionizing d…

DeepSpeed shares findings and innovations for MoE models and systems that 1) reduce training cost by 5x, 2) reduce MoE parameter size by up to 3.7x and 3) reduce MoE inference latency by 7.3x at an unprecedented scale and offer up to 4.5x faster and 9x cheaper inference for MoE models compared to quality-equivalent dense models.
微软研究院博客

| DeepSpeed Team 和 Andrey Proskurin

In the last three years, the largest tra…

Collage of four images. 1) a VR haptic pivot device 2) Ashley Lorens of Microsoft Research 3) an image of tractor on a farm 4) image of Race and Technology lecture series speakers.
微软研究院博客

Over the past 30 years, Microsoft Resear…

A line graph comparing the end-to-end performance of Meta’s MoE language model using Azure NDm A100 v4 VMs with and without Tutel. The x-axis is the number of A100 (80GB) GPUs, beginning at 8 and going up to 512, and the y-axis is the throughput (K tokens/s), beginning with 0 and going up to 1,000 in intervals of 100. Tutel always achieves higher throughput than fairseq.
微软研究院博客

| Wei Cui, Yifan Xiong, Peng Cheng, 和 Rafael Salas

Mixture of experts (MoE) is a deep learn…