Project Turing header: electric pulse on black background

AI at Scale

Models, infrastructure and hardware for next-generation AI applications

ニュース&特集

DeepSpeed MoE powers eight times bigger models using expert-parallelism + ZeRO-Offload compared with expert-parallelism only. A graph shows supported model sizes on NVIDIA A100 GPUs. DeepSpeed MoE scales near-linearly with respect to the number of GPUs. Z-code MoE (10B) consistently outperforms other systems on BLEU scores for an in-house 50 language test dataset. Read more in the blog post. 
Microsoft Research ブログ

| DeepSpeed Team と Z-code Team

Today, we are proud to announce DeepSpee…

Technical diagram of MEB model. MEB is a sparse neural network model composed of an input layer taking in binary features, a feature embedding layer transforming each binary feature into a 15-dimension vector, a sum pooling layer applied on each of 49 feature groups and concatenated to produce a 735-dimension vector, which is then passed through two dense layers to produce a click probability. Features shown in this figure are generated from the example query “Microsoft Windows” and document www.microsoft.com/en-us/windows.
Microsoft Research ブログ

| Junyan Chen, Frédéric Dubut, Jason (Zengzhong) Li, と Rangan Majumder

Recently, Transformer-based deep learnin…

DeepSpeed multi GPU inference offers up to 6.9 times throughput improvement for large deep learning model inference. Progressive Layer Dropping offers 2.8 times faster convergence for large model training. 1-bit LAMB offers up to 4.6 times less communication overhead. Single GPU speedups for inference: 2.1 times on BERT Base, 4.4 times on BERT Large, 3.8 times on GPT 2, 3.5 times on GPT 2 XL, 1.9 times on GPT Neo. Multi GPU speedups for inference: 6.2 times for Turing NLG, 3.7 times for 175 billion parameter language model.
Microsoft Research ブログ

| DeepSpeed Team, Rangan Majumder, と Andrey Proskurin

Last month, the DeepSpeed Team announced…

graphical user interface, text, application, email
Microsoft Research ブログ

| Rangan Majumder, Alec Berntson, Daxin Jiang (姜大昕), Jianfeng Gao, Furu Wei, と Nan Duan

Azure Cognitive Search (opens in new tab…

ニュース | WindowsClub

One fantastic feature in search engines …

ニュース | OnMSFT

Microsoft has launched a new language sy…

ニュース | ZDNet

Microsoft has explained how it is using …

ニュース | NextWeb

Microsoft has unveiled an AI system call…

Diagram shows Model Architecture of Microsoft Vision Model ResNet-50
Microsoft Research ブログ

| Jingwen Lu, Jidong Long (龙继东), と Rangan Majumder

At Microsoft Bing, our mission is to del…