Project Turing header: electric pulse on black background

AI at Scale

Models, infrastructure and hardware for next-generation AI applications

新闻与深度文章

DeepSpeed MoE powers eight times bigger models using expert-parallelism + ZeRO-Offload compared with expert-parallelism only. A graph shows supported model sizes on NVIDIA A100 GPUs. DeepSpeed MoE scales near-linearly with respect to the number of GPUs. Z-code MoE (10B) consistently outperforms other systems on BLEU scores for an in-house 50 language test dataset. Read more in the blog post. 
微软研究院博客

| DeepSpeed Team 和 Z-code Team

Today, we are proud to announce DeepSpee…

Technical diagram of MEB model. MEB is a sparse neural network model composed of an input layer taking in binary features, a feature embedding layer transforming each binary feature into a 15-dimension vector, a sum pooling layer applied on each of 49 feature groups and concatenated to produce a 735-dimension vector, which is then passed through two dense layers to produce a click probability. Features shown in this figure are generated from the example query “Microsoft Windows” and document www.microsoft.com/en-us/windows.
微软研究院博客

| Junyan Chen, Frédéric Dubut, Jason (Zengzhong) Li, 和 Rangan Majumder

Recently, Transformer-based deep learnin…

DeepSpeed multi GPU inference offers up to 6.9 times throughput improvement for large deep learning model inference. Progressive Layer Dropping offers 2.8 times faster convergence for large model training. 1-bit LAMB offers up to 4.6 times less communication overhead. Single GPU speedups for inference: 2.1 times on BERT Base, 4.4 times on BERT Large, 3.8 times on GPT 2, 3.5 times on GPT 2 XL, 1.9 times on GPT Neo. Multi GPU speedups for inference: 6.2 times for Turing NLG, 3.7 times for 175 billion parameter language model.
微软研究院博客

| DeepSpeed Team, Rangan Majumder, 和 Andrey Proskurin

Last month, the DeepSpeed Team announced…

graphical user interface, text, application, email
微软研究院博客

| Rangan Majumder, Alec Berntson, Daxin Jiang (姜大昕), Jianfeng Gao, Furu Wei, 和 Nan Duan

Azure Cognitive Search (opens in new tab…

新闻报道 | WindowsClub

One fantastic feature in search engines …

新闻报道 | OnMSFT

Microsoft has launched a new language sy…

新闻报道 | ZDNet

Microsoft has explained how it is using …

新闻报道 | NextWeb

Microsoft has unveiled an AI system call…

Diagram shows Model Architecture of Microsoft Vision Model ResNet-50
微软研究院博客

| Jingwen Lu, Jidong Long (龙继东), 和 Rangan Majumder

At Microsoft Bing, our mission is to del…