Project Turing header: electric pulse on black background

AI at Scale

Models, infrastructure and hardware for next-generation AI applications

新闻与深度文章

Microsoft Research Focus 03: Week of November 7th, 2022
微软研究院博客

Welcome to Research Focus, a new series …

Three bar plots. The first plot shows that the model size of XTC-BERT is 32 times smaller than that of BERT, and two dots show the accuracy of BERT and XTC-BERT, which are 83.95 and 83.44, respectively. The second one shows that INT8 using ZeroQuant can be 2.6 times faster than Baseline with FP16 using PyTorch and ZeoQuant can reduce the number of GPUs for inference from 2 to 1, which in total provides 5.2 times efficiency. It also shows that ZeroQuant has 50.4 accuracy compared to 50.5 using Baseline PyTorch. The third plot shows that ZeroQuant is more than 5000 times cheaper than baseline to compress a model, and the accuracy of ZeroQuant is 42.26 compared to 42.35 of baseline.
微软研究院博客

| DeepSpeed Team 和 Andrey Proskurin

Large-scale models are revolutionizing d…

新闻报道 | ZDNet

Microsoft is updating its Translator and…

DeepSpeed shares findings and innovations for MoE models and systems that 1) reduce training cost by 5x, 2) reduce MoE parameter size by up to 3.7x and 3) reduce MoE inference latency by 7.3x at an unprecedented scale and offer up to 4.5x faster and 9x cheaper inference for MoE models compared to quality-equivalent dense models.
微软研究院博客

| DeepSpeed Team 和 Andrey Proskurin

In the last three years, the largest tra…

SuperGLUE leaderboards showing T-NLRv5 at the top
微软研究院博客

| Jianfeng Gao 和 Saurabh Tiwary

As part of Microsoft AI at Scale (opens …

Z-Code multilingual model

新闻报道 | Microsoft Translator Blog

Microsoft is on a quest for AI at Scale …

An illustration of how the image text contrastive and translation text contrastive tasks work together to help align the space of images, English text and non-English text. On the left side of the illustration, the three domains—Image Domain, English Domain, and Non-English Domain--are segregated. An arrow labeled “Image-Captions training data” points to another depiction of the three domains where the image domain and the English domain intersect but the non-English domain is still separate and shown in gray to show that it’s not significantly affected. A two headed arrow with the label “Image-Text contrastive loss” is drawn between the image and English domains. Towards the bottom of the image, an arrow labeled “Parallel corpus training data” points to another depiction of the three domains where the English domain and the non-English domain intersect but the image domain is separate and shown in gray to indicate that it is not significantly affected. A two-headed arrow with the label “Translated Text Contrastive loss” is drawn between the English and non-English domains. Finally, a third arrow with the label “Resulting Effect” is drawn to the right of the image which points to a depiction of all three domains intersecting.
微软研究院博客

| Saurabh Tiwary

Today, the Microsoft Turing team (opens …

Figure 1. Trend of sizes of state-of-the-art NLP models over time
微软研究院博客

| Ali Alvi 和 Paresh Kharya

We are excited to introduce the DeepSpee…

XTREME leaderboard showing T-ULRv5 at the top.
微软研究院博客

| Saurabh Tiwary 和 Lidong Zhou

Today, we are excited to announce that w…