DeepSpeed MoE powers eight times bigger models using expert-parallelism + ZeRO-Offload compared with expert-parallelism only. A graph shows supported model sizes on NVIDIA A100 GPUs. DeepSpeed MoE scales near-linearly with respect to the number of GPUs. Z-code MoE (10B) consistently outperforms other systems on BLEU scores for an in-house 50 language test dataset. Read more in the blog post. 
Microsoft Research Blog

| DeepSpeed Team and Z-code Team

Today, we are proud to announce DeepSpeed MoE, a high-performance system that supports massive scale mixture of experts (MoE) models as part of the DeepSpeed (opens in new tab) optimization library. MoE models are an emerging class of sparsely activated…

hand holding a book bridging the gap in primary education for children passing by
Articles

Our guidelines for informing inclusive product design for students are a work in progress. We began with Microsoft’s universal design principles and the Universal Design for Learning guidelines and supplemented those with research other teams within the company conducted with…

In the news | Microsoft Educator Developer Blog

The Microsoft Learn Student Ambassadors community is for students who want to use tech to solve real-world problems with like-minded peers, establish themselves as mentors and leaders in their community, and amplify their impact. The Microsoft Learn Student Ambassadors, Health League…

comodgan

In the news | AI Lab

CoModGAN is an image completion tool that uses AI to complete an image that is missing significant amounts of visual information. Two neural networks—a generator tasked with filing in missing information and a discriminator that analyzes the realism of the…

Portraits of Microsoft researchers Sid Suri and Jaime Teevan photographed in black and white. Both smile and look forward. Teevan, on the right, is holding a cell phone in the lower right of the frame.
Microsoft Research Podcast

In this episode of The New Future of Work series, Chief Scientist Jaime Teevan and Senior Principal Researcher Siddharth Suri explore the many ways people were impacted by work shifts during the COVID-19 pandemic. They talk about how race, gender, income, and other factors are indicative of how…

An illustration of resolving a bad merge into a safe merge. Moving from left to right, circles on a continuous line represent code commits in a version control system. A circle labeled “Base” is the most common ancestor of the commits marked A and B, respectively. All three commits pass the project’s quality gates, denoted by green check marks alongside each of these commits. The subsequent merge results in a failure of some quality gate, denoted by a blue circle labeled “Bad merge” with a red x above it. The repair uses machine learning, denoted by an abstract image of a neural network, and program verification and synthesis, denoted by a formal inference rule containing math symbols, to construct a safe merge that passes the quality gate, denoted by a circle outlined in green with a green check mark above it.   
Microsoft Research Blog

| Shuvendu Lahiri

Since the computing world began embracing an open-source approach to programming, building software has become increasingly collaborative. Members of development teams with as few as two developers and as many as thousands are simultaneously editing different components in creating software…

People on a remote conference call
Articles

Direct customer interaction is key to understanding users’ changing needs. Over the past 18 months, researchers have adapted the methods they use to meet with customers in real time as there are many asynchronous methods that researchers use to practice…

Articles

编者按：微软亚洲研究院“铸星计划”旨在发掘和助力新一代青年学者，使其成为科研创新能力突出、走在世界科技前沿的学术带头人。 无论是与领域内顶尖研究员合作的机会，还是最新、最丰富的数据集和强大的支持资源，抑或是产业界独有的实际应用场景，都吸引着青年才俊们来到微软亚洲研究院探索领域内前沿新知。 年轻、开拓、探索，是铸星计划的关键词；合作、创新、成就，是每个学术新星发光闪耀的必经之途。通过微软亚洲研究院“...