Research
Last month, the DeepSpeed Team announced ZeRO-Infinity, a step forward in training models with tens of trillions of parameters. In addition to creating optimizations for scale, our team strives to introduce features that also improve speed, cost, and usability. As the DeepSpeed optimization library evolves,…
Accessibility and inclusion represent a growing space in the technology landscape, and how research and development are being used to empower people across abilities is expanding in exciting ways. Instead of treating disabilities as conditions in need of solutions—as has been the case over the…
The genetic sequence for COVID-19 was first published in January 2020. Before the end of the year, new vaccines—which typically take five to ten years to develop—were approved for emergency use in multiple nations. This astounding achievement is emblematic of a “reset” in the relationship…
Understanding video is one of the most challenging problems in AI, and an important underlying requirement is learning multimodal representations that capture information about objects, actions, sounds, and their long-range statistical dependencies from audio-visual signals. Recently, transformers have been successful in vision-and-language tasks such as image…
Over the course of the COVID-19 pandemic, some truths about the nature of work have been underscored: it is uniquely complex, quickly shifting, and increasingly technology-mediated. Teaching, medicine, mental health, and other professions—previously thought to be near-impossible to do remotely—have all abruptly moved to online…
Editor's Note: Bimanual controllers are frequently used to enhance the realism and immersion of virtual reality experiences such as games and simulations. Researchers have typically relied on mechanical linkages between the controllers to recreate the sensation of holding different objects with both hands. However, those…
One key aspiration of AI is to develop natural and effective task-oriented conversational systems. Task-oriented conversational systems use a natural language interface to collaborate with and support people in accomplishing specific goals and activities. They go beyond chitchat conversation. For example, as personal digital assistants,…
EverParse (opens in new tab) is a framework for generating provably secure parsers and formatters used to improve the security of critical code bases at Microsoft. EverParse is developed as part of Project Everest (opens in new tab), a collaboration between Microsoft Research labs in…
Project Alexandria is a research project within Microsoft Research Cambridge dedicated to discovering entities, or topics of information, and their associated properties from unstructured documents. This research lab has studied knowledge mining research for over a decade, using the probabilistic programming framework Infer.NET. Project Alexandria…
Since the DeepSpeed optimization library was introduced last year, it has rolled out numerous novel optimizations for training large AI models—improving scale, speed, cost, and usability. As large models have quickly evolved over the last year, so too has DeepSpeed. Whether enabling researchers to create…
Many of our online activities, from receiving and sending emails to searching for information to streaming movies, are driven behind the scenes by cloud-based distributed architectures. Writing concurrent software—programs with multiple logical threads of execution—is of paramount importance to scale to these growing computing needs.…
From BiT (928 million parameters (opens in new tab)) to GPT-3 (175 billion parameters (opens in new tab)), state-of-the-art machine learning models are rapidly growing in size. With the greater expressivity and easier trainability of these models come skyrocketing training costs, deployment difficulties, and even…