Microsoft Research Blog

ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters 

February 13, 2020 | DeepSpeed Team, Rangan Majumder, and Junhua Wang
The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly advances large model training by…

Recent Posts

  1. Project Rocket platform—designed for easy, customizable live video analytics—is open source 

    January 22, 2020 | Ganesh Ananthanarayanan, Yuanchao Shu, Landon Cox, and Victor Bahl

    Thanks to advances in computer vision and deep neural networks (DNNs) in what can arguably be described as the golden age of vision, AI, and machine learning, video analytics systems—systems performing analytics on live camera streams—are becoming more accurate. This accuracy offers opportunities to support…

  2. When bias begets bias: A source of negative feedback loops in AI systems 

    January 21, 2020 | Lydia T. Liu

    Is bias in AI self-reinforcing? Decision-making systems that impact criminal justice, financial institutions, human resources, and many other areas often have bias. This is especially true of algorithmic systems that learn from historical data, which tends to reflect existing societal biases. In many high-stakes applications,…

  3. By making text-based games more accessible to RL agents, Jericho framework opens up exciting natural language challenges 

    January 16, 2020 | Matthew Hausknecht, Marc-Alexandre Côté, Eric Yuan, and Prithviraj Ammanabrolu

    You’re in a field. In front of you, there’s a white house. The door is boarded shut. The immediate challenge—investigate the house. The game—Zork I: The Great Underground Empire, a treasure-seeking adventure in which you’ll encounter monsters, a thief, and other interesting characters along the…

Explore More

  • Events & conferences

    Events & conferences 

    Meet our community of researchers, learn about exciting research topics, and grow your network

  • Podcasts

    Podcasts 

    Ongoing conversations at the cutting edge of research

  • Microsoft Research Forum

    Microsoft Research Forum 

    Join us for a continuous exchange of ideas about research in the era of general AI