Turing-NLG: A 17-billion-parameter language model by Microsoft
| Corby Rosset
This figure was adapted from a similar image published in DistilBERT. Turing Natural Language Generation (T-NLG) is a 17 billion parameter language model by Microsoft that outperforms the state of the art on many downstream NLP tasks. We present a…
ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters
| DeepSpeed Team, Rangan Majumder, and Junhua Wang
The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly…
In the news | MSPowerUser
VROOM: Microsoft Research envisions a workplace filled with Telepresence Robots and AR headsets (video)
Microsoft Research have posted a video on a project called VROOM for Virtual Robot Overlay for Online Meetings. Microsoft imagines users being able to visit locations via a telepresence robot and being represented remotely via an avatar which overlays the…
In the news | Future of Information and Communication Conference
Ecommerce Fraud Detection Through Fraud Islands and Multi-layer Machine Learning Model
Main challenge for e-commerce transaction fraud prevention is that fraud patterns are rather dynamic and diverse. This paper introduces two innovative methods, fraud islands (link analysis) and multi-layer machine learning model, which can effectively tackle the challenge of detecting diverse…
Microsoft Scheduler and dawn of Intelligent PDAs with Dr. Pamela Bhattacharya
In a world where productivity is paramount and only a handful of people have personal assistants, many of us are frustrated by the amount of time we spend in meetings, and worse, the amount time we spend planning, scheduling and…
In the news | Fortune
A.I. and tackling the risk of “digital redlining”
Last week, a Dutch court ordered the government in the Netherlands to stop using a machine-learning algorithm for detecting welfare fraud, citing human rights violations. The system, called System Risk Indicator (SyRI) in English, was being used by four Dutch…
In the news | The Register
Meet Clippy 9000: Microsoft brags about building Earth’s largest AI language model, refuses to let it out of the lab
There’s a new giant AI language model in town: enter Microsoft’s Turing-NLG system, which apparently contains a whopping 17 billion parameters, making it the largest publicly known model of its class yet.
In the news | Neowin
Microsoft builds the world’s largest transformer-based language generation model
Transformer-based language generation models have enabled better conversational applications. Though they still have their shortcomings, which were recently exposed by a team at MIT, researchers continue improving them to build better, larger, and more robust models.