![An illustration of the KEAR architecture represented by five panels side by side. The first contains an input question—“What is a treat that your dog will enjoy?”—and the answer choices “salad,” “petted,” “affection,” “bone,” and “lots of attention.” The second panel has three boxes, each representing retrieval from a specific knowledge source. A box labeled “Knowledge Graph” has a silhouette of a dog and underneath it and labeled “desires” a silhouette of a dog being petted; a heart representing “affection”; a bone; and clapping hands representing “lots of attention.” A box labeled “relevant questions” has the question “What do dogs like to eat?” and the accompanying answer “Bones.” A boxed labeled “dictionary” contains the definition of “bone”: “a composite material making up the skeleton of most vertebrates.” The third panel, labeled “concatenation with input,” contains the input question followed by “Dog, desires, bone. Dog, desires, lots of attention” followed by the relevant question and finally the dictionary definition of bone. In between each is a separation token [SEP]. The fourth panel is labeled “language model” and contains a quote box labeled “language services,” a cube labeled “model,” and left and right braces punctuation within a circle labeled “language understanding.” The fifth panel is labeled “output” and includes silhouettes of each of the five answer choices. The silhouette of the bone is highlighted in blue, representing the appropriate response.](https://www.microsoft.com/en-us/research/wp-content/uploads/2021/12/1400x788_Common_Sense_still_no_logo-1-768x432.jpg){kind=logo}
![An illustration of the KEAR architecture represented by five panels side by side. The first contains an input question—“What is a treat that your dog will enjoy?”—and the answer choices “salad,” “petted,” “affection,” “bone,” and “lots of attention.” The second panel has three boxes, each representing retrieval from a specific knowledge source. A box labeled “Knowledge Graph” has a silhouette of a dog and underneath it and labeled “desires” a silhouette of a dog being petted; a heart representing “affection”; a bone; and clapping hands representing “lots of attention.” A box labeled “relevant questions” has the question “What do dogs like to eat?” and the accompanying answer “Bones.” A boxed labeled “dictionary” contains the definition of “bone”: “a composite material making up the skeleton of most vertebrates.” The third panel, labeled “concatenation with input,” contains the input question followed by “Dog, desires, bone. Dog, desires, lots of attention” followed by the relevant question and finally the dictionary definition of bone. In between each is a separation token [SEP]. The fourth panel is labeled “language model” and contains a quote box labeled “language services,” a cube labeled “model,” and left and right braces punctuation within a circle labeled “language understanding.” The fifth panel is labeled “output” and includes silhouettes of each of the five answer choices. The silhouette of the bone is highlighted in blue, representing the appropriate response.](https://www.microsoft.com/en-us/research/wp-content/uploads/2021/12/1400x788_Common_Sense_still_no_logo-1-scaled.jpg){kind=logo}
KEAR (Knowledgeable External Attention for commonsense Reasoning)—along with recent milestones in computer vision and neural text-to-speech—is part of a larger Azure AI (opens in new tab) mission to provide relevant, meaningful AI solutions and services that work better for people because they better capture how people learn and work—with…
Neural Text-to-Speech—along with recent milestones in computer vision and question answering—is part of a larger Azure AI (opens in new tab) mission to provide relevant, meaningful AI solutions and services that work better for people because they better capture how people learn and work—with improved…
Over the past 30 years, Microsoft Research has undergone a shift in how it approaches innovation, broadening its mission to include not only advancing the state of computing but also using technology to tackle some of the world’s most pressing challenges. That evolution has never…
The Project Florence Team Florence v1.0—along with recent milestones in Neural Text-to-Speech and question answering—is part of a larger Azure AI (opens in new tab)mission to provide relevant, meaningful AI solutions and services that work better for people because they better capture how people learn…
The drug development process is an iterative one that consists of discovery, design, and testing. Historically, drugs were derived from plants and discovered through trial-and-error experiments. Fortunately, this drug discovery process now occurs in a lab, with each iteration of custom-designed compounds producing a more…
Finding and fixing bugs in code is a time-consuming, and often frustrating, part of everyday work for software developers. Can deep learning address this problem and help developers deliver better software, faster? In a new paper, Self-Supervised Bug Detection and Repair, presented at the 2021…
Recent progress in natural language understanding (NLU) has been driven in part by the availability of large-scale benchmarks that provide an environment for researchers to test and measure the performance of AI models. Most of these benchmarks are designed for academic settings--typically datasets that feature…
As part of Microsoft AI at Scale (opens in new tab), the Turing family of NLP models are being used at scale across Microsoft to enable the next generation of AI experiences. Today, we are happy to announce that the latest Microsoft Turing model (T-NLRv5)…
Editor’s note: The researchers would like to acknowledge co-authors Christopher Takahashi, Gagan Gupta, Jake Smith, Richard Rouse, Paul Berndt, Sergey Yekhanin, David Ward, Siena Ang, Patrick Garvan, Hsing-Yeh Parker, Rob Carlson, Douglas Carmean, and Luis Ceze for their contributions to this work. Current estimates by…
Generative image models offer a unique value by creating new images. Such images can be sharp super-resolution versions of existing images or even realistic-looking synthetic photographs. Generative Adversarial Networks (GANs) and their variants have demonstrated pioneering success with the framework of training two networks against…
Mixture of experts (MoE) is a deep learning model architecture in which computational cost is sublinear to the number of parameters, making scaling easier. Nowadays, MoE is the only approach demonstrated to scale deep learning models to trillion-plus parameters, paving the way for models capable…
Today, we’re excited to announce the release of SynapseML (previously MMLSpark), an open-source library that simplifies the creation of massively scalable machine learning (ML) pipelines. Building production-ready distributed ML pipelines can be difficult, even for the most seasoned developer. Composing tools from different ecosystems often…
Meet our community of researchers, learn about exciting research topics, and grow your network
Ongoing conversations at the cutting edge of research
Join us for a continuous exchange of ideas about research in the era of general AI
