Research Collection: Research Supporting Responsible AI
Editor’s Note: In the diverse and multifaceted world of research, individual contributions can add up to significant results over time. In this new series of posts, we’re connecting the dots to provide an overview of how researchers at Microsoft and their collaborators are working towards significant customer and societal outcomes that are broader than any single discipline. Here, we’ve curated a selection of the work Microsoft researchers are doing to advance responsible AI. Researchers Saleema Amershi, Ece Kamar, Kristin Lauter, Jenn Wortman Vaughan, and Hanna Wallach contributed to this post.
Responsible AI is really all about the how: how do we design, develop and deploy these systems that are fair, reliable, safe and trustworthy. And to do this, we need to think of Responsible AI as a set of socio-technical problems. We need to go beyond just improving the data and models. We also have to think about the people who are ultimately going to be interacting with these systems.”
—Dr. Saleema Amershi, Principal Researcher at Microsoft Research and Co-chair of the Aether Human-AI Interaction & Collaboration Working Group
Microsoft is committed to the advancement and use of AI grounded in principles that put people first and benefit society. We are putting these principles into practice throughout the company by embracing diverse perspectives, fostering continuous learning, and proactively responding as AI technology evolves.
Researchers at Microsoft are making significant contributions to the advancement of responsible AI practices, techniques and technologies – spanning areas of human-AI interaction and collaboration, fairness, intelligibility and transparency, privacy, reliability and safety, and other areas of research.
Multiple research efforts on responsible AI at Microsoft have been supported and coordinated by the company’s Aether Committee and its set of expert working groups. Aether is a cross-company board that plays a key role in the company’s work to operationalize responsible AI at scale, with efforts on formulating and making recommendations on issues and processes—and with hosting deep dives on technical challenges and tools around responsible AI.
Aether working groups focus on important opportunity areas, including human-AI interaction and collaboration, bias and fairness, intelligibility and transparency, reliability and safety, engineering practices, and sensitive uses of AI. Microsoft researchers actively lead and participate in the work of Aether, conducting research across disciplines and engaging with organizations and experts inside and outside of the company.
We embrace open collaboration across disciplines to strengthen and accelerate responsible AI, spanning software engineering and development to social sciences, user research, law and policy. To further this collaboration, we open-source many tools and datasets that others can use to contribute and build upon.
This work builds on Microsoft’s long history of innovation to make computing more accessible and dependable for people around the world – including the creation of the Microsoft Security Development Lifecycle, the Trustworthy Computing initiative, and pioneering work in accessibility and localization.
This page provides an overview of some key areas where Microsoft researchers are contributing to more responsible, secure and trustworthy AI systems. For more perspective on responsible AI and other technology and policy issues, check out our recent podcast with Microsoft President and Chief Legal Officer Brad Smith. For background on the Aether Committee, listen to this podcast with Microsoft’s Chief Scientist and Aether chair Eric Horvitz.
The fairness of AI systems is crucially important now that AI plays an increasing role in our daily lives. That’s why Microsoft researchers are advancing the frontiers of research on this topic, focusing on many different aspects of fairness, including:
- Definitions: Different types of fairness-related harms that occur in the context of AI, including harms that are specific to people with disabilities and harms arising from data quality issues, methodological pitfalls, and ethical limitations.
- Development practices: Ways to make fairness a priority throughout the AI development and deployment lifecycle by identifying industry practitioners’ needs for support in developing fairer AI systems and understanding organizational challenges and opportunities around fairness in AI.
- Applications: Fairness-related harms in natural language processing and information retrieval, such as gender stereotypes reflected in word embeddings, problematic predictive text, and homogenous search results, as well as ways to leverage some of these harms to achieve fairer outcomes in other tasks.
- The law: The relationship between AI and the law, including tensions between antidiscrimination laws and the use of AI systems in employment from both a disparate treatment and a disparate impact perspective.
For those who wish to prioritize fairness in their own AI systems, Microsoft researchers, in collaboration with Azure ML, have released Fairlearn, an open-source Python package that enables developers of AI systems to assess their systems’ fairness and mitigate any negative impacts for groups of people, such as those defined in terms of race, gender, age, or disability status. Fairlearn, which focuses specifically on harms of allocation or quality of service, draws on two papers by Microsoft researchers on incorporating quantitative fairness metrics into classification settings and regression settings, respectively. Of course, even with precise, targeted software tools like Fairlearn, it’s still easy for teams to overlook fairness considerations, especially when they are up against tight deadlines. This is especially true because fairness in AI sits at the intersection of technology and society and can’t be addressed with purely technical approaches. Microsoft researchers have therefore co-designed a fairness checklist to help teams reflect on their decisions at every stage of the AI lifecycle, in turn helping them anticipate fairness issues well before deployment.
- Download: Fairlearn
- Webinar: Machine Learning and Fairness with Dr. Jennifer Wortman Vaughan and Dr. Hanna Wallach
- Podcast: Fairness in Machine Learning with Hanna Wallach, Co-chair of the Aether Bias and Fairness Working Group
- Podcast: Advancing accessibility with Dr. Meredith Ringel Morris
- Tutorial: Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (cross-industry collaboration with researchers from Google and LinkedIn)
- Tutorial: Challenges of Incorporating Algorithmic ‘Fairness’ into Practice (cross-industry collaboration with researchers from Spotify and CMU)
- Blog: What’s in a name? Using Bias to Fight Bias in Occupational Classification
- Blog: Machine Learning for fair decisions
- Paper: Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI (CHI 2020 Best Paper Award)
- Additional papers: See cited papers
Transparency and Intelligibility
Intelligibility can uncover potential sources of unfairness, help users decide how much trust to place in a system, and generally lead to more usable products. It also can improve the robustness of machine learning systems by making it easier for data scientists and developers to identify and fix bugs. Because intelligibility is a fundamentally human concept, it’s crucial to take a human-centered approach to designing and evaluating methods for achieving intelligibility. That’s why Microsoft researchers are questioning common assumptions about what makes a model “interpretable,” studying data scientists’ understanding and use of existing intelligibility tools and how to make these tools more useable, and exploring the intelligibility of common metrics like accuracy.
For those eager to incorporate intelligibility into their own pipeline, Microsoft researchers have released InterpretML, an open-source Python package that exposes common model intelligibility techniques to practitioners and researchers. InterpretML includes implementations of both “glassbox” models (like Explainable Boosting Machines, which build on Generalized Additive Models) and techniques for generating explanations of blackbox models (like the popular LIME and SHAP, both developed by current Microsoft researchers).
Beyond model intelligibility, a thorough understanding of the characteristics and origins of the data used to train a machine learning model can be fundamental to building more responsible AI. The Datasheets for Datasets project proposes that every dataset be accompanied by a datasheet that documents relevant information about its creation, key characteristics, and limitations. Datasheets can help dataset creators uncover possible sources of bias in their data or unintentional assumptions they’ve made, help dataset consumers figure out whether a dataset is right for their needs, and help end users gain trust. In collaboration with the Partnership on AI, Microsoft researchers are developing best practices for documenting all components of machine learning systems to build more responsible AI.
- Download: InterpretML
- Webinar: Transparency and Intelligibility Throughout the Machine Learning Lifecycle with Dr. Jennifer Wortman Vaughan, Co-chair of the Aether Intelligibility and Transparency Working Group
- Podcast: Making Intelligence Intelligible with Dr. Rich Caruana, Co-chair of the Aether Intelligibility and Transparency Working Group
- Podcast: Visualizing Data and Other Big Ideas with Dr. Steven Drucker
- Book chapter: A Human-Centered Agenda for Intelligible Machine Learning
- Partner project: ABOUTML, with Partnership on AI
- Paper: Datasheets for Datasets
- Paper: Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning (CHI 2020 Honorable Mention Award)
- Paper: Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
- Project: DICE
- Additional papers: See cited papers
Reliability and Safety
Reliability is a principle that applies to every AI system that functions in the world and is required for creating trustworthy systems. A reliable system functions consistently and as intended, not only in the lab conditions in which it is trained, but also in the open world or when they are under attack from adversaries. When systems function in the physical world or when their shortcomings can pose risks to human lives, problems in system reliability translate to risks in safety.
To understand the way reliability and safety problems occur in AI systems, our researchers have been investigating how blind spots in data sets, mismatches between training environments and execution environments, distributional shifts and problems in model specifications can lead to shortcomings in AI systems. Given the various sources for failures, the key to ensuring system reliability is rigorous evaluation during system development and deployment so that unexpected performance failures can be minimized and system developers can be guided for continuous improvement. That is why Microsoft researchers have been developing new techniques for model debugging and error analysis that can reveal patterns that are correlated with disproportional error regions in evaluation data. Current efforts in this space include turning research ideas into tools for developers to use.
We recognize that when AI systems are used in applications that are critical for our society, in most cases to support human work, aggregate accuracy is not sufficient to quantify machine performance. Researchers have shown that model updates can lead to issues with backward compatibility (i.e., new errors occurring as a result of an update), even when overall model accuracy improves, which highlights that model performance should be seen as a multi-faceted concept with human-centered considerations.
- Podcast: Life at the Intersection of AI and Society with Dr. Ece Kamar, Co-chair of the Aether Reliability and Safety Group
- Podcast: How Programming Languages Quietly Run the World with Dr. Ben Zorn
- Paper: Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
- Paper: Discovering Blind Spots in Reinforcement Learning
- Paper: Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
- Paper: A Case for Backward Compatibility for Human-AI Teams
Human-AI Interaction and Collaboration
Advances in AI have the potential to enhance human capabilities and improve our lives. At the same time, the complexities and probabilistic nature of AI-based technologies presents unique challenges for safe, fair, and responsible human-AI interaction. That’s why Microsoft researchers are taking a human-centered approach to ensure that what we build benefits people and society, and that how we build it begins and ends with people in mind.
A human-centered approach to AI starts with identifying a human or societal need and then tailor-making AI technologies to support that need. Taking this approach, Microsoft researchers are creating new AI-based technologies to promote human and societal well-being including technologies to augment human capabilities, support mental health and focus and attention, and to understand the circulation patterns of fake news.
A human-centered approach to technology development also emphasizes the need for people to effectively understand and control those technologies to achieve their goals. This is inherently difficult for AI technologies that behave in probabilistic ways, may change over time, and are based on possibly multiple complex and entangled models. Microsoft researchers are therefore developing guidance and exploring new ways to support intuitive, fluid and responsible human interaction with AI including how to help people decide when to trust an AI or when to question it, to set appropriate expectations about an AI system’s capabilities and performance, to support the safe hand-off between people and AI-based systems, and to enable people and AI systems to interact and collaborate in physical space.
Finally, a human-centered approach to responsible AI requires understanding the unique challenges practitioners face building AI systems and then working to address those challenges. Microsoft researchers are therefore studying data scientists, machine learning software engineers, and interdisciplinary AI-UX teams, and creating new tools and platforms to support data analysis, characterizing and debugging AI failures, and developing of human-centered AI technologies such as emotion-aware and physically situated AI systems.
- Podcast: Responsible AI with Dr. Saleema Amershi, Co-chair of the Aether Human-AI Interaction and Collaboration Working Group
- Podcast: Adaptive systems, machine learning and collaborative AI with Dr. Besmira Nushi
- Podcast: Data science and machine learning for human well-being with Jina Suh
- Paper and guidance: Guidelines for Human-AI Interaction (CHI 2019 Honorable Mention Award)
- Paper: Software Engineering for Machine Learning: A Case Study (ICSE 2019 Best Paper Award)
- Paper: Managing Messes in Computational Notebooks (CHI 2019 Best Paper Award)
- Additional papers: See cited papers
Private AI is a Microsoft Research project to enable Privacy Preserving Machine Learning (PPML). The CryptoNets paper from ICML 2016 demonstrated that deep learning on encrypted data is feasible using a new technology called Homomorphic Encryption. Practical solutions and approaches for Homomorphic Encryption were pioneered by the Cryptography Group at Microsoft Research in this 2011 paper which showed a wide range of applications for providing security and privacy in the cloud for healthcare, genomics, and finance.
Homomorphic Encryption allows for computation to be done on encrypted data, without requiring access to a secret decryption key. The results of the computation are encrypted and can be revealed only by the owner of the key. Among other things, this technique can help to preserve individual privacy and control over personal data.
Microsoft researchers have been working to make Homomorphic Encryption simpler and more widely available, particularly through the open-source SEAL library. To learn more, listen to this podcast, take this webinar on Private AI from the National Academies, or this webinar from Microsoft Research on SEAL.
- Podcast: Takes from the Cryptography Lab with Dr. Kristin Lauter
- Download: Microsoft SEAL
- Webinar: Homomorphic Encryption with Microsoft SEAL
- National Academy of Sciences Webinar: Keeping the Internet Safe
- Talk: Private AI: Machine Learning on Encrypted Data
- Project: Homomorphic Encryption
- Paper: Can Homomorphic Encryption be Practical?
- Paper: CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy
- Additional papers: See cited papers
Partnerships and Support for Student Work
Working with Academic and Commercial Partners
Microsoft Research supports and works closely with Data & Society, which is committed is committed to identifying thorny issues at the intersection of technology and society, providing and encouraging research that can ground informed, evidence-based public debates, and building a network of researchers and practitioners who can anticipate issues and offer insight and direction. Microsoft Research also supports the AI Now Institute at New York University, an interdisciplinary research center dedicated to understanding the social implications of artificial intelligence.
Microsoft is also a member of the Partnership on AI (PAI), a multi-stakeholder organization that brings together academics, researchers, civil society organizations, companies building and utilizing AI technology, and other groups working to better understand AI’s impacts. Microsoft researchers are contributing to a number of PAI projects, including the ABOUT ML work referenced above.
Supporting Student Work on Responsible AI
The Ada Lovelace Fellowship and PhD Fellowship continue a Microsoft Research tradition of providing promising doctoral students in North America with funding to support their studies and research. Many of these fellowships’ 2020 recipients are doing work to advance the responsible and beneficial use of technology, including enhancing fairness in natural language processing, reducing bias, promoting social equality, and improving the mental, emotional, and social health of people with dementia.
- Blog: 2020 Ada Lovelace and PhD Fellowships help recipients achieve broad research and educational goals
- Toward Fairness in AI for People with Disabilities: A Research Roadmap
- Improving fairness in machine learning systems: What do industry practitioners need?
- Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI (CHI 2020 Best Paper Award)
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes (NAACL 2019 Best Thematic Paper Award) [blog post here]
- Stretching Human Laws to Apply to Machines: The Dangers of a ‘Colorblind’ Computer
- A Reductions Approach to Fair Classification [blog post here]
- Fair Regression: Quantitative Definitions and Reduction-based Algorithms
Transparency and Intelligibility
- Datasheets for Datasets
- Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning (CHI 2020 Honorable Mention Award)
- Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
- Manipulating and Measuring Model Interpretability
- Understanding the Effect of Trust in Machine Learning Models (CHI 2019 Honorable Mention Award)
- Explaining Machine Learning Classifiers through Diverse Counterfactual Examples
- Accurate Intelligible Models with Pairwise Interactions
- InterpretML: A Unified Framework for Machine Learning Interpretability
Reliability and Safety
- A Case for Backward Compatibility for Human-AI Teams
- Discovering Blind Spots in Reinforcement Learning
- Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
- Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Human-AI Interaction and Collaboration
- Guidelines for Human-AI Interaction (CHI 2019 Honorable Mention Award)
- Software Engineering for Machine Learning: A Case Study (ICSE 2019 Best Paper Award)
- Sketching NLP: A Case Study of Exploring the Right Things to Design with Language Intelligence (CHI 2019 Honorable Mention Award)
- Managing Messes in Computational Notebooks (CHI 2019 Best Paper Award)