{"id":630972,"date":"2020-01-13T17:04:06","date_gmt":"2020-01-14T01:04:06","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=630972"},"modified":"2020-02-08T08:54:35","modified_gmt":"2020-02-08T16:54:35","slug":"aaai-2020-tutorial-guidelines-for-human-ai-interaction","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/aaai-2020-tutorial-guidelines-for-human-ai-interaction\/","title":{"rendered":"AAAI 2020 Tutorial – Guidelines for Human-AI Interaction"},"content":{"rendered":"

Organizers<\/strong>: Adam Fourney (opens in new tab)<\/span><\/a>, Besmira Nushi (opens in new tab)<\/span><\/a>, Dan Weld (opens in new tab)<\/span><\/a>, Saleema Amershi (opens in new tab)<\/span><\/a><\/p>\n

Location<\/strong>: Hilton New York Midtown, NY (room: Sutton South)<\/p>\n

Date | Time<\/strong>: Saturday, February 8, 2020 | 8:30 am \u2013 10:15 am<\/p>\n

Tutorial Slides<\/strong><\/a><\/p>\n

Considerable research attention has focused on improving the raw performance of AI and ML systems, but much less on the best ways to facilitate effective human-AI interaction. Due to their probabilistic behavior and inherent uncertainty, AI-based systems are fundamentally different from traditional computing systems and mismatches between AI capabilities and user experience (UX) design can cause frustrating and even harmful outcomes. Therefore, the development of and deployment of beneficial AI systems affording appropriate user experiences requires guidelines to help AI developers make informed decisions with respect to model selection, objective function design, and data collection. This tutorial will introduce the audience with a comprehensive set of guidelines for building systems and interfaces designed for fluid human-AI interaction. The guidelines were validated through a rigorous, 3-step process described in the CHI 2019 paper,\u00a0Guidelines for Human-AI Interaction<\/a>. They recommend best practices for how AI systems should behave upon initial interaction, during regular interaction, when they\u2019re inevitably wrong, and over time. Most importantly, the tutorial will also reflect upon the research and engineering challenges whose solutions can enable the implementation of such guidelines for real-world AI systems.<\/p>\n

\n
\n

This is the first time this tutorial is being organized and we hope it will promote an inter-community discussion on how to build and deploy human-centered machine learning. The audience needs to be familiar with basic concepts in AI and ML, such as training and validation, optimization techniques, and objective functions. For feedback and questions please reach us at aiguidelines@microsoft.com<\/a>.<\/p>\n<\/div>\n<\/div>\n

Schedule<\/strong><\/h4>\n
8:30 – 9:15: Introduction to human-AI interaction guidelines<\/strong> (by Saleema Amershi and Adam Fourney)<\/div>\n

An introduction to user interaction guidelines as they relate to AI-based systems. This will include a discussion of the limitations of traditional guidelines in supporting AI and an overview of the new guidelines for human-AI interaction. The human-AI interaction guidelines will be explained through real-world examples from our user studies and follow-up evaluations.<\/p>\n

9:15 – 9:45: Implications for ML and engineering<\/strong> (by Besmira Nushi)<\/div>\n

A summary of implications and pre-requisites that human-AI collaboration in general and the presented guidelines in particular pose on algorithm design, engineering, and tool infrastructure. Mapping current active challenges in machine learning with the presented guidelines.<\/p>\n

9:45 – 10:15: Algorithmic challenges for human-centered AI<\/strong> (by Dan Weld)<\/div>\n

Decision-theoretic fundamentals of mixed-initiative and adaptive interfaces. Principles of explicable, legible, predicable and transparent AI. Algorithms for explaining learned classifiers and UI tradeoffs in communicating explanations.<\/p>\n

Relevant Literature<\/strong><\/h4>\n

Horvitz, Eric. “Principles of mixed-initiative user interfaces.” In\u00a0Proceedings of the SIGCHI conference on Human Factors in Computing Systems<\/i>, pp. 159-166. ACM, 1999. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh et al. “Guidelines for human-AI interaction.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems<\/i>, p. 3. ACM, 2019. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Rafal Kocielnik, Saleema Amershi, and Paul N. Bennett. “Will You Accept an Imperfect AI?: Exploring Designs for Adjusting End-user Expectations of AI Systems.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems<\/i>, p. 411. ACM, 2019. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, and Eric Horvitz. “Updates in human-ai teams: Understanding and addressing the performance\/compatibility tradeoff.” In Proceedings of the AAAI Conference on Artificial Intelligence<\/i>, vol. 33, pp. 2429-2437. 2019. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. “Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance.” In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing<\/i>, vol. 7, no. 1, pp. 2-11. 2019. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Daniel S. Weld, and Gagan Bansal. “The challenge of crafting intelligible intelligence.” Communications of the ACM<\/i> 62, no. 6 (2019): 70-79. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Besmira Nushi, Ece Kamar, and Eric Horvitz. “Towards accountable ai: Hybrid human-machine analyses for characterizing system failure.” In Sixth AAAI Conference on Human Computation and Crowdsourcing<\/i>. 2018. Pdf (opens in new tab)<\/span><\/a><\/p>\n

Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. “Manipulating and measuring model interpretability.” arXiv preprint arXiv:1802.07810<\/i> (2018). Pdf (opens in new tab)<\/span><\/a><\/p>\n

Resources on the HAI Guidelines<\/strong><\/h4>\n

Learn the guidelines<\/p>\n