SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso, Nick Saw, Tim Schoonbeek, Maia Stiber
November 2025
Sean Andrist
Principal Researcher
论文与出版物
-
-
-
Converting Spatial to Social: Using Persistent Homology to Understand Social Groups
Valerie K. Chen, Claire Liang, Julie A. Shah, Sean Andrist
Proceedings of the 27th International Conference on Multimodal Interaction | October 2025
-
New Initiatives at ICMI: Mentorship in Focus
Catharine Oertel, Laura Cabrera-Quiros, Hayley Hung, Mohammad Soleymani, Sean Andrist, Dimosthenis Kontogiorgos, Chirag Raman
Voices of SIGCHI | March 2025
-
-
-
“Uh, This One?”: Leveraging Behavioral Signals for Detecting Confusion during Physical Tasks
Maia Stiber, Dan Bohus, Sean Andrist
Proceedings of the 26th International Conference on Multimodal Interaction | November 2024
-
“Is This It?”: Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Emily Bao, Eric Horvitz, Ann Paradiso
Companion Proceedings of the 26th International Conference on Multimodal Interaction | August 2024
-
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research – Extended Abstract
Dan Bohus, Sean Andrist, Nick Saw, Ann Paradiso, Ishani Chakraborty, Mahdi Rad
2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | March 2024, pp. 889-890
-
Interaction-Shaping Robotics: Robots that Influence Interactions between Other Agents
Sarah Gillet, Marynel Vázquez, Sean Andrist, Iolanda Leite, S. Sebo
ACM Transactions on Human-Robot Interaction | February 2024
-
-
-
Platform for Situated Intelligence and OpenSense: A Tutorial on Building Multimodal Interactive Applications for Research
Sean Andrist, Dan Bohus, Zongjian Li, Mohammad Soleymani
October 2023
-
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Felipe Vieira Frujeri, Neel Joshi, Marc Pollefeys
ICCV 2023 | September 2023
-
Symbiotic Society with Avatars (SSA): Beyond Space and Time
Hooman Hedayati, S. Seo, T. Kanda, Daniel J. Rea, Sean Andrist, Y. Nakano, H. Ishiguro
Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction | March 2023
-
-
-
Continual Learning about Objects in the Wild: An Interactive Approach
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Eric Horvitz
International Conference on Multimodal Interaction | November 2022
-
Developing Mixed Reality Applications with Platform for Situated Intelligence
Sean Andrist, Dan Bohus, Ashley Feniello, Nick Saw
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | February 2022, pp. 48-50
-
-
-
Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Pat Sweeney, Anne Loomis Thompson, Eric Horvitz
MSR-TR-2021-2 | March 2021
作者：Microsoft
-
-
-
Lessons Learned in Designing AI for Autistic Adults
Andrew Begel, John Tang, Sean Andrist, Mike Barnett, Tony Carbary, Piali Choudhury, Ed Cutrell, Alberto Fung, Sasa Junuzovic, Daniel McDuff, Kael Rowan, Shibashankar Sahoo, Jennifer Frances Waldern, Jessica Wolk, Hui Zheng, Annuska Zolyomi
ASSETS 2020: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility | October 2020
-
Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence
AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction: Trust & Explainability in Artificial Intelligence for Human-Robot Interaction | October 2020
-
REFORM: Recognizing F-formations for Social Robots
Hooman Hedayati, Annika Muehlbradt, Daniel J. Szafir, Sean Andrist
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020) | August 2020
PDF PDF Publication Publication Publication Publication Publication 项目
-
Now, Over Here: Leveraging Extended Attentional Capabilities in Human-Robot Interaction.
Xiang Zhi Tan, Sean Andrist, Dan Bohus, Eric Horvitz
ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20 Late-breaking Reports) | March 2020
-
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
-
-
-
Social cognitive systems in smart environments: Approaches for learning, reasoning, and adaptation
Amir Aly, Shashank Pathak, Sean Andrist, Armando Tacchella
Cognitive Systems Research | December 2019, 第 58 卷
-
Managing Stress: The Needs of Autistic Adults in Video Calling
Annuska Zolyomi, Andrew Begel, Jennifer Frances Waldern, John Tang, Mike Barnett, Ed Cutrell, Daniel McDuff, Sean Andrist, Meredith Ringel Morris
CSCW 2019 | November 2019
Best Paper Honorable Mention
-
Demonstrating a Framework for Rapid Development of Physically Situated Interactive Systems
Sean Andrist, Dan Bohus, Ashley Feniello
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Recognizing F-Formations in the Open World
Hooman Hedayati, Daniel Szafir, Sean Andrist
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
-
-
A network analytic approach to gaze coordination during a collaborative task
Sean Andrist, A. R. Ruis, David Williamson Shaffer
Computers in Human Behavior | December 2018, 第 89 卷: pp. 339-348
-
-
-
Rapid Development of Multimodal Interactive Systems: A Demonstration of Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Mihai Jalobeanu
ICMI 2017 Proceedings of the 19th ACM International Conference on Multimodal Interaction | November 2017
-
A study in scene shaping: Adjusting F-formations in the wild
Dan Bohus, Sean Andrist, Eric Horvitz
2017 AAAI Fall Symposium: Natural Communication for Human-Robot Collaboration | September 2017
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters
Sean Andrist, Michael Gleicher, Bilge Mutlu
Proceedings of CHI 2017 | May 2017
-
Human-Agent Collaboration: Can an Agent be a Partner?
Rachel K. E. Bellamy, Sean Andrist, Timothy Bickmore, Elizabeth F. Churchill, Thomas Erickson
Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17) | May 2017
-
-
-
Gaze Mechanisms for Situated Interaction with Embodied Agents
May 2016
博士论文：University of Wisconsin - Madison | May 2016
-
Are You Messing with Me? Querying About the Sincerity of Interactions in the Open World
Sean Andrist, Dan Bohus, Zhou Yu, Eric Horvitz
2016 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’16 Late-breaking Reports) | January 2016
-
Turn-Taking and Coordination in Human-Machine Interaction
Sean Andrist, Dan Bohus, Bilge Mutlu, David Schlangen
AI Magazine, Winter Issue | January 2016, 第 34 卷
-
-
-
Using Gaze Patterns to Predict Task Intent in Collaboration
Chien-Ming Huang, Sean Andrist, Allison Sauppe, Bilge Mutlu
frontiers in Pyschology | July 2015, 第 6 卷(1049): pp. 1-12
-
Look Together: Analyzing Gaze Coordination with Epistemic Network Analysis
Sean Andrist, Wesley Collier, Michael Gleicher, Bilge Mutlu, David Shaffer
frontiers in Pyschology | July 2015, 第 6 卷(1016): pp. 1-15
-
Look Like Me: Matching Robot Personality via Gaze to Increase Motivation
Sean Andrist, Bilge Mutlu, Adriana Tapus
33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15) | April 2015
Best of CHI Honorable Mention Award
-
Effects of Culture on the Credibility of Robot Speech: A Comparison between English and Arabic
Sean Andrist, Micheline Ziadee, Halim Boukaram, Bilge Mutlu, Majd Sakr
Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ’15) | March 2015
-
Gaze and Attention Management for Embodied Conversational Agents
Tomislav Pejsa, Sean Andrist, Michael Gleicher, Bilge Mutlu
ACM Transactions on Interactive Intelligent Systems (TiiS) | March 2015, 第 5 卷(1)
-
A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and Perception
Sean Andrist, et al
February 2015
-
-
-
Conversational Gaze Aversion for Humanlike Robots
Sean Andrist, Xiang Zhi Tan, Michael Gleicher, Bilge Mutlu
HRI '14 Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction | March 2014
Best Paper Award Nominee
-
“Look Me in the Eyes”: A Survey of Eye and Gaze Animation for Virtual Agents and Artificial Systems
K. Ruhland, Sean Andrist, J. B. Badler, C. E. Peters, N. I. Badler, M. Gleicher, B. Mutlu, R. McDonnell
Eurographics 2014 – State of the Art Reports (EG ’14 STARs) | February 2014
-
-
-
Managing Chaos: Models of Turn-taking in Character-multichild Interactions
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
15th ACM on International conference on multimodal interaction | December 2013
-
Fun and Fair: Influencing Turn-taking in a Multi-party Game with a Virtual Agent
Sean Andrist, Iolanda Leite, Jill Lehman
IDC '13 Proceedings of the 12th International Conference on Interaction Design and Children | June 2013
-
Take or Wait? Learning Turn-Taking from Multiparty Data
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
AAAI Conference on Artificial Intelligence (Late-Breaking Developments) | April 2013
-
Conversational Gaze Aversion for Virtual Agents
Sean Andrist, Bilge Mutlu, Michael Gleicher
13th International Conference on Intelligent Virtual Agents (IVA ’13) | February 2013
-
Controllable Models of Gaze Behavior for Virtual Agents and Humanlike Robots
ICMI '13 - 15th ACM on International conference on multimodal interaction | January 2013
-
Rhetorical Robots: Making Robots More Effective Speakers Using Linguistic Cues of Expertise
Sean Andrist, Erin Spannan, Bilge Mutlu
8th ACM/IEEE International Conference on Human-Robot Interaction (HRI ’13) | January 2013
-
-
-
Designing Effective Gaze Mechanisms for Virtual Agents
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
CHI '12 - SIGCHI Conference on Human Factors in Computing Systems | November 2012
-
A Head-Eye Coordination Model for Animating Gaze Shifts of Virtual Characters
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
14th International Conference on Multimodal Interaction (ICMI ’12), 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction (Gaze-In ’12) | July 2012
Article No. 4
-
-
-
Decision and Coordination Strategies for RoboCup Rescue Agents
Maitreyi Nanjanath, Alexander J. Erlandson, Sean Andrist, Aravind Ragipindi, Abdul A. Mohammed, Ankur S. Sharma, Maria Gini
Second International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR ’10) | February 2010
-
-
-
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso, Nick Saw, Tim Schoonbeek, Maia Stiber
November 2025
-
Converting Spatial to Social: Using Persistent Homology to Understand Social Groups
Valerie K. Chen, Claire Liang, Julie A. Shah, Sean Andrist
Proceedings of the 27th International Conference on Multimodal Interaction | October 2025
-
New Initiatives at ICMI: Mentorship in Focus
Catharine Oertel, Laura Cabrera-Quiros, Hayley Hung, Mohammad Soleymani, Sean Andrist, Dimosthenis Kontogiorgos, Chirag Raman
Voices of SIGCHI | March 2025
-
“Uh, This One?”: Leveraging Behavioral Signals for Detecting Confusion during Physical Tasks
Maia Stiber, Dan Bohus, Sean Andrist
Proceedings of the 26th International Conference on Multimodal Interaction | November 2024
-
“Is This It?”: Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Emily Bao, Eric Horvitz, Ann Paradiso
Companion Proceedings of the 26th International Conference on Multimodal Interaction | August 2024
-
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research – Extended Abstract
Dan Bohus, Sean Andrist, Nick Saw, Ann Paradiso, Ishani Chakraborty, Mahdi Rad
2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | March 2024, pp. 889-890
-
Interaction-Shaping Robotics: Robots that Influence Interactions between Other Agents
Sarah Gillet, Marynel Vázquez, Sean Andrist, Iolanda Leite, S. Sebo
ACM Transactions on Human-Robot Interaction | February 2024
-
Platform for Situated Intelligence and OpenSense: A Tutorial on Building Multimodal Interactive Applications for Research
Sean Andrist, Dan Bohus, Zongjian Li, Mohammad Soleymani
October 2023
-
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Felipe Vieira Frujeri, Neel Joshi, Marc Pollefeys
ICCV 2023 | September 2023
-
Symbiotic Society with Avatars (SSA): Beyond Space and Time
Hooman Hedayati, S. Seo, T. Kanda, Daniel J. Rea, Sean Andrist, Y. Nakano, H. Ishiguro
Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction | March 2023
-
Continual Learning about Objects in the Wild: An Interactive Approach
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Eric Horvitz
International Conference on Multimodal Interaction | November 2022
-
Developing Mixed Reality Applications with Platform for Situated Intelligence
Sean Andrist, Dan Bohus, Ashley Feniello, Nick Saw
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | February 2022, pp. 48-50
-
Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Pat Sweeney, Anne Loomis Thompson, Eric Horvitz
MSR-TR-2021-2 | March 2021
作者：Microsoft
-
Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence
AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction: Trust & Explainability in Artificial Intelligence for Human-Robot Interaction | October 2020
-
REFORM: Recognizing F-formations for Social Robots
Hooman Hedayati, Annika Muehlbradt, Daniel J. Szafir, Sean Andrist
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020) | August 2020
PDF PDF Publication Publication Publication Publication Publication 项目
-
Now, Over Here: Leveraging Extended Attentional Capabilities in Human-Robot Interaction.
Xiang Zhi Tan, Sean Andrist, Dan Bohus, Eric Horvitz
ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20 Late-breaking Reports) | March 2020
-
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
-
Social cognitive systems in smart environments: Approaches for learning, reasoning, and adaptation
Amir Aly, Shashank Pathak, Sean Andrist, Armando Tacchella
Cognitive Systems Research | December 2019, 第 58 卷
-
Demonstrating a Framework for Rapid Development of Physically Situated Interactive Systems
Sean Andrist, Dan Bohus, Ashley Feniello
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Recognizing F-Formations in the Open World
Hooman Hedayati, Daniel Szafir, Sean Andrist
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Rapid Development of Multimodal Interactive Systems: A Demonstration of Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Mihai Jalobeanu
ICMI 2017 Proceedings of the 19th ACM International Conference on Multimodal Interaction | November 2017
-
A study in scene shaping: Adjusting F-formations in the wild
Dan Bohus, Sean Andrist, Eric Horvitz
2017 AAAI Fall Symposium: Natural Communication for Human-Robot Collaboration | September 2017
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters
Sean Andrist, Michael Gleicher, Bilge Mutlu
Proceedings of CHI 2017 | May 2017
-
Human-Agent Collaboration: Can an Agent be a Partner?
Rachel K. E. Bellamy, Sean Andrist, Timothy Bickmore, Elizabeth F. Churchill, Thomas Erickson
Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17) | May 2017
-
Gaze Mechanisms for Situated Interaction with Embodied Agents
May 2016
博士论文：University of Wisconsin - Madison | May 2016
-
Are You Messing with Me? Querying About the Sincerity of Interactions in the Open World
Sean Andrist, Dan Bohus, Zhou Yu, Eric Horvitz
2016 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’16 Late-breaking Reports) | January 2016
-
Turn-Taking and Coordination in Human-Machine Interaction
Sean Andrist, Dan Bohus, Bilge Mutlu, David Schlangen
AI Magazine, Winter Issue | January 2016, 第 34 卷
-
Using Gaze Patterns to Predict Task Intent in Collaboration
Chien-Ming Huang, Sean Andrist, Allison Sauppe, Bilge Mutlu
frontiers in Pyschology | July 2015, 第 6 卷(1049): pp. 1-12
-
Look Together: Analyzing Gaze Coordination with Epistemic Network Analysis
Sean Andrist, Wesley Collier, Michael Gleicher, Bilge Mutlu, David Shaffer
frontiers in Pyschology | July 2015, 第 6 卷(1016): pp. 1-15
-
Look Like Me: Matching Robot Personality via Gaze to Increase Motivation
Sean Andrist, Bilge Mutlu, Adriana Tapus
33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15) | April 2015
Best of CHI Honorable Mention Award
-
Gaze and Attention Management for Embodied Conversational Agents
Tomislav Pejsa, Sean Andrist, Michael Gleicher, Bilge Mutlu
ACM Transactions on Interactive Intelligent Systems (TiiS) | March 2015, 第 5 卷(1)
-
Effects of Culture on the Credibility of Robot Speech: A Comparison between English and Arabic
Sean Andrist, Micheline Ziadee, Halim Boukaram, Bilge Mutlu, Majd Sakr
Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ’15) | March 2015
-
A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and Perception
Sean Andrist, et al
February 2015
-
Conversational Gaze Aversion for Humanlike Robots
Sean Andrist, Xiang Zhi Tan, Michael Gleicher, Bilge Mutlu
HRI '14 Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction | March 2014
Best Paper Award Nominee
-
“Look Me in the Eyes”: A Survey of Eye and Gaze Animation for Virtual Agents and Artificial Systems
K. Ruhland, Sean Andrist, J. B. Badler, C. E. Peters, N. I. Badler, M. Gleicher, B. Mutlu, R. McDonnell
Eurographics 2014 – State of the Art Reports (EG ’14 STARs) | February 2014
-
Managing Chaos: Models of Turn-taking in Character-multichild Interactions
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
15th ACM on International conference on multimodal interaction | December 2013
-
Fun and Fair: Influencing Turn-taking in a Multi-party Game with a Virtual Agent
Sean Andrist, Iolanda Leite, Jill Lehman
IDC '13 Proceedings of the 12th International Conference on Interaction Design and Children | June 2013
-
Take or Wait? Learning Turn-Taking from Multiparty Data
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
AAAI Conference on Artificial Intelligence (Late-Breaking Developments) | April 2013
-
Conversational Gaze Aversion for Virtual Agents
Sean Andrist, Bilge Mutlu, Michael Gleicher
13th International Conference on Intelligent Virtual Agents (IVA ’13) | February 2013
-
Controllable Models of Gaze Behavior for Virtual Agents and Humanlike Robots
ICMI '13 - 15th ACM on International conference on multimodal interaction | January 2013
-
Rhetorical Robots: Making Robots More Effective Speakers Using Linguistic Cues of Expertise
Sean Andrist, Erin Spannan, Bilge Mutlu
8th ACM/IEEE International Conference on Human-Robot Interaction (HRI ’13) | January 2013
-
Designing Effective Gaze Mechanisms for Virtual Agents
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
CHI '12 - SIGCHI Conference on Human Factors in Computing Systems | November 2012
-
A Head-Eye Coordination Model for Animating Gaze Shifts of Virtual Characters
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
14th International Conference on Multimodal Interaction (ICMI ’12), 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction (Gaze-In ’12) | July 2012
Article No. 4
-
Decision and Coordination Strategies for RoboCup Rescue Agents
Maitreyi Nanjanath, Alexander J. Erlandson, Sean Andrist, Aravind Ragipindi, Abdul A. Mohammed, Ankur S. Sharma, Maria Gini
Second International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR ’10) | February 2010
-
-
-
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso, Nick Saw, Tim Schoonbeek, Maia Stiber
November 2025
-
Converting Spatial to Social: Using Persistent Homology to Understand Social Groups
Valerie K. Chen, Claire Liang, Julie A. Shah, Sean Andrist
Proceedings of the 27th International Conference on Multimodal Interaction | October 2025
-
New Initiatives at ICMI: Mentorship in Focus
Catharine Oertel, Laura Cabrera-Quiros, Hayley Hung, Mohammad Soleymani, Sean Andrist, Dimosthenis Kontogiorgos, Chirag Raman
Voices of SIGCHI | March 2025
-
“Uh, This One?”: Leveraging Behavioral Signals for Detecting Confusion during Physical Tasks
Maia Stiber, Dan Bohus, Sean Andrist
Proceedings of the 26th International Conference on Multimodal Interaction | November 2024
-
“Is This It?”: Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Emily Bao, Eric Horvitz, Ann Paradiso
Companion Proceedings of the 26th International Conference on Multimodal Interaction | August 2024
-
Continual Learning about Objects in the Wild: An Interactive Approach
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Eric Horvitz
International Conference on Multimodal Interaction | November 2022
-
Lessons Learned in Designing AI for Autistic Adults
Andrew Begel, John Tang, Sean Andrist, Mike Barnett, Tony Carbary, Piali Choudhury, Ed Cutrell, Alberto Fung, Sasa Junuzovic, Daniel McDuff, Kael Rowan, Shibashankar Sahoo, Jennifer Frances Waldern, Jessica Wolk, Hui Zheng, Annuska Zolyomi
ASSETS 2020: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility | October 2020
-
Managing Stress: The Needs of Autistic Adults in Video Calling
Annuska Zolyomi, Andrew Begel, Jennifer Frances Waldern, John Tang, Mike Barnett, Ed Cutrell, Daniel McDuff, Sean Andrist, Meredith Ringel Morris
CSCW 2019 | November 2019
Best Paper Honorable Mention
-
Demonstrating a Framework for Rapid Development of Physically Situated Interactive Systems
Sean Andrist, Dan Bohus, Ashley Feniello
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Recognizing F-Formations in the Open World
Hooman Hedayati, Daniel Szafir, Sean Andrist
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters
Sean Andrist, Michael Gleicher, Bilge Mutlu
Proceedings of CHI 2017 | May 2017
-
Human-Agent Collaboration: Can an Agent be a Partner?
Rachel K. E. Bellamy, Sean Andrist, Timothy Bickmore, Elizabeth F. Churchill, Thomas Erickson
Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17) | May 2017
-
Using Gaze Patterns to Predict Task Intent in Collaboration
Chien-Ming Huang, Sean Andrist, Allison Sauppe, Bilge Mutlu
frontiers in Pyschology | July 2015, 第 6 卷(1049): pp. 1-12
-
Look Together: Analyzing Gaze Coordination with Epistemic Network Analysis
Sean Andrist, Wesley Collier, Michael Gleicher, Bilge Mutlu, David Shaffer
frontiers in Pyschology | July 2015, 第 6 卷(1016): pp. 1-15
-
Look Like Me: Matching Robot Personality via Gaze to Increase Motivation
Sean Andrist, Bilge Mutlu, Adriana Tapus
33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15) | April 2015
Best of CHI Honorable Mention Award
-
Gaze and Attention Management for Embodied Conversational Agents
Tomislav Pejsa, Sean Andrist, Michael Gleicher, Bilge Mutlu
ACM Transactions on Interactive Intelligent Systems (TiiS) | March 2015, 第 5 卷(1)
-
Effects of Culture on the Credibility of Robot Speech: A Comparison between English and Arabic
Sean Andrist, Micheline Ziadee, Halim Boukaram, Bilge Mutlu, Majd Sakr
Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ’15) | March 2015
-
A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and Perception
Sean Andrist, et al
February 2015
-
Conversational Gaze Aversion for Humanlike Robots
Sean Andrist, Xiang Zhi Tan, Michael Gleicher, Bilge Mutlu
HRI '14 Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction | March 2014
Best Paper Award Nominee
-
“Look Me in the Eyes”: A Survey of Eye and Gaze Animation for Virtual Agents and Artificial Systems
K. Ruhland, Sean Andrist, J. B. Badler, C. E. Peters, N. I. Badler, M. Gleicher, B. Mutlu, R. McDonnell
Eurographics 2014 – State of the Art Reports (EG ’14 STARs) | February 2014
-
Managing Chaos: Models of Turn-taking in Character-multichild Interactions
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
15th ACM on International conference on multimodal interaction | December 2013
-
Fun and Fair: Influencing Turn-taking in a Multi-party Game with a Virtual Agent
Sean Andrist, Iolanda Leite, Jill Lehman
IDC '13 Proceedings of the 12th International Conference on Interaction Design and Children | June 2013
-
Take or Wait? Learning Turn-Taking from Multiparty Data
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
AAAI Conference on Artificial Intelligence (Late-Breaking Developments) | April 2013
-
Conversational Gaze Aversion for Virtual Agents
Sean Andrist, Bilge Mutlu, Michael Gleicher
13th International Conference on Intelligent Virtual Agents (IVA ’13) | February 2013
-
Controllable Models of Gaze Behavior for Virtual Agents and Humanlike Robots
ICMI '13 - 15th ACM on International conference on multimodal interaction | January 2013
-
Rhetorical Robots: Making Robots More Effective Speakers Using Linguistic Cues of Expertise
Sean Andrist, Erin Spannan, Bilge Mutlu
8th ACM/IEEE International Conference on Human-Robot Interaction (HRI ’13) | January 2013
-
Designing Effective Gaze Mechanisms for Virtual Agents
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
CHI '12 - SIGCHI Conference on Human Factors in Computing Systems | November 2012
-
A Head-Eye Coordination Model for Animating Gaze Shifts of Virtual Characters
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
14th International Conference on Multimodal Interaction (ICMI ’12), 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction (Gaze-In ’12) | July 2012
Article No. 4
-
Decision and Coordination Strategies for RoboCup Rescue Agents
Maitreyi Nanjanath, Alexander J. Erlandson, Sean Andrist, Aravind Ragipindi, Abdul A. Mohammed, Ankur S. Sharma, Maria Gini
Second International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR ’10) | February 2010
-
-
-
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso, Nick Saw, Tim Schoonbeek, Maia Stiber
November 2025
-
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research – Extended Abstract
Dan Bohus, Sean Andrist, Nick Saw, Ann Paradiso, Ishani Chakraborty, Mahdi Rad
2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | March 2024, pp. 889-890
-
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Felipe Vieira Frujeri, Neel Joshi, Marc Pollefeys
ICCV 2023 | September 2023
-
-
-
A network analytic approach to gaze coordination during a collaborative task
Sean Andrist, A. R. Ruis, David Williamson Shaffer
Computers in Human Behavior | December 2018, 第 89 卷: pp. 339-348
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
-
-
Converting Spatial to Social: Using Persistent Homology to Understand Social Groups
Valerie K. Chen, Claire Liang, Julie A. Shah, Sean Andrist
Proceedings of the 27th International Conference on Multimodal Interaction | October 2025
-
New Initiatives at ICMI: Mentorship in Focus
Catharine Oertel, Laura Cabrera-Quiros, Hayley Hung, Mohammad Soleymani, Sean Andrist, Dimosthenis Kontogiorgos, Chirag Raman
Voices of SIGCHI | March 2025
-
“Uh, This One?”: Leveraging Behavioral Signals for Detecting Confusion during Physical Tasks
Maia Stiber, Dan Bohus, Sean Andrist
Proceedings of the 26th International Conference on Multimodal Interaction | November 2024
-
“Is This It?”: Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Emily Bao, Eric Horvitz, Ann Paradiso
Companion Proceedings of the 26th International Conference on Multimodal Interaction | August 2024
-
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research – Extended Abstract
Dan Bohus, Sean Andrist, Nick Saw, Ann Paradiso, Ishani Chakraborty, Mahdi Rad
2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | March 2024, pp. 889-890
-
Interaction-Shaping Robotics: Robots that Influence Interactions between Other Agents
Sarah Gillet, Marynel Vázquez, Sean Andrist, Iolanda Leite, S. Sebo
ACM Transactions on Human-Robot Interaction | February 2024
-
Symbiotic Society with Avatars (SSA): Beyond Space and Time
Hooman Hedayati, S. Seo, T. Kanda, Daniel J. Rea, Sean Andrist, Y. Nakano, H. Ishiguro
Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction | March 2023
-
Developing Mixed Reality Applications with Platform for Situated Intelligence
Sean Andrist, Dan Bohus, Ashley Feniello, Nick Saw
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | February 2022, pp. 48-50
-
Social cognitive systems in smart environments: Approaches for learning, reasoning, and adaptation
Amir Aly, Shashank Pathak, Sean Andrist, Armando Tacchella
Cognitive Systems Research | December 2019, 第 58 卷
-
A network analytic approach to gaze coordination during a collaborative task
Sean Andrist, A. R. Ruis, David Williamson Shaffer
Computers in Human Behavior | December 2018, 第 89 卷: pp. 339-348
-
Turn-Taking and Coordination in Human-Machine Interaction
Sean Andrist, Dan Bohus, Bilge Mutlu, David Schlangen
AI Magazine, Winter Issue | January 2016, 第 34 卷
-
Using Gaze Patterns to Predict Task Intent in Collaboration
Chien-Ming Huang, Sean Andrist, Allison Sauppe, Bilge Mutlu
frontiers in Pyschology | July 2015, 第 6 卷(1049): pp. 1-12
-
Look Together: Analyzing Gaze Coordination with Epistemic Network Analysis
Sean Andrist, Wesley Collier, Michael Gleicher, Bilge Mutlu, David Shaffer
frontiers in Pyschology | July 2015, 第 6 卷(1016): pp. 1-15
-
Gaze and Attention Management for Embodied Conversational Agents
Tomislav Pejsa, Sean Andrist, Michael Gleicher, Bilge Mutlu
ACM Transactions on Interactive Intelligent Systems (TiiS) | March 2015, 第 5 卷(1)
-
-
-
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso, Nick Saw, Tim Schoonbeek, Maia Stiber
November 2025
-
-
-
Platform for Situated Intelligence and OpenSense: A Tutorial on Building Multimodal Interactive Applications for Research
Sean Andrist, Dan Bohus, Zongjian Li, Mohammad Soleymani
October 2023
-
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Felipe Vieira Frujeri, Neel Joshi, Marc Pollefeys
ICCV 2023 | September 2023
-
Continual Learning about Objects in the Wild: An Interactive Approach
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Eric Horvitz
International Conference on Multimodal Interaction | November 2022
-
Lessons Learned in Designing AI for Autistic Adults
Andrew Begel, John Tang, Sean Andrist, Mike Barnett, Tony Carbary, Piali Choudhury, Ed Cutrell, Alberto Fung, Sasa Junuzovic, Daniel McDuff, Kael Rowan, Shibashankar Sahoo, Jennifer Frances Waldern, Jessica Wolk, Hui Zheng, Annuska Zolyomi
ASSETS 2020: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility | October 2020
-
Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence
AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction: Trust & Explainability in Artificial Intelligence for Human-Robot Interaction | October 2020
-
REFORM: Recognizing F-formations for Social Robots
Hooman Hedayati, Annika Muehlbradt, Daniel J. Szafir, Sean Andrist
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020) | August 2020
PDF PDF Publication Publication Publication Publication Publication 项目
-
Now, Over Here: Leveraging Extended Attentional Capabilities in Human-Robot Interaction.
Xiang Zhi Tan, Sean Andrist, Dan Bohus, Eric Horvitz
ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20 Late-breaking Reports) | March 2020
-
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) | February 2020
-
Managing Stress: The Needs of Autistic Adults in Video Calling
Annuska Zolyomi, Andrew Begel, Jennifer Frances Waldern, John Tang, Mike Barnett, Ed Cutrell, Daniel McDuff, Sean Andrist, Meredith Ringel Morris
CSCW 2019 | November 2019
Best Paper Honorable Mention
-
Demonstrating a Framework for Rapid Development of Physically Situated Interactive Systems
Sean Andrist, Dan Bohus, Ashley Feniello
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Recognizing F-Formations in the Open World
Hooman Hedayati, Daniel Szafir, Sean Andrist
14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) | March 2019
-
Rapid Development of Multimodal Interactive Systems: A Demonstration of Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Mihai Jalobeanu
ICMI 2017 Proceedings of the 19th ACM International Conference on Multimodal Interaction | November 2017
-
A study in scene shaping: Adjusting F-formations in the wild
Dan Bohus, Sean Andrist, Eric Horvitz
2017 AAAI Fall Symposium: Natural Communication for Human-Robot Collaboration | September 2017
-
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
Sean Andrist, Dan Bohus, Ece Kamar, Eric Horvitz
ICSR 2017: Social Robotics | July 2017
-
Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters
Sean Andrist, Michael Gleicher, Bilge Mutlu
Proceedings of CHI 2017 | May 2017
-
Human-Agent Collaboration: Can an Agent be a Partner?
Rachel K. E. Bellamy, Sean Andrist, Timothy Bickmore, Elizabeth F. Churchill, Thomas Erickson
Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17) | May 2017
-
Are You Messing with Me? Querying About the Sincerity of Interactions in the Open World
Sean Andrist, Dan Bohus, Zhou Yu, Eric Horvitz
2016 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’16 Late-breaking Reports) | January 2016
-
Look Like Me: Matching Robot Personality via Gaze to Increase Motivation
Sean Andrist, Bilge Mutlu, Adriana Tapus
33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15) | April 2015
Best of CHI Honorable Mention Award
-
Effects of Culture on the Credibility of Robot Speech: A Comparison between English and Arabic
Sean Andrist, Micheline Ziadee, Halim Boukaram, Bilge Mutlu, Majd Sakr
Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ’15) | March 2015
-
Conversational Gaze Aversion for Humanlike Robots
Sean Andrist, Xiang Zhi Tan, Michael Gleicher, Bilge Mutlu
HRI '14 Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction | March 2014
Best Paper Award Nominee
-
“Look Me in the Eyes”: A Survey of Eye and Gaze Animation for Virtual Agents and Artificial Systems
K. Ruhland, Sean Andrist, J. B. Badler, C. E. Peters, N. I. Badler, M. Gleicher, B. Mutlu, R. McDonnell
Eurographics 2014 – State of the Art Reports (EG ’14 STARs) | February 2014
-
Managing Chaos: Models of Turn-taking in Character-multichild Interactions
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
15th ACM on International conference on multimodal interaction | December 2013
-
Fun and Fair: Influencing Turn-taking in a Multi-party Game with a Virtual Agent
Sean Andrist, Iolanda Leite, Jill Lehman
IDC '13 Proceedings of the 12th International Conference on Interaction Design and Children | June 2013
-
Take or Wait? Learning Turn-Taking from Multiparty Data
Iolanda Leite, Hannaneh Hajishirzi, Sean Andrist, Jill Lehman
AAAI Conference on Artificial Intelligence (Late-Breaking Developments) | April 2013
-
Conversational Gaze Aversion for Virtual Agents
Sean Andrist, Bilge Mutlu, Michael Gleicher
13th International Conference on Intelligent Virtual Agents (IVA ’13) | February 2013
-
Controllable Models of Gaze Behavior for Virtual Agents and Humanlike Robots
ICMI '13 - 15th ACM on International conference on multimodal interaction | January 2013
-
Rhetorical Robots: Making Robots More Effective Speakers Using Linguistic Cues of Expertise
Sean Andrist, Erin Spannan, Bilge Mutlu
8th ACM/IEEE International Conference on Human-Robot Interaction (HRI ’13) | January 2013
-
Designing Effective Gaze Mechanisms for Virtual Agents
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
CHI '12 - SIGCHI Conference on Human Factors in Computing Systems | November 2012
-
A Head-Eye Coordination Model for Animating Gaze Shifts of Virtual Characters
Sean Andrist, Tomislav Pejsa, Bilge Mutlu, Michael Gleicher
14th International Conference on Multimodal Interaction (ICMI ’12), 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction (Gaze-In ’12) | July 2012
Article No. 4
-
Decision and Coordination Strategies for RoboCup Rescue Agents
Maitreyi Nanjanath, Alexander J. Erlandson, Sean Andrist, Aravind Ragipindi, Abdul A. Mohammed, Ankur S. Sharma, Maria Gini
Second International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR ’10) | February 2010
-
-
-
Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Pat Sweeney, Anne Loomis Thompson, Eric Horvitz
MSR-TR-2021-2 | March 2021
作者：Microsoft
-
-
-
-
Gaze Mechanisms for Situated Interaction with Embodied Agents
May 2016
博士论文：University of Wisconsin - Madison | May 2016
-