Platform for Situated Intelligence Workshop

Platform for Situated Intelligence Workshop


The Platform for Situated Intelligence Workshop took place on April 27-28, 2021.

Platform for Situated Intelligence is an open-source framework from Microsoft Research that significantly accelerates development and research on multimodal, integrative-AI applications. This framework makes it easier to build applications that process multimodal streaming data (audio, video, depth, etc.), combine multiple (AI) technologies, and operate in the open world in real-time.

This two-day workshop included both high-level overviews and in-depth tutorials on various aspects of the framework, from the low-level infrastructure for working with multimodal, streaming data, to development and visualization tools, to the ecosystem of available components and integrations with third-party technologies such as ROS, Python, Microsoft Teams, Unity, etc. The first day of the workshop will focus on high-level overviews and community building, whereas the second day will focus more on tutorials on specific topics. See the agenda page for more information on what was included in the schedule.

Additional resources

For a quick, high-level overview of the Platform for Situated Intelligence framework, please see the video below, or this blog post. A detailed technical description of the framework is available in this paper. Additional resources are available on the project website and GitHub repository.

Organizing Committee

Dan Bohus
Sean Andrist
Noboru Kuno
Jinoos Safavian

Microsoft’s Event Code of Conduct

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. This includes virtual events Microsoft hosts and participates in, where we seek to create a respectful, friendly, and inclusive experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location.

We do not tolerate any behavior that is degrading to any gender, race, sexual orientation or disability, or any behavior that would violate Microsoft’s Anti-Harassment and Anti-Discrimination Policy, Equal Employment Opportunity Policy, or Standards of Business Conduct. In short, the entire experience must meet our culture standards. We encourage everyone to assist in creating a welcoming and safe environment. Please report any concerns, harassing behavior, or suspicious or disruptive activity. Microsoft reserves the right to ask attendees to leave at any time at its sole discretion.


*All times are displayed in Pacific Time (UTC -08:00).

Day 1 | April 27, 2021

9:00 AM–9:10 AM Welcome and Workshop overview | Video Dan Bohus and Sean Andrist
9:10 AM–9:25 AM Opening remarks | Video Eric Horvitz
9:25 AM–10:00 AM Platform for Situated Intelligence overview + Q&A | Video Dan Bohus
10:00 AM–10:45 AM Platform for Situated Intelligence: A coding tutorial + Q&A | Video Dan Bohus
10:45 AM–11:00 AM Break
11:00 AM–12:00 PM Multimodal data visualization and annotation with Platform for Situated Intelligence Studio + Q&A | Video Sean Andrist
12:00 PM-12:30 PM Lunch break
12:30 PM–1:30 PM Building an open-source community (panel discussion) | Video Sean Andrist, Dan Bohus, Daniel McDuff, Mohammad Soleymani, Kalin Stefanov, Zhi Tan, Willie Wilson

Day 2 | April 28, 2021

9:00 AM–9:05 AM Welcome back for day 2 | Video Sean Andrist
9:05 AM–9:50 AM Debugging and diagnostics + Q&A | Video Nick Saw
9:50 AM–10:30 AM Interoperating with Python + Q&A | Video Ashley Feniello
10:30 AM–10:45 AM Break
10:45 AM–11:20 AM Interoperating with ROS + Q&A | Video Ashley Feniello
11:20 AM–11:45 AM Interoperating with HoloLens 2 + Q&A | Video Dan Bohus
11:45 AM–12:15 PM Interoperating with Teams + Q&A | Video Sean Andrist
12:15 PM–12:45 PM Lunch break
12:45 PM–1:15 PM A toolkit for Situated Interactive applications | Video Sean Andrist
1:15 PM–1:30 PM Closing remarks | Video Dan Bohus and Sean Andrist