{"id":389792,"date":"2018-01-23T19:10:28","date_gmt":"2018-01-24T03:10:28","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=389792"},"modified":"2022-04-08T13:38:44","modified_gmt":"2022-04-08T20:38:44","slug":"platform-situated-intelligence","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/platform-situated-intelligence\/","title":{"rendered":"Platform for Situated Intelligence"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720.jpg\" class=\"attachment-full size-full\" alt=\"Situated Intelligence header\" style=\"object-position: 69% 50%\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720.jpg 1920w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720-300x113.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720-1024x384.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720-768x288.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720-1536x576.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/01\/PSI_header4_08-2020_1920x720-1600x600.jpg 1600w\" sizes=\"auto, (max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 align-self-center\">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 id=\"platform-for-situated-intelligence\" class=\"h2\">Platform for Situated Intelligence<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/psi\" target=\"_blank\" rel=\"noopener noreferrer\"><u>Platform for Situated Intelligence<\/u><span class=\"sr-only\"> (opens in new tab)<\/span><\/a> is an open-source framework that significantly accelerates development and research on multimodal, integrative-AI applications. The framework was designed to alleviate the engineering challenges that arise when developing systems and applications that process multimodal streaming sensor data (such as audio, video, depth, etc.) and need to leverage and coordinate a variety of component AI technologies. Examples include multimodal interactive systems such as social robots and embodied conversational agents, systems for ambient intelligence and smart spaces, applications based on small devices that work with streaming sensor data, etc.<\/p>\n\n\n\n<p>The framework is cross platform, built on .NET standard, and leverages the affordances of a modern, managed programming language, while addressing the needs of multimodal, integrative-AI applications. The framework provides:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>infrastructure<\/strong> for working with multimodal, temporally streaming data;<\/li><li><strong>tools<\/strong> for multimodal data visualization, annotation, and processing;<\/li><li><strong>components<\/strong> that encapsulate various sensors, processing technologies.<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"Gfs808q8S0I\" data-poster=\"https:\/\/img.youtube.com\/vi\/Gfs808q8S0I\/maxresdefault.jpg\"><iframe aria-hidden=\"true\" tabindex=\"-1\" title=\"Platform for Situated Intelligence Overview\" width=\"500\" height=\"281\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/Gfs808q8S0I?feature=oembed&rel=0&enablejsapi=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div>\n<\/div><\/figure>\n\n\n\n<p>To learn more about Platform for Situated Intelligence, please see a high-level overview provided in <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/platform-for-situated-intelligence-an-open-source-framework-for-multimodal-integrative-ai\/\"><u>this blog post<\/u><\/a>. An in-depth description of various aspects of the framework is available in <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/arxiv.org\/abs\/2103.15975\" target=\"_blank\" rel=\"noopener noreferrer\">this technical report<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. To get started using the framework and to find in-depth documentation, please see the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/psi\" target=\"_blank\" rel=\"noopener noreferrer\"><u>GitHub repository<\/u><span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. We continue to evolve the framework with novel capabilities, and we invite contributions across the board: from simply using it and filing issues or providing bug fixes, to writing and releasing new components, to developing and contributing new features.<\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>The Platform for Situated Intelligence project focuses on constructing an open, extensible platform to enable the fast development of multimodal, integrative-AI systems.<\/p>\n","protected":false},"featured_media":686187,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13545,13554,13560],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-389792","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-human-language-technologies","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[480885,605784,714808,727057,1031103,1043688],"related-downloads":[],"related-videos":[745852,745861,1134901,1147700],"related-groups":[144633,396845,1084857,1105932],"related-events":[726709,687360],"related-opportunities":[],"related-posts":[478827,688410,712588,965166,994098,1027704],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Sean Andrist","user_id":36443,"people_section":"Current Team Members","alias":"sandrist"},{"type":"user_nicename","display_name":"Dan Bohus","user_id":31581,"people_section":"Current Team Members","alias":"dbohus"},{"type":"user_nicename","display_name":"Ashley Feniello","user_id":31116,"people_section":"Current Team Members","alias":"ashleyf"},{"type":"user_nicename","display_name":"Eric Horvitz","user_id":32033,"people_section":"Collaborators","alias":"horvitz"},{"type":"user_nicename","display_name":"Lev Nachmanson","user_id":32653,"people_section":"Collaborators","alias":"levnach"},{"type":"user_nicename","display_name":"Pat Sweeney","user_id":36999,"people_section":"Collaborators","alias":"patricsw"}],"msr_research_lab":[199565],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/389792","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":46,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/389792\/revisions"}],"predecessor-version":[{"id":833890,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/389792\/revisions\/833890"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/686187"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=389792"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=389792"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=389792"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=389792"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=389792"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}