{"id":239291,"date":"2016-06-16T17:42:42","date_gmt":"2016-06-17T00:42:42","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=239291"},"modified":"2016-09-21T12:26:02","modified_gmt":"2016-09-21T19:26:02","slug":"acquisition-reconstruction-transmission-display-live-3d-content","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/acquisition-reconstruction-transmission-display-live-3d-content\/","title":{"rendered":"Acquisition, reconstruction, transmission, and display of live 3D content"},"content":{"rendered":"<p>We are developing a system for the acquisition, transmission and display of real-time 3D digital content. Our goal is to enable live, immersive 3D communications and entertainment experiences. Our strategy is to acquire the 3D signal within a cubical volume. We represent this signal using high resolution colored voxels, and have created algorithms for acquisition, encoding, decoding, streaming, and display of this voxel data designed specifically for modern massively data-parallel GPUs. This work evolved into project <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/holoportation-3\/\">Holoporation<\/a>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-239321 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/Qsmall.png\" alt=\"Qsmall\" width=\"582\" height=\"162\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/Qsmall.png 582w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/Qsmall-300x84.png 300w\" sizes=\"auto, (max-width: 582px) 100vw, 582px\" \/><\/p>\n<h2><\/h2>\n<h2>Project milestones<\/h2>\n<p><strong>2012 Q3:<\/strong> Synthetic Data Analysis for Immersive Telepresence System (Acquisition -> Surface-Reconstruction -> Rendering).<\/p>\n<p><strong>2012 Q4:<\/strong> Real Data for Immersive Telepresence System (collaboration with SBG Kent)<\/p>\n<p><strong>2013 Q1-2 (RIG 1): <\/strong>Real Time System for 3D Teleportation (8 RGBs, collaboration with <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/chazhang\/\" target=\"_blank\">Cha Zhang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/zhang\/\" target=\"_blank\">Zhengyou Zhang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<\/p>\n<ul>\n<li>\u00a0\u00a0\u00a0\u00a0Techfest 2013<\/li>\n<li>\u00a0\u00a0\u00a0\u00a0DC Techfair 2013 (organized by <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/petelee\/default.aspx\" target=\"_blank\">Peter Lee<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<\/li>\n<li>\u00a0\u00a0\u00a0\u00a0Seattle Science Fair 2013 (organized by <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/petelee\/default.aspx\" target=\"_blank\">Peter Lee<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)<\/li>\n<li>\u00a0\u00a0\u00a0\u00a0HPG paper:\u00a0 &#8220;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/cloop\/HPG2013.pdf\">Real-Time High-Resolution Sparse Voxelization with Application to Image-Based Modeling<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>,&#8221; <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/cloop\/HPG2013.mp4\">video<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li>\u00a0\u00a0\u00a0\u00a0Collaboration with Artist <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/news\/features\/lresearchart-120313.aspx\">James George for Studio 99<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n<p><strong>2013 Q4:<\/strong> Rig 1 + Voxel Streaming (3D content is remotely rendered)<\/p>\n<p><strong>2014 Q2 (RIG 2):<\/strong> 8 RGBs + 1 Kinect Depth (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/cloop\/Rig2.mp4\" target=\"_blank\">Video<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>) (collaboration with <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/pachou\/publications.aspx\" target=\"_blank\">Phil Chou<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> on derivation of &#8220;Bayesian Fusion&#8221;)<\/p>\n<ul>\n<li>\u00a0\u00a0\u00a0\u00a0Mini-cave demo in building 99.<\/li>\n<\/ul>\n<p><strong>2014 Q3 &#8211; 2015 Q2 (RIG 3):<\/strong> 4 Pods, (1RGB + 2Monos)\/Pod (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/cloop\/Rig3.mp4\" target=\"_blank\">Video<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>)\u00a0(<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/www.dtic.ua.es\/~sorts\/\" target=\"_blank\">Sergio Orts-Escalano<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> internship,\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/qincai\/\" target=\"_blank\">Qin Cai<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> joined the project)<\/p>\n<ul>\n<li>\u00a0\u00a0\u00a0\u00a0Demo to AR\/VR Displays: Mini-Cave, Oculus (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/cloop\/OculusDemo.mp4\" target=\"_blank\">Video<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>), <a href=\"https:\/\/www.microsoft.com\/microsoft-hololens\/en-us\" target=\"_self\">HoloLens<\/a><\/li>\n<li>\u00a0\u00a0\u00a0\u00a0Computer Vision Summit (in conjunction with\u00a0Techfest 2015)<\/li>\n<li>\u00a0\u00a0\u00a0\u00a0Transatlantic Telepresence call to MSRC TAB (Thanks to <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/apps\/search\/default.aspx?q=Jonathan+Taylor&sp=1\" target=\"_blank\">John Taylor<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> for remote assistance)<\/li>\n<\/ul>\n<p><strong>2015 Q3 &#8211; Q4 (RIG 4):<\/strong> 8 Pods, (1RGB + 2Monos)\/Pod. <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" href=\"http:\/\/research.microsoft.com\/en-us\/groups\/i3d\/\" target=\"_blank\">I3D group<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> effort for full body capture and &#8220;remote streaming&#8221; HoloLens rendering.<\/p>\n<ul>\n<li>\u00a0\u00a0\u00a0\u00a0DTR (Disruptive Technology Review) 2015 w\/ Microsoft Senior Leadership Team.<\/li>\n<li>\u00a0\u00a0\u00a0\u00a0First Time Two-Way Voxel Teleportation<\/li>\n<li>\u00a0\u00a0\u00a0 Project is renamed <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/holoportation-3\/\">&#8220;Holoportation&#8221;<\/a>.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>We are developing a system for the acquisition, transmission and display of real-time 3D digital content. Our goal is to enable live, immersive 3D communications and entertainment experiences. Our strategy is to acquire the 3D signal within a cubical volume. We represent this signal using high resolution colored voxels, and have created algorithms for acquisition, [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13562,13551],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-239291","msr-project","type-msr-project","status-publish","hentry","msr-research-area-computer-vision","msr-research-area-graphics-and-multimedia","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"2012-06-01","related-publications":[],"related-downloads":[],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/239291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/239291\/revisions"}],"predecessor-version":[{"id":294482,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/239291\/revisions\/294482"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=239291"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=239291"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=239291"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=239291"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=239291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}