{"id":295553,"date":"2020-03-04T08:04:13","date_gmt":"2020-01-29T17:00:09","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=295553"},"modified":"2024-07-08T11:32:27","modified_gmt":"2024-07-08T18:32:27","slug":"project-tokyo","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-tokyo\/","title":{"rendered":"Project Tokyo"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720.jpg\" class=\"attachment-full size-full\" alt=\"Theo, right, a 12-year-old boy who is blind, interacts with Microsoft senior research software development engineers Tim Regan, middle, and Martin Grayson, left, during user testing of the Project Tokyo system.\" style=\"\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720.jpg 1920w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720-300x113.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720-1024x384.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720-768x288.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720-1536x576.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/ProjectTokyo_1920x720-1600x600.jpg 1600w\" sizes=\"auto, (max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 align-self-center\">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"project-tokyo\">Project Tokyo<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>Project Tokyo aims to understand how to create a visual agent technology that is both useful and usable in the real world by focusing on how AI technology can help to augment people\u2019s own capabilities. Specifically, how can agent technologies amplify existing skills and abilities to help people do more? Project Tokyo explores these questions through human-centric technical innovation, partnering closely with users. The project has worked closely with people who are blind and low vision to concretely anchor our exploration into their needs, interests, and abilities. As early adopters of agent technologies, we have found that blind and low-vision users are particularly well suited for this research.<\/p>\n\n\n\n<p>Project Tokyo has inspired a wide variety of research: Ethnographic work with Paralympic athletes and spectators guided system concept and design. Work with a team of blind and low-vision users helped to refine interaction approaches. Recent work is exploring how such a system might support blind children building up their social skills. Not least, we are researching how to customize such AI systems through machine teaching techniques so that they are right for the user, regardless of disability label.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3614-1024x683.jpg\" alt=\"Theo, left, a 12-year-old boy who is blind, interacts with Microsoft senior research software development engineers Tim Regan, middle, and Martin Grayson, right, during user testing of the Project Tokyo system. Photo by Jonathan Banks.\" class=\"wp-image-633471\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3614-1024x683.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3614-300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3614-768x512.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3614.jpg 1182w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"682\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_4610-1024x682.jpg\" alt=\"Peter Bosher, left background, an audio engineer who is blind, checks out the latest iteration of the Project Tokyo system at Microsoft\u2019s research lab in Cambridge, UK. Bosher wears a modified Microsoft HoloLens that streams images of its visual field to computer hardware for processing. A dashboard on the laptop screen shows the visual field. Microsoft researchers Cecily Morrison, right background, and left on the screen, and Martin Grayson, right on the screen, are visible to the HoloLens. Photo by Jonathan Banks\" class=\"wp-image-633465\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_4610-1024x682.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_4610-300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_4610-768x511.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_4610.jpg 1184w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3945-1024x683.jpg\" alt=\"Peter Bosher, middle, an audio engineer who is blind who worked with the Project Tokyo team early in the design process, checks out the latest iteration of the system at Microsoft\u2019s research lab in Cambridge, UK, with researchers Martin Grayson, left, and Cecily Morrison, right. Photo by Jonathan Banks.\" class=\"wp-image-633468\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3945-1024x683.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3945-300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3945-768x512.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/10\/Project-Tokyo_3945.jpg 1182w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n\n\n<h3 class=\"wp-block-heading\" id=\"past-interns\">Past interns<\/h3>\n\n\n\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.leestearns.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Lee Stearns<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/arthurcarabott.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Arthur Carabott<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.ruixueliu.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ruixue Liu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/stephangarbin.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Stephan Garbin<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/mi.eng.cam.ac.uk\/~tr346\/\" target=\"_blank\" rel=\"noopener noreferrer\">Thomas Roddick<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.bennettc.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Cynthia Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/harshadha-balasubramanian-446946136\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener noreferrer\">Harshadha Balasubramanian<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/biomedia.doc.ic.ac.uk\/person\/daniel-coelho-de-castro\/\" target=\"_blank\" rel=\"noopener noreferrer\">Daniel \u200bCoelho de Castro\u200b<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.researchgate.net\/profile\/Vasiliki_Kladouchou\" target=\"_blank\" rel=\"noopener noreferrer\">Vasiliki Kladouchou<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/geertroumen.nl\/portfolio\/index.php?page=welcome\" target=\"_blank\" rel=\"noopener noreferrer\">Geert Roumen<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/kailasharul\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener noreferrer\">Kai Arulkumaran<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/elenafalomo.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Elena Falomo<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/naesseth.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Christian A. Naesseth<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.robots.ox.ac.uk\/~daniela\/\" target=\"_blank\" rel=\"noopener noreferrer\">Daniela Massiceti<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/sorig\" target=\"_blank\" rel=\"noopener noreferrer\">Esben S\u00f8rig<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"phd-students\">PhD students<\/h3>\n\n\n\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/luisazintgraf.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Luisa Zintgraf<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/sorig\" target=\"_blank\" rel=\"noopener noreferrer\">Esben S\u00f8rig<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><br><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sarahbennett4\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener noreferrer\">Sarah Bennett<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Project Tokyo aims to understand how to create a visual agent technology that is both useful and usable in the real world by focusing on how AI technology can help to augment people\u2019s own capabilities.<\/p>\n","protected":false},"featured_media":633282,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13562,13545,13554],"msr-locale":[268875],"msr-impact-theme":[261667],"msr-pillar":[],"class_list":["post-295553","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-computer-vision","msr-research-area-human-language-technologies","msr-research-area-human-computer-interaction","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[633447,853434,774382,774328,757495,752956,717130,691599,638919,431538,633438,633429,593947,593941,574524,488048,463683],"related-downloads":[],"related-videos":[748942],"related-groups":[283244,371909],"related-events":[641895],"related-opportunities":[],"related-posts":[562767,598893,657531,747019,823765,965166,994098],"related-articles":[],"tab-content":[{"id":0,"name":"Past collaborators","content":"<h3>Past Interns<\/h3>\r\n<a href=\"http:\/\/www.leestearns.com\/\" target=\"_blank\" rel=\"noopener\">Lee Stearns<\/a>\r\n<a href=\"https:\/\/arthurcarabott.com\/\" target=\"_blank\" rel=\"noopener\">Arthur Carabott<\/a>\r\n<a href=\"https:\/\/www.ruixueliu.com\/\" target=\"_blank\" rel=\"noopener\">Ruixue Liu<\/a>\r\n<a href=\"http:\/\/stephangarbin.com\/\" target=\"_blank\" rel=\"noopener\">Stephan Garbin<\/a>\r\n<a href=\"http:\/\/mi.eng.cam.ac.uk\/~tr346\/\" target=\"_blank\" rel=\"noopener\">Thomas Roddick<\/a>\r\n<a href=\"https:\/\/www.bennettc.com\/\" target=\"_blank\" rel=\"noopener\">Cynthia Bennett<\/a>\r\n<a href=\"https:\/\/www.linkedin.com\/in\/harshadha-balasubramanian-446946136\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener\">Harshadha Balasubramanian<\/a>\r\n<a href=\"https:\/\/biomedia.doc.ic.ac.uk\/person\/daniel-coelho-de-castro\/\" target=\"_blank\" rel=\"noopener\">Daniel \u200bCoelho de Castro\u200b<\/a>\r\n<a href=\"https:\/\/www.researchgate.net\/profile\/Vasiliki_Kladouchou\" target=\"_blank\" rel=\"noopener\">Vasiliki Kladouchou<\/a>\r\n<a href=\"https:\/\/geertroumen.nl\/portfolio\/index.php?page=welcome\" target=\"_blank\" rel=\"noopener\">Geert Roumen<\/a>\r\n<a href=\"https:\/\/www.linkedin.com\/in\/kailasharul\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener\">Kai Arulkumaran<\/a>\r\n<a href=\"https:\/\/elenafalomo.com\/\" target=\"_blank\" rel=\"noopener\">Elena Falomo<\/a>\r\n<a href=\"https:\/\/naesseth.github.io\/\" target=\"_blank\" rel=\"noopener\">Christian A. Naesseth<\/a>\r\n<a href=\"http:\/\/www.robots.ox.ac.uk\/~daniela\/\" target=\"_blank\" rel=\"noopener\">Daniela Massiceti<\/a>\r\n<a href=\"https:\/\/github.com\/sorig\" target=\"_blank\" rel=\"noopener\">Esben S\u00f8rig<\/a>\r\n<h3>PhD Students<\/h3>\r\n<a href=\"https:\/\/luisazintgraf.com\/\" target=\"_blank\" rel=\"noopener\">Luisa Zintgraf<\/a>\r\n<a href=\"https:\/\/github.com\/sorig\" target=\"_blank\" rel=\"noopener\">Esben S\u00f8rig<\/a>\r\n<a href=\"https:\/\/www.linkedin.com\/in\/sarahbennett4\/?originalSubdomain=uk\" target=\"_blank\" rel=\"noopener\">Sarah Bennett<\/a>"}],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Ed Cutrell","user_id":31490,"people_section":"Project members","alias":"cutrell"},{"type":"user_nicename","display_name":"Martin Grayson","user_id":32893,"people_section":"Project members","alias":"mgrayson"},{"type":"user_nicename","display_name":"Camilla Longden","user_id":36311,"people_section":"Project members","alias":"calongde"},{"type":"user_nicename","display_name":"Cecily Morrison","user_id":31356,"people_section":"Project members","alias":"cecilym"},{"type":"user_nicename","display_name":"Katja Hofmann","user_id":32468,"people_section":"Past members","alias":"kahofman"},{"type":"user_nicename","display_name":"Advait Sarkar","user_id":37146,"people_section":"Past members","alias":"advait"},{"type":"user_nicename","display_name":"David Sweeney","user_id":31553,"people_section":"Past members","alias":"dasweene"},{"type":"guest","display_name":"Alex Taylor","user_id":633402,"people_section":"Past members","alias":""},{"type":"user_nicename","display_name":"Anja Thieme","user_id":35948,"people_section":"Past members","alias":"anthie"},{"type":"guest","display_name":"Sebastian Tschiatschek","user_id":633534,"people_section":"Past members","alias":""},{"type":"guest","display_name":"Peter Bosher","user_id":636300,"people_section":"External collaborators","alias":""},{"type":"guest","display_name":"Rebecca Fiebrink","user_id":434226,"people_section":"External collaborators","alias":""},{"type":"guest","display_name":"Ewa Luger","user_id":633411,"people_section":"External collaborators","alias":""},{"type":"guest","display_name":"Linda Pring","user_id":633543,"people_section":"External collaborators","alias":""},{"type":"guest","display_name":"Shimon Whiteson","user_id":633408,"people_section":"External collaborators","alias":""}],"msr_research_lab":[199561],"msr_impact_theme":["Empowerment"],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/295553","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":23,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/295553\/revisions"}],"predecessor-version":[{"id":1109727,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/295553\/revisions\/1109727"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/633282"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=295553"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=295553"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=295553"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=295553"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=295553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}