Project Tokyo

Project Tokyo

Established: October 31, 2016

Publications

Groups

Overview

Project Tokyo aims to understand how to create a visual agent technology that is both useful and usable in the real world by focusing on how AI technology can help to augment people’s own capabilities. Specifically, how can agent technologies amplify existing skills and abilities to help people do more? Project Tokyo explores these questions through human-centric technical innovation, partnering closely with users. The project has worked closely with people who are blind and low vision to concretely anchor our exploration into their needs, interests, and abilities. As early adopters of agent technologies, we have found that blind and low-vision users are particularly well suited for this research.

Project Tokyo has inspired a wide variety of research: Ethnographic work with Paralympic athletes and spectators guided system concept and design. Work with a team of blind and low-vision users helped to refine interaction approaches. Recent work is exploring how such a system might support blind children building up their social skills. Not least, we are researching how to customize such AI systems through machine teaching techniques so that they are right for the user, regardless of disability label.

Theo, left, a 12-year-old boy who is blind, interacts with Microsoft senior research software development engineers Tim Regan, middle, and Martin Grayson, right, during user testing of the Project Tokyo system. Photo by Jonathan Banks.

Peter Bosher, left background, an audio engineer who is blind, checks out the latest iteration of the Project Tokyo system at Microsoft’s research lab in Cambridge, UK. Bosher wears a modified Microsoft HoloLens that streams images of its visual field to computer hardware for processing. A dashboard on the laptop screen shows the visual field. Microsoft researchers Cecily Morrison, right background, and left on the screen, and Martin Grayson, right on the screen, are visible to the HoloLens. Photo by Jonathan Banks

Peter Bosher, middle, an audio engineer who is blind who worked with the Project Tokyo team early in the design process, checks out the latest iteration of the system at Microsoft’s research lab in Cambridge, UK, with researchers Martin Grayson, left, and Cecily Morrison, right. Photo by Jonathan Banks.

People

Project members

Past members

External collaborators

Past collaborators

Blogs & Podcasts

Theo, left, a 12-year-old boy who is blind, interacts with Microsoft senior research software development engineers Tim Regan, middle, and Martin Grayson, right, during user testing of the Project Tokyo system.

Using AI, people who are blind are able to find familiar faces in a room

“All of a sudden we don’t have to say, ‘Hey you are blind and I just made this accessible to you.’ We say, ‘Hey, you are you and I have just built a system that works for you,’” she said. “I don’t need to know anything about you. I don’t need a label on you. I can make something that is right for you because I have a system that you can take and adapt to yourself.”

Microsoft Innovation Stories | January 28, 2020

Image of Ed Cutrell for the Microsoft Research Podcast

Inclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell

Dr. Ed Cutrell is a Principal Researcher in the Ability group at Microsoft Research and he’s convinced that great technology should be available to everyone. Working in the fields of Accessibility and Information and Communication Technologies for Development (aka ICT4D), his research has explored computing solutions for people across the resource and ability spectrum, both here and around the world.

Microsoft Research Podcast | September 4, 2019

Image of Cecily Morrison for the Microsoft Research Podcast

Empowering people with AI with Dr. Cecily Morrison

You never know how an incident in your own life might inspire a breakthrough in science, but Dr. Cecily Morrison, a researcher in the Human Computer Interaction group at Microsoft Research Cambridge, can attest to how even unexpected events can cause us to see things through a different – more inclusive – lens and, ultimately, give rise to innovations in research that impact everyone.

Microsoft Research Podcast | January 23, 2019