Portrait of Andy Wilson

Andy Wilson

Principal Researcher/ Research Manager

About

I am a principal researcher at Microsoft Research. My work is focused on applying sensing techniques to enable new styles of human-computer interaction. Today that means multi-touch and gesture-based interfaces, display technologies, depth cameras and so-called “natural” interfaces. In 2002 I helped found the Surface Computing group at Microsoft. I manage the Natural Interaction Research group at Microsoft Research.

Before joining Microsoft, I obtained my BA at Cornell University, and MS and PhD at the MIT Media Laboratory. I grew up in a small town in Pennsylvania and now live in the Wallingford neighborhood of Seattle.

You can view videos of my work and collaborators’ work on my YouTube channel.

News
November 2015: I am the proud recipient of ACM ICMI’s Lasting Impact Award for my 2004 paper “TouchLight: an imaging touch screen and display for gesture-based interaction“.

At BUILD 2015 we introduced the RoomAlive Toolkit, an open source SDK that calibrates multiple Kinect and multple projectors to enable interactive projection mapping experiences like that of RoomAlive and IllumiRoom. Ben Lower and I introduce the Toolkit and describe some of its components in a Channel9 video available here. Source code is on GitHub here.

How I like to think about Kalman filters

Professional Activities
I’ve been active in the UIST, CHI, Tabletop (now Interactive Tabletops and Surfaces), ICMI, and Ubicomp academic communities, where I’ve have had the pleasure of serving on various program committees. I co-chaired IEEE Tabletop 2007, chaired ACM UIST 2009 and ACM SUI 2014. I tend to publish under the name “Andrew D. Wilson“.

Projects

Room2Room: Life-Size Telepresence in a Projected Augmented Reality Environment

Established: March 2, 2016

Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. We enable a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their virtual copy into the remote space at life-size scale. This creates an illusion of the remote person’s presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze).

Tablet and Stylus Interaction

Established: October 3, 2014

We explore grip and motion sensing to afford new techniques that leverage how users naturally manipulate tablet and stylus devices during pen-and-touch interaction. We can detect whether the user holds the pen in a writing grip or tucked between his fingers. We can distinguish bare-handed inputs, such as drag and pinch gestures, from touch gestures produced by the hand holding the pen, and we can sense which hand grips the tablet, and determine the screen's…

Dyadic Mano-a-Mano

Established: October 3, 2014

Mano-a-Mano is a unique spatial augmented reality system that combines dynamic projection mapping, multiple perspective views and device-less interaction to support face-to-face, or dyadic, interaction with 3D virtual objects. Its main advantage over more traditional AR approaches is users are able to interact with 3D virtual objects and each other without cumbersome devices that obstruct face to face interaction. ACM Digital Download Dyadic projected spatial augmented reality  

RoomAlive

Established: October 1, 2014

RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment.

RoomAlive Toolkit

Established: October 1, 2014

The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop new immersive augmented reality experiences similar to those of the IllumiRoom and RoomAlive research projects. The RoomAlive Toolkit is provided as open source under the MIT License. The code is available for download at GithHub: https://github.com/Kinect/RoomAliveToolkit.…

IllumiRoom: Peripheral Projected Illusions for Interactive Experiences

Established: January 4, 2013

IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.         Read the CHI 2013 Best-Paper: IllumiRoom: Peripheral Projected Illusions for Interactive Experiences (7.6 MB .pdf) IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to…

HoloDesk: Direct 3D Interactions with a Situated See-Through Display

Established: February 24, 2012

HoloDesk is a novel interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics.     Project Overview: HoloDesk is an interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics. A virtual image of a 3D scene is rendered through a half silvered…

LightSpace

Established: October 2, 2010

LightSpace combines elements of surface computing and augmented reality research to create a highly interactive space where any surface, and even the space between surfaces, is fully interactive. Our concept transforms the ideas of surface computing into the new realm of spatial computing. Instrumented with multiple depth cameras and projectors, LightSpace is a small room installation designed to explore a variety of interactions and computational strategies related to interactive displays and the space that they…

Publications

2017

2016

2015

2014

2013

2012

2011

2010

Pen + Touch = New Tools
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, Bill Buxton, in UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology, ACM, October 3, 2010, View abstract, Download PDF

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

The KidsRoom
Aaron F. Bobick, Stephen S. Intille, James W. Davis, Freedom Baird, Claudio S. Pinhanez, Lee W. Campbell, Yuri A. Ivanov, Arjan Schütte, Andy Wilson, in Communications of the ACM, ACM, March 1, 2000, View abstract, Download PDF

1999

1998

1997

1996

Smart Spaces
Ali Azarbayejani, Christopher Wren, Andy Wilson, Dave Becker, Nuria Oliver, Deb Roy, Flavia Sparacino, François Bérard, in SIGGRAPH '96 ACM SIGGRAPH 96 Visual Proceedings: The art and interdisciplinary programs of SIGGRAPH '96, ACM, August 8, 1996, View abstract, View external link

1995

Projects

Downloads

RoomAlive Toolkit sample calibration files

March 2017

The RoomAlive Toolkit calibrates multiple projectors and cameras to enable immersive, dynamic projection mapping experiences such as RoomAlive. This download includes sample calibration files which can be used with the RoomAlive Toolkit.

    Click the icon to access this download

  • Website