Portrait of Andy Wilson

Andy Wilson

Principal Researcher/ Research Manager


I am a principal researcher at Microsoft Research. My work is focused on applying sensing techniques to enable new styles of human-computer interaction. Today that means multi-touch and gesture-based interfaces, display technologies, depth cameras and so-called "natural" interfaces. In 2002 I helped found the Surface Computing group at Microsoft. I manage the Natural Interaction Research group at Microsoft Research.

Before joining Microsoft, I obtained my BA at Cornell University, and MS and PhD at the MIT Media Laboratory. I grew up in a small town in Pennsylvania and now live in the Wallingford neighborhood of Seattle.

You can view videos of my work and collaborators' work on my YouTube channel.

November 2015: I am the proud recipient of ACM ICMI's Lasting Impact Award for my 2004 paper "TouchLight: an imaging touch screen and display for gesture-based interaction".

At BUILD 2015 we introduced the RoomAlive Toolkit, an open source SDK that calibrates multiple Kinect and multple projectors to enable interactive projection mapping experiences like that of RoomAlive and IllumiRoom. Ben Lower and I introduce the Toolkit and describe some of its components in a Channel9 video available here. Source code is on GitHub here.

How I like to think about Kalman filters

Professional Activities
I've been active in the UIST, CHI, Tabletop (now Interactive Tabletops and Surfaces), ICMI, and Ubicomp academic communities, where I've have had the pleasure of serving on various program committees. I co-chaired IEEE Tabletop 2007, chaired ACM UIST 2009 and ACM SUI 2014. I tend to publish under the name "Andrew D. Wilson".


Sparse Haptic Proxy

Established: May 4, 2017

  The user of Virtual Reality, may find himself in different virtual worlds, such as a spy game (middle) or a space simulator(right) , yet they all give him tangible feedback using the same physical geometry in the real world (left). We propose a class of passive haptics that we call Sparse Haptic Proxy: a set of geometric primitives that simulate touch feedback in elaborate virtual reality scenes. Unlike previous passive haptics that replicate the…

Room2Room: Life-Size Telepresence in a Projected Augmented Reality Environment

Established: March 2, 2016

Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. We enable a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their virtual copy into the remote space at life-size scale. This creates an illusion of the remote person’s presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze).

Rich Haptic Feedback in Virtual Reality

Established: July 1, 2015

Publications Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proc. of ACM UIST 2016. PDF Video     Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proc. of ACM CHI 2016. 

Dyadic Mano-a-Mano

Established: October 3, 2014

Mano-a-Mano is a unique spatial augmented reality system that combines dynamic projection mapping, multiple perspective views and device-less interaction to support face-to-face, or dyadic, interaction with 3D virtual objects. Its main advantage over more traditional AR approaches is users are able to interact with 3D virtual objects and each other without cumbersome devices that obstruct face to face interaction. ACM Digital Download Dyadic projected spatial augmented reality  


RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment.

RoomAlive Toolkit

Established: October 1, 2014

The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop new immersive augmented reality experiences similar to those of the IllumiRoom and RoomAlive research projects. The RoomAlive Toolkit is provided as open source under the MIT License. The code is available for download at GithHub: https://github.com/Kinect/RoomAliveToolkit.…

IllumiRoom: Peripheral Projected Illusions for Interactive Experiences

Established: January 4, 2013

IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.         Read the CHI 2013 Best-Paper: IllumiRoom: Peripheral Projected Illusions for Interactive Experiences (7.6 MB .pdf) IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to…

HoloDesk: Direct 3D Interactions with a Situated See-Through Display

Established: February 24, 2012

HoloDesk is a novel interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics.     Project Overview: HoloDesk is an interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics. A virtual image of a 3D scene is rendered through a half silvered…


Established: October 2, 2010

LightSpace combines elements of surface computing and augmented reality research to create a highly interactive space where any surface, and even the space between surfaces, is fully interactive. Our concept transforms the ideas of surface computing into the new realm of spatial computing. Instrumented with multiple depth cameras and projectors, LightSpace is a small room installation designed to explore a variety of interactions and computational strategies related to interactive displays and the space that they…










Pen + Touch = New Tools
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, Bill Buxton, in UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology, ACM, October 3, 2010, View abstract, Download PDF











The KidsRoom
Aaron F. Bobick, Stephen S. Intille, James W. Davis, Freedom Baird, Claudio S. Pinhanez, Lee W. Campbell, Yuri A. Ivanov, Arjan Schütte, Andy Wilson, in Communications of the ACM, ACM, March 1, 2000, View abstract, Download PDF





Smart Spaces
Ali Azarbayejani, Christopher Wren, Andy Wilson, Dave Becker, Nuria Oliver, Deb Roy, Flavia Sparacino, François Bérard, in SIGGRAPH '96 ACM SIGGRAPH 96 Visual Proceedings: The art and interdisciplinary programs of SIGGRAPH '96, ACM, August 8, 1996, View abstract, View external link




RoomAlive Toolkit sample calibration files

March 2017

The RoomAlive Toolkit calibrates multiple projectors and cameras to enable immersive, dynamic projection mapping experiences such as RoomAlive. This download includes sample calibration files which can be used with the RoomAlive Toolkit.

    Click the icon to access this download

  • Website