Natural Interaction Research

Established: October 12, 2011

The Natural Interaction Research group aims to enrich and reimagine the human-computer interface. Our team explores a wide variety of interaction topics including sensing and display hardware, touch and stylus input, spatial and augmented reality, and user modeling.











Mouse 2.0: Multi-touch meets the mouse
Nicolas Villar, Shahram Izadi, Dan Rosenfeld, Hrvoje Benko, John Helmes, Jonathan Westhues, Steve Hodges, Eyal Ofek, Alex Butler, Xiang Cao, Billy Chen, in Proceedings of UIST 2009, ACM Symposium on User Interface Software and Technology. p. 33-42. UIST 2009 Best Paper Award., Association for Computing Machinery, Inc., October 1, 2009, View abstract, Download PDF



Link description

Wearable Multitouch Interaction


April 18, 2011


Andy Wilson, Chris Harrison, and Hrvoje Benko


Microsoft Research / Carnegie Mellon University


As we May Ink?

Established: January 6, 2015

As We May Ink? Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences Abstract This project sheds light on gaps and discrepancies between the experiences afforded by analog pens and their digital counterparts. Despite the long history (and…

Rich haptic feedback in Virtual Reality

Established: July 1, 2015

Publications Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proc. of ACM UIST 2016. PDF Video     Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal…

Lightwear: An Exploration in Wearable Light Therapy

Established: June 2, 2014

We explored the social acceptability and user experience of wearable form factors as a portable option for Bright Light Therapy (BLT). BLT remains the predominant therapy for Seasonal Affective Disorder despite a non-compliance rate of ~70% commonly attributed to the…


Established: June 1, 2015

DuoSkin is a fabrication process that enables anyone to create customized functional devices that can be attached directly on their skin. Using gold metal leaf, a material that is cheap, skin-friendly, and robust for everyday wear, we demonstrate three types…

Room2Room: Life-size telepresence in a projected augmented reality environment

Established: March 2, 2016

Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. We enable a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their…

Peeking Template Matching for Depth Extension

Established: October 5, 2015

We propose a method that extends a given depth image into regions in 3D that are not visible from the point of view of the camera. The algorithm detects repeated 3D structures in the visible scene and suggests a set…

RoomAlive Toolkit

Established: October 1, 2014

The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop…

Dyadic Mano-a-Mano

Established: October 3, 2014

Mano-a-Mano is a unique spatial augmented reality system that combines dynamic projection mapping, multiple perspective views and device-less interaction to support face-to-face, or dyadic, interaction with 3D virtual objects. Its main advantage over more traditional AR approaches is users are…


Established: October 1, 2014

RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and…

IllumiRoom: Peripheral Projected Illusions for Interactive Experiences

Established: January 4, 2013

IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.         Read the CHI 2013 Best-Paper: IllumiRoom:…


Established: October 2, 2010

LightSpace combines elements of surface computing and augmented reality research to create a highly interactive space where any surface, and even the space between surfaces, is fully interactive. Our concept transforms the ideas of surface computing into the new realm…


Scanning Inside the Box at SIGGRAPH

SIGGRAPH 2013, the 40th International Conference and Exhibition on Computer Graphics and Interactive Techniques, occurs in Anaheim, California, from July 21 to 25. Sponsored by the Association for Computing Machinery, the conference is expected to attract 20,000 professionals eager to…

July 2013

Microsoft Research Blog

Touch to Feel the Virtual World

Haptic technology, which simulates the sense of touch through tactile feedback mechanisms, has been described as “doing for the sense of touch what computer graphics does for vision.” Haptics are already common in devices such as smartphones, where touch sensations…

July 2013

Microsoft Research Blog

CHI 2013: an Immersive Event

Springtime in Paris this year sees the Association for Computing Machinery’s 31st Conference on Human Factors in Computing Systems (CHI) in full swing from April 27 through May 2, welcoming experts and students from more than 60 countries. A large…

April 2013

Microsoft Research Blog

Social Search Social Builds Bridges

By Janie Chang, Writer, Microsoft Research Two conferences, same region, consecutive weeks—sounds like a perfect opportunity for a social. That’s certainly what sprang to mind for Meredith Ringel Morris, researcher in the Natural Interaction Research Group at Microsoft Research Redmond…

February 2012

Microsoft Research Blog

Two Extremes of Touch Interaction

By Janie Chang, Writer, Microsoft Research Microsoft Research Redmond researchers Hrvoje Benko and Scott Saponas have been investigating the use of touch interaction in computing devices since the mid-’00s. Now, two sharply different yet related projects demonstrate novel approaches to…

October 2011

Microsoft Research Blog