Portrait of Hrvoje Benko

Hrvoje Benko

Senior Researcher

About

I am a senior researcher in Natural Interaction Research group at Microsoft Research. My research interests include virtual and augmented reality, computational illumination, surface computing, new input form factors and devices, as well as touch and freehand gestural input. I am fascinated by the intersection point where the digital technology meets the curved, physical, 3D space we live in.

I have been active in the Human-Computer Interaction research community and I have had the pleasure of serving on various program committees (CHI, UIST, SIGGRAPH, ISMAR, ITS). I was the general chair for ACM UIST 2014 and the program chair for ACM UIST 2012. I am the Information Director for ACM TOCHI journal.

In 2010, I worked with Microsoft Hardware to turn one of our research projects into a product. We created Microsoft Touch Mouse which enables the user to use touch gestures on top of the mouse to control their Windows experience.

Prior to working at Microsoft, I received my PhD at Columbia University, working on augmented reality projects that combine immersive experiences with interactive tabletops.

Projects

Muscle-Computer Interfaces (muCIs)

Many human-computer interaction technologies are currently mediated by physical transducers such as mice, keyboards, pens, dials, and touch-sensitive surfaces. While these transducers have enabled powerful interaction paradigms and leverage our human expertise in interacting with physical objects, they tether computation…

Rich haptic feedback in Virtual Reality

Established: July 1, 2015

Publications Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proc. of ACM UIST 2016. PDF Video     Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal…

Room2Room: Life-size telepresence in a projected augmented reality environment

Established: March 2, 2016

Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. We enable a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their…

RoomAlive Toolkit

Established: October 1, 2014

The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop…

Tablet and Stylus Interaction

Established: October 3, 2014

We explore grip and motion sensing to afford new techniques that leverage how users naturally manipulate tablet and stylus devices during pen-and-touch interaction. We can detect whether the user holds the pen in a writing grip or tucked between his…

Dyadic Mano-a-Mano

Established: October 3, 2014

Mano-a-Mano is a unique spatial augmented reality system that combines dynamic projection mapping, multiple perspective views and device-less interaction to support face-to-face, or dyadic, interaction with 3D virtual objects. Its main advantage over more traditional AR approaches is users are…

RoomAlive

Established: October 1, 2014

RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and…

IllumiRoom: Peripheral Projected Illusions for Interactive Experiences

Established: January 4, 2013

IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.         Read the CHI 2013 Best-Paper: IllumiRoom:…

LightSpace

Established: October 2, 2010

LightSpace combines elements of surface computing and augmented reality research to create a highly interactive space where any surface, and even the space between surfaces, is fully interactive. Our concept transforms the ideas of surface computing into the new realm…

Publications

2016

2015

2014

2013

2012

2011

2010

Pen + Touch = New Tools
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, Bill Buxton, in UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology, ACM, October 3, 2010, View abstract, Download PDF

2009

Mouse 2.0: Multi-touch meets the mouse
Nicolas Villar, Shahram Izadi, Dan Rosenfeld, Hrvoje Benko, John Helmes, Jonathan Westhues, Steve Hodges, Eyal Ofek, Alex Butler, Xiang Cao, Billy Chen, in Proceedings of UIST 2009, ACM Symposium on User Interface Software and Technology. p. 33-42. UIST 2009 Best Paper Award., Association for Computing Machinery, Inc., October 1, 2009, View abstract, Download PDF

2008

2007

2006

2005

2004

2003

2001

Projects

Other

Conference Organization

  • Program Committee member for ACM SIGGRAPH 2015
  • Program Committee member for ACM CHI 2015
  • Demo Co-Chair for ACM ICMI 2015
  • General Chair ACM UIST 2014
  • Program Committee member for ACM CHI 2014
  • Program Committee member for ACM ISMAR 2013
  • Program Co-Chair for ACM UIST 2012
  • Program Committee member for ACM ISMAR 2012
  • Program Committee member for ACM CHI 2012
  • Program Committee member for ACM ITS 2011
  • Program Committee member for Video Showcase at ACM CHI 2011
  • Program Committee member for ACM UIST 2010
  • Program Committee member and Doctoral Symposium Co-Chair for ACM ITS 2010
  • Program Committee member for ACM ICMI/MLMI 2010
  • Program Committee member for IEEE 3DUI 2010
  • Webmaster for ACM UIST 2009
  • Program Committee Member for IEEE 3DUI 2009
  • Demo Co-Chair for ACM UIST 2008
  • Program Committee member and Publicity Chair for 2nd International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN ’08), January 8-10, 2008, Cancun, Mexico.
  • Publicity Co-Chair for The Sixth IEEE International Symposium on Mixed and Augmented Reality (ISMAR ’07), November 13-16, 2007, Nara, Japan.
  • Co-Chair for the First Annual Columbia Computer Science Student Research Symposium, December 8th 2006, Davis Auditorium, Columbia University, NY.
  • Student Volunteer Organizing Committee, IEEE and ACM ISAR 2001 (International Symposium on Augmented Reality). New York, NY. October 29-30, 2001.

Editor

  • Associate Editor of IEEE Computer Graphics and Applications
  • Guest Editor for the Special Issue of IEEE Computer Graphics and Applications on Interacting beyond the Screen. To be published in May/June 2014.
  • Guest Editor for the Special Issue of Computers and Graphics on Touching the 3rd Dimension. Published by Elsevier in Dec. 2012.

Peer Reviewer

  • ACM SIGCHI: 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015
  • ACM SIGGRAPH: 2003, 2008, 2009, 2010, 2011, 2012, 2014, 2015
  • ACM SIGGRAPH ASIA: 2012, 2013
  • ACM User Interface Software and Technology (UIST): 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013
  • ACM Interactive Tabletops and Surfaces (ITS): 2009, 2010, 2011, 2012, 2013
  • ToCHI Journal: 2009, 2012
  • IEEE’s Transactions on Visualization and Computer Graphics (TVGC): 2008, 2009, 2011
  • International Conference on Multimodal Interfaces (ICMI): 2010
  • IEEE Virtual Reality (VR): 2004, 2005, 2007
  • IEEE Symposium on 3D User Interfaces: 2007, 2008, 2009, 2010, 2011
  • IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR): 2003, 2004, 2005, 2010
  • IEEE International Symposium on Wearable Computing (ISWC): 2007
  • IEEE Visualization: 2003, 2006
  • Pervasive: 2005
  • SmartGraphics: 2004

Supervised Students & Interns

  • Tomislav Pejsa (Summer 2014) coadvised with E. Ofek, A. Wilson
  • Julian Kantor (Summer 2014) coadvised with E. Ofek, A. Wilson
  • Feng Zheng (Summer 2014) coadvised with E. Ofek, A. Wilson
  • Yan Wang (Spring 2014) coadvised with E. Ofek, A. Wilson
  • Jarrod Knibbe (Summer 2013)
  • Ravish Mehra (Summer 2013)
  • Brett Jones (Summer 2013) coadvised with E. Ofek, A. Wilson
  • Rajinder Sodhi (Summer 2013) coadvised with E. Ofek, A. Wilson
  • Michael Murdock (Summer 2013) coadvised with E. Ofek, A. Wilson
  • Felipe Bacim de Araujo e Silva (Summer 2012) coadvised with M. Sinclair
  • Brett Jones (Summer 2012)
  • Rajinder Sodhi (Summer 2011)
  • Chris Harisson (Spring 2011)
  • Roland Aigner (Fall 2010) coadvised with D. Wigdor
  • Ricardo Costa Jota (Summer 2010)
  • David Holman (Summer 2010)
  • Dustin Freeman (Spring 2009) coadvised with D. Wigdor
  • Miguel Nacenta (Summer 2008) coadvised with A. Wilson and P. Baudisch
  • Bjoern Hartmann (Summer 2008) coadvised with M. Morris and A. Wilson
  • Bhashinee Garg (Spring 2007) – Exploration of dual-sided multi-touch interactions on a handheld device
  • Ivor Baksa (Spring 2006) – Development of AR conversational agent
  • Shezan Baig (Spring 2004) – TabletPC AR system for archeological visualization
  • Erik Peterson (Spring 2004) – Interface develpment for situated AR multimedia
  • Shezan Baig (Fall 2003) – Adaptive meshing algorithm for large meshes
  • Sajid Sadi (Spring 2003) – P5 Glove gesture recognizer for SenseShapes
  • Zachariah Munoz (Fall 2002, Spring 2003) – Modeling of the Cathedral of St. John the Divine
  • Sajid Sadi (Fall 2002) – Control interface for SenseShapes

Student Volunteer

  • SIGCHI 2007
  • SIGMM 2004
  • ISAR 2001

Courses Taught

  • Introduction to Programming in Java – COMS W3101-03 (Spring 2005)

Teaching Assistant

  • Video Games Design and Technology – COMS W4995-01 (Spring 2003)
  • User Interface Design – COMS W4170 (Fall 2001)
  • Introduction to Computing in Java (Fall 1999)