Portrait of Eyal Ofek

Eyal Ofek

Senior Researcher

About

I am a senior researcher at Microsoft Research (MSR).

My research interests include Computer Vision for Human-Computer Interaction and Augmented Reality.

For more Up-To-Date information, please see Eyal’s external homepage at: http://eyalofek.org/

 

 

 

Projects

Sparse Haptic Proxy

Established: May 4, 2017

  The user of Virtual Reality, may find himself in different virtual worlds, such as a spy game (middle) or a space simulator(right) , yet they all give him tangible feedback using the same physical geometry in the real world (left).  We propose a class of passive haptics that we call Sparse Haptic Proxy: a set of geometric primitives that simulate touch feedback in elaborate virtual reality scenes. Unlike previous passive haptics that replicate the…

Room2Room: Life-size telepresence in a projected augmented reality environment

Established: March 2, 2016

Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. We enable a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their virtual copy into the remote space at life-size scale. This creates an illusion of the remote person’s presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze).

GlassHands: Interaction Around Unmodified Mobile Devices Using Sunglasses

Established: January 1, 2016

We present a novel approach for extending the input space around unmodified mobile devices. Using built-in front facing cameras of unmodified handheld devices, GlassHands estimates hand poses and gestures through reflections in sunglasses, ski goggles or visors. Thereby, GlassHands creates an enlarged input space, rivaling input reach on large touch displays. We introduce the idea along with its technical concept and implementation. We demonstrate the feasibility and potential of our proposed approach in several application…

Peeking Template Matching for Depth Extension

Established: October 5, 2015

We propose a method that extends a given depth image into regions in 3D that are not visible from the point of view of the camera. The algorithm detects repeated 3D structures in the visible scene and suggests a set of 3D extension hypotheses, which are then combined together through a global 3D MRF discrete optimization. A collaboration with Simon Korman and Prof. Shai Avidan of Tel Aviv University.

Rich Haptic Feedback in Virtual Reality

Established: July 1, 2015

Publications Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proc. of ACM UIST 2016. PDF Video     Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proc. of ACM CHI 2016. 

RoomAlive

Established: October 1, 2014

RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment. ACM Digital Downloads RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units Dyadic projected spatial augmented reality

RoomAlive Toolkit

Established: October 1, 2014

The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop new immersive augmented reality experiences similar to those of the IllumiRoom and RoomAlive research projects. The RoomAlive Toolkit is provided as open source under the MIT License. The code is available for download at GithHub: https://github.com/Kinect/RoomAliveToolkit.…

IllumiRoom: Peripheral Projected Illusions for Interactive Experiences

Established: January 4, 2013

IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.         Read the CHI 2013 Best-Paper: IllumiRoom: Peripheral Projected Illusions for Interactive Experiences (7.6 MB .pdf) IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to…

Technology for Enhancing Conversations

Established: January 1, 2011

Face to face or remote conversations can sometimes be challenging, be it for the language barriers or because of lack of topics that may keep a conversation smoothly flowing. We look at technology solutions to help enhance conversations across a number of projects, including translating utterances seamlessly from one language to another and helping people find conversation topics of mutual interest based on prior context or existing context on the internet.

Publications

2017

2016

2015

2014

2013

2011

2010

2009

Mouse 2.0: Multi-touch meets the mouse
Nicolas Villar, Shahram Izadi, Dan Rosenfeld, Hrvoje Benko, John Helmes, Jonathan Westhues, Steve Hodges, Eyal Ofek, Alex Butler, Xiang Cao, Billy Chen, in Proceedings of UIST 2009, ACM Symposium on User Interface Software and Technology. p. 33-42. UIST 2009 Best Paper Award., Association for Computing Machinery, Inc., October 1, 2009, View abstract, Download PDF

2008

2007

2006

2005

2003

Depth Key
Ronen Gvili, Amir Kaplan, Eyal Ofek, Giora Yahav, in Stereoscopic Displays and Applications: The Engineering Reality of Virtual Reality 2003 (Proceedings of SPIE/IS&T Volume 5006),, January 1, 2003, View abstract

2002

1998

1997

Projects

Other

Professional Service

Co-Chair

PC Member

Reviewer

  • ACM SIGGRAPH,
  • ACM SIGGRAPH Asia,
  • ACM SIGSPATIAL,
  • CVPR,
  • ICCV,
  • Pacific Graphics,
  • IEEE transcation on PAMI
  • CHI
  • UIST
  • ISMAR

Awards

  • Talpiot program member 1984
  • Charles Clor Scholarship, 1992
  • Best paper, UIST 2009
  • Best Paper, CHI 2013
  • Best Paper, CSCW 2016

Patents / Publication

PATENTS / PUBLICATION #YEAR OF PUBLICATION / ISSUEDTITLECITATIONS
93552682016Managing access by applications to perceptual information
92983452016Block View For Geographic Navigation
91714032016Contour Completion For Augmenting Surface Reconstructions.
91672902015City Scene Video Sharing On Digital Maps.
91461192015Scrubbing Variable Content Paths
91231592015Interactive Geo-Positioning Of Imagery
91109212015Map Editing with Little User Input
90320422015Audio Presentation Of Condensed Spatial Contextual Information.
89706912015Removal Of Rayleigh Scattering From Images
90088592015Navigation Instructions Using Low-Bandwidth Signaling
89716412015Spatial Image Index And Associated Updating Functionality.
89706912015Removal Of Rayleigh Scattering From Images.
2014/0267,2282014Mapping augmented reality experience to various Environments0
8,831,3802014Viewing media in the context of street-level images0
8,823,7972014Simulated video with extra viewpoints and enhanced resolution for traffic cameras0
8,817,0492014Automated fitting of interior maps to general maps0
8,773,4242014User interfaces for interacting with top-down maps of reconstructed 3-D scences0
8,774,5202014Geo-relevance for images0
8,769,3962014Calibration and annotation of video content0
2014/0180,6952014GENERATION OF CONVERSATION TO ACHIEVE A GOAL0
2014/0180,8332014Path Queries0
2014/0165,0372014REUSABLE APPLICATION USER EXPERIENCE0
2014/0129,3942014VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS0
2014/0098,1072014Transitioning Between Top-Down Maps and Local Navigation of Reconstructed 3-D Scenes0
8,694,3832014Path queries0
8,688,3682014Image-based localization for addresses0
8,675,0182014Electromechanical surface of rotational elements for motion compensation of a moving object0
2014/0051,5102014IMMERSIVE DISPLAY WITH PERIPHERAL ILLUSIONS0
8,645,2302014Virtual closet for storing and accessing virtual representations of items0
8,640,0202014Adjustable and progressive mobile device street view0
8,633,9422014View generation using interpolated values0
8,624,9022014Transitioning between top-down maps and local navigation of reconstructed 3-D scenes0
8,611,6432013Spatially registering user photographs0
8,581,9002013Computing transitions between captured driving runs0
8,577,1702013Shadow detection in a single image0
2013/0287,3072013DATA DIFFERENCE GUIDED IMAGE CAPTURING0
8,570,3252013Filter and surfacing virtual content in virtual worlds0
8,560,5152013Automatic generation of markers based on social interaction0
8,550,9092013Geographic data acquisition by user motivation0
8,548,7252013Visual assessment of landmarks0
8,549,6272013Detection of objectionable videos0
2013/0246,5122013COGNITIVE AGENT0
2013/0229,4342013VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES0
2013/0212,1382013CURVE REPRESENTATIONS0
8,503,7942013Data difference guided image capturing0
8,494,5662013Hybrid mobile phone geopositioning0
8,487,9272013Validating user generated three-dimensional models0
8,488,0402013Mobile and server-side computational photography0
2013/0173,6532013PATH COMPOSITION FOR PLANNING0
2013/0156,2202013SELECTIVE SPATIAL AUDIO COMMUNICATION0
8,463,2892013Depersonalizing location traces0
2013/0142,3842013ENHANCED NAVIGATION THROUGH MULTI-SENSOR POSITIONING0
8,453,0602013Panoramic ring user interface0
8,447,1362013Viewing media in the context of street-level images4
8,446,3672013Camera-based multi-touch mouse0
8,447,1052013Data driven interpolation using geodesic affinity0
8,442,7162013Identifying physical locations of entities0
8,433,1362013Tagging video using character recognition and propagation1
8,428,9082013Cognitive agent0
2013/0071,0162013SHADOW DETECTION IN A SINGLE IMAGE1
2013/0051,6232013Geo-Relevance for Images2
2013/0050,4722013REMOVAL OF RAYLEIGH SCATTERING FROM IMAGES0
2013/0009,9542013SPATIALLY REGISTERING USER PHOTOGRAPHS0
2013/0006,5172013PROVIDING ROUTES THROUGH INFORMATION COLLECTION AND RETRIEVAL3
2013/0006,9532013SPATIALLY ORGANIZED IMAGE COLLECTIONS ON MOBILE DEVICES0
2012/0331,0092012MAP EDITING WITH LITTLE USER INPUT0
2012/0331,0932012AUDIO PRESENTATION OF CONDENSED SPATIAL CONTEXTUAL INFORMATION0
8,340,4162012Techniques for robust color transfer0
2012/0322,4582012DEPERSONALIZING LOCATION TRACES0
2012/0315,9922012GEOGRAPHIC DATA ACQUISITION BY USER MOTIVATION2
8,326,0482012Geo-relevance for images2
8,319,7962012Generating a texture from multiple images0
8,311,2872012Validation and correction of map data using oblique images0
2012/0274,6422012AUTOMATED FITTING OF INTERIOR MAPS TO GENERAL MAPS0
8,301,9962012Annotating images with instructions0
8,295,5892012Spatially registering user photographs0
8,296,1942012Method, medium, and system for ranking dishes at eating establishments0
2012/0262,5522012City scene video sharing on digital maps0
2012/0239,5132012VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS1
2012/0233,1982012CREATING VIRTUAL WORLDS FROM EXISTING DATA STRUCTURES0
8,266,1322012Map aggregation0
2012/0207,3862012Updating A Low Frame Rate Image Using A High Frame Rate Image Stream0
2012/0210,1712012COGNITIVE AGENT0
8,237,7912012Visualizing camera feeds on a map3
2012/0155,7782012Spatial Image Index and Associated Updating Functionality2
8,204,3382012Factoring repeated content within and among images0
8,200,0372012Importance guided image transformation0
2012/0141,0462012MAP WITH MEDIA ICONS1
8,195,4302012Cognitive agent0
2012/0133,6652012Generating A Texture From Multiple Images0
8,189,9252012Geocoding by image matching0
8,184,9112012Flexible matching with combinational similarity1
2012/0109,9422012IDENTIFYING PHYSICAL LOCATIONS OF ENTITIES1
2012/0084,3232012GEOGRAPHIC TEXT SEARCH USING IMAGE-MINED DATA7
2012/0072,3022012Data-Driven Item Value Estimation0
2012/0062,7482012VISUALIZING VIDEO WITHIN EXISTING STILL IMAGES0
8,125,4932012Generating a texture from multiple images0
2012/0027,2502012DATA DIFFERENCE GUIDED IMAGE CAPTURING0
8,103,4452012Dynamic map rendering as a function of a user parameter17
2012/0016,5862012VISUAL ASSESSMENT OF LANDMARKS0
8,098,2452012Smart navigation for 3D maps0
2011/0317,9142011TECHNIQUES FOR ROBUST COLOR TRANSFER0
8,085,9902011Hybrid maps with embedded street-side images3
2011/0312,3742011MOBILE AND SERVER-SIDE COMPUTATIONAL PHOTOGRAPHY1
2011/0313,7792011AUGMENTATION AND CORRECTION OF LOCATION BASED DATA THROUGH USER FEEDBACK1
2011/0299,7662011DATA DRIVEN INTERPOLATION USING GEODESIC AFFINITY1
2011/0298,8012011VIEW GENERATION USING INTERPOLATED VALUES0
2011/0298,9282011SIMULATED VIDEO WITH EXTRA VIEWPOINTS AND ENHANCED RESOLUTION FOR TRAFFIC CAMERAS0
2011/0302,5272011ADJUSTABLE AND PROGRESSIVE MOBILE DEVICE STREET VIEW0
2011/0294,5152011HYBRID MOBILE PHONE GEOPOSITIONING2
2011/0286,6602011Spatially Registering User Photographs3
8,060,3022011Visual assessment of landmarks2
8,059,8882011Semi-automatic plane extrusion for 3D modeling1
2011/0221,6642011VIEW NAVIGATION ON MOBILE DEVICE3
8,009,1782011Augmenting images for panoramic display4
2011/0200,2522011FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY0
2011/0195,7812011MULTI-TOUCH MOUSE IN GAMING APPLICATIONS1
2011/0187,7162011USER INTERFACES FOR INTERACTING WITH TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES2
2011/0187,7042011GENERATING AND DISPLAYING TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES0
2011/0187,7232011TRANSITIONING BETWEEN TOP-DOWN MAPS AND LOCAL NAVIGATION OF RECONSTRUCTED 3-D SCENES0
7,991,2832011Geotagging photographs using annotations4
2011/0173,5652011VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES4
7,967,4512011Multi-directional image displaying device7
2011/0137,7412011PATH QUERIES0
7,957,5962011Flexible matching with combinational similarity4
2011/0130,9502011TRAVEL DIRECTIONS WITH TRAVEL-TIME ESTIMATES5
7,946,9212011Camera based orientation for mobile devices1
7,941,2712011Displaying images related to a requested path4
7,912,6372011Landmark-based routing8
7,886,9782011Techniques for decoding images of barcodes5
7,889,9472011Image completion1
2011/0018,9022011HYBRID MAPS WITH EMBEDDED STREET-SIDE IMAGES2
2010/0325,5892010BLOCK VIEW FOR GEOGRAPHIC NAVIGATION4
2010/0316,3002010DETECTION OF OBJECTIONABLE VIDEOS2
2010/0315,4112010COMPUTING TRANSITIONS BETWEEN CAPTURED DRIVING RUNS1
7,852,3152010Camera and acceleration based interface for presentations2
2010/0312,6092010Personalizing Selection of Advertisements Utilizing Digital Image Analysis5
2010/0312,4702010Scrubbing Variable Content Paths0
2010/0313,1132010Calibration and Annotation of Video Content0
2010/0310,1822010GEOCODING BY IMAGE MATCHING0
7,840,0322010Street-side maps and paths7
7,831,0892010Modeling and texturing digital surface models in a mapping application2
2010/0265,1782010CAMERA-BASED MULTI-TOUCH MOUSE7
2010/0246,9652010TAGGING VIDEO USING CHARACTER RECOGNITION AND PROPAGATION6
2010/0250,1202010MANAGING STORAGE AND DELIVERY OF NAVIGATION IMAGES5
2010/0246,8902010DETECTION OF OBJECTS IN IMAGES2
2010/0251,1692010AUTOMATIC GENERATION OF MARKERS BASED ON SOCIAL INTERACTION2
2010/0245,3612010CONTEXT-BASED MANAGEMENT OF MARKERS1
2010/0250,1262010VISUAL ASSESSMENT OF LANDMARKS1
2010/0250,1962010COGNITIVE AGENT1
2010/0245,3442010ANNOTATING OR EDITING THREE DIMENSIONAL SPACE0
2010/0245,3762010FILTER AND SURFACING VIRTUAL CONTENT IN VIRTUAL WORLDS0
2010/0250,3662010MERGE REAL-WORLD AND VIRTUAL MARKERS0
7,805,0662010System for guided photography based on image capturing device rendered user recommendations according to embodiments2
2010/0241,5252010IMMERSIVE VIRTUAL COMMERCE3
2010/0241,5142010AUTOMATED PROVISION OF USER FEEDBACK1
2010/0241,9462010ANNOTATING IMAGES WITH INSTRUCTIONS0
2010/0235,0762010ESTIMATION OF FUEL CONSUMPTION FROM GPS TRAILS0
2010/0235,0782010DRIVING DIRECTIONS WITH MAPS AND VIDEOS0
2010/0235,3562010ORGANIZATION OF SPATIAL SENSOR DATA0
2010/0225,6652010MAP AGGREGATION0
2010/0218,0942010SECOND-PERSON AVATARS0
7,777,6482010Mode information displayed in a mapping application21
2010/0134,4842010THREE DIMENSIONAL JOURNALING ENVIRONMENT2
2010/0118,0252010MODE INFORMATION DISPLAYED IN A MAPPING APPLICATION55
2010/0085,3502010OBLIQUE DISPLAY WITH ADDITIONAL DETAIL4
2010/0085,3712010OPTIMAL 2D TEXTURING FROM MULTIPLE IMAGES0
2010/0080,5512010Geotagging Photographs Using Annotations1
2010/0080,4662010Smart Navigation for 3D Maps0
2010/0080,4892010Hybrid Interface for Interactively Registering Images to Digital Models0
2010/0074,5382010VALIDATION AND CORRECTION OF MAP DATA USING OBLIQUE IMAGES1
2009/0323,0292009MULTI-DIRECTIONAL IMAGE DISPLAYING DEVICE2
2009/0285,4822009DETECTING TEXT USING STROKE WIDTH BASED TEXT DETECTION4
2009/0284,5272009VALIDATING USER GENERATED THREE-DIMENSIONAL MODELS4
2009/0237,5102009VISUALIZING CAMERA FEEDS ON A MAP8
2009/0208,1102009FACTORING REPEATED CONTENT WITHIN AND AMONG IMAGES2
2009/0190,8572009IMPORTANCE GUIDED IMAGE TRANSFORMATION2
2009/0162,0422009GUIDED PHOTOGRAPHY BASED ON IMAGE CAPTURING DEVICE RENDERED USER RECOMMENDATIONS3
7,548,6592009Video enhancement20
2009/0141,9662009INTERACTIVE GEO-POSITIONING OF IMAGERY6
2009/0110,3272009Semi-automatic plane extrusion for 3D modeling0
2009/0092,2772009Geo-Relevance for Images4
2009/0078,7722009Techniques for decoding images of barcodes7
2009/0079,7522009Generating a texture from multiple images2
2009/0058,8552009ELECTROMECHANICAL SURFACE OF ROTATIONAL ELEMENTS FOR MOTION COMPENSATION OF A MOVING OBJECT0
7,477,2362009Remote control of on-screen interactions4
2009/0002,3942009AUGMENTING IMAGES FOR PANORAMIC DISPLAY9
2009/0003,7022009IMAGE COMPLETION3
2008/0319,6592008LANDMARK-BASED ROUTING16
2008/0319,6602008LANDMARK-BASED ROUTING11
2008/0319,6582008LANDMARK-BASED ROUTING7
2008/0273,7952008FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY8
2008/0249,7022008Image-based localization for addresses1
2008/0120,0232008DISPLAYING IMAGES RELATED TO A REQUESTED PATH3
2008/0066,0002008PANORAMIC RING USER INTERFACE2
2008/0050,0112008MODELING AND TEXTURING DIGITAL SURFACE MODELS IN A MAPPING APPLICATION6
2008/0043,0202008USER INTERFACE FOR VIEWING STREET SIDE IMAGERY29
2007/0273,5582007DYNAMIC MAP RENDERING AS A FUNCTION OF A USER PARAMETER64
2007/0236,4512007Camera and Acceleration Based Interface for Presentations3
2007/0210,9372007DYNAMIC RENDERING OF MAP INFORMATION61
2007/0076,9202007STREET SIDE MAPS AND PATHS27
2006/0294,5382006Inserting advertising content into video programming14
2006/0287,0832006Camera based orientation for mobile devices8
2006/0257,0422006Video enhancement28
2006/0248,4622006Remote control of on-screen interactions2