Portrait of Eyal Ofek

Eyal Ofek

Senior Researcher

I am a senior researcher in at Microsoft Research. My research interests include computer vision for human-computer interaction, Augmented Reality (AR)/Virtual Reality (VR), Haptics and  interactive projection mapping.
I obtained my PhD at the Hebrew University of Jerusalem, and founded a couple of companies in the area of computer graphics, including the successful Photon-Paint for the Amiga computer, and managing the software R&D at 3DV Systems, developing the world first active real-time depth cameras, later bought by Microsoft in 2008.
During 2004-2005 I was a researcher at Microsoft Research Asia (MSRA), working on issues such as video completion, reconstruction of hair from images, and camera based interaction, following by founding the Virtual Earth Research Lab, (A.K.A. Bing Mapping and Mobile Research Lab). The lab developed and shipped new innovations to Microsoft Virtual Earth and Bing, such as the first Street-View Site (2006), Image Privacy, automatic geo-positioning user…

Research Highlights

Augmented Reality/Virtual Reality

Augmented Reality/Virtual Reality

I am interesting in the way that AR & VR effect and being effected by the real environment, and ways in which AR &VR can influence new experiences, for example social interaction.

Haptics

Haptics

While visual display and 3D audio are progressing at a fast pase, Haptics is still and mostly untapped sense. Many times, due to the severe limitations of haptic rendering capability, there is a need to manipulate our senses to generate a convincing AR & VR experience of touch.

Interactive Projection Mapping

Interactive Projection Mapping

The use of projector enable large area augmentation that can be shared by multiple users.

Camera Based Interaction

Camera Based Interaction

Cameras are cheep and powerful sensors. I am looking in ways that we can leverage of cameras capabilities to compensate for the physical limitations of existing displays and input techniques.

Computer Vision

Computer Vision

Text detection, Analysis by Synthesis, 2D & 3D completion

Computer Graphics

Computer Graphics

Professional Service

Co-Chair

PC Member

Reviewer

  • ACM SIGGRAPH,
  • ACM SIGGRAPH Asia,
  • ACM SIGSPATIAL,
  • CVPR,
  • ICCV,
  • Pacific Graphics,
  • IEEE transcation on PAMI
  • CHI
  • UIST
  • ISMAR

Awards

  • Talpiot program member 1984
  • Charles Clor Scholarship, 1992
  • Best paper, UIST 2009
  • Best Paper, CHI 2013
  • Best Paper, CSCW 2016

Patents / Publication

PATENTS / PUBLICATION # YEAR OF PUBLICATION / ISSUED TITLE CITATIONS
9355268 2016 Managing access by applications to perceptual information
9298345 2016 Block View For Geographic Navigation
9171403 2016 Contour Completion For Augmenting Surface Reconstructions.
9167290 2015 City Scene Video Sharing On Digital Maps.
9146119 2015 Scrubbing Variable Content Paths
9123159 2015 Interactive Geo-Positioning Of Imagery
9110921 2015 Map Editing with Little User Input
9032042 2015 Audio Presentation Of Condensed Spatial Contextual Information.
8970691 2015 Removal Of Rayleigh Scattering From Images
9008859 2015 Navigation Instructions Using Low-Bandwidth Signaling
8971641 2015 Spatial Image Index And Associated Updating Functionality.
8970691 2015 Removal Of Rayleigh Scattering From Images.
2014/0267,228 2014 Mapping augmented reality experience to various Environments 0
8,831,380 2014 Viewing media in the context of street-level images 0
8,823,797 2014 Simulated video with extra viewpoints and enhanced resolution for traffic cameras 0
8,817,049 2014 Automated fitting of interior maps to general maps 0
8,773,424 2014 User interfaces for interacting with top-down maps of reconstructed 3-D scences 0
8,774,520 2014 Geo-relevance for images 0
8,769,396 2014 Calibration and annotation of video content 0
2014/0180,695 2014 GENERATION OF CONVERSATION TO ACHIEVE A GOAL 0
2014/0180,833 2014 Path Queries 0
2014/0165,037 2014 REUSABLE APPLICATION USER EXPERIENCE 0
2014/0129,394 2014 VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS 0
2014/0098,107 2014 Transitioning Between Top-Down Maps and Local Navigation of Reconstructed 3-D Scenes 0
8,694,383 2014 Path queries 0
8,688,368 2014 Image-based localization for addresses 0
8,675,018 2014 Electromechanical surface of rotational elements for motion compensation of a moving object 0
2014/0051,510 2014 IMMERSIVE DISPLAY WITH PERIPHERAL ILLUSIONS 0
8,645,230 2014 Virtual closet for storing and accessing virtual representations of items 0
8,640,020 2014 Adjustable and progressive mobile device street view 0
8,633,942 2014 View generation using interpolated values 0
8,624,902 2014 Transitioning between top-down maps and local navigation of reconstructed 3-D scenes 0
8,611,643 2013 Spatially registering user photographs 0
8,581,900 2013 Computing transitions between captured driving runs 0
8,577,170 2013 Shadow detection in a single image 0
2013/0287,307 2013 DATA DIFFERENCE GUIDED IMAGE CAPTURING 0
8,570,325 2013 Filter and surfacing virtual content in virtual worlds 0
8,560,515 2013 Automatic generation of markers based on social interaction 0
8,550,909 2013 Geographic data acquisition by user motivation 0
8,548,725 2013 Visual assessment of landmarks 0
8,549,627 2013 Detection of objectionable videos 0
2013/0246,512 2013 COGNITIVE AGENT 0
2013/0229,434 2013 VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES 0
2013/0212,138 2013 CURVE REPRESENTATIONS 0
8,503,794 2013 Data difference guided image capturing 0
8,494,566 2013 Hybrid mobile phone geopositioning 0
8,487,927 2013 Validating user generated three-dimensional models 0
8,488,040 2013 Mobile and server-side computational photography 0
2013/0173,653 2013 PATH COMPOSITION FOR PLANNING 0
2013/0156,220 2013 SELECTIVE SPATIAL AUDIO COMMUNICATION 0
8,463,289 2013 Depersonalizing location traces 0
2013/0142,384 2013 ENHANCED NAVIGATION THROUGH MULTI-SENSOR POSITIONING 0
8,453,060 2013 Panoramic ring user interface 0
8,447,136 2013 Viewing media in the context of street-level images 4
8,446,367 2013 Camera-based multi-touch mouse 0
8,447,105 2013 Data driven interpolation using geodesic affinity 0
8,442,716 2013 Identifying physical locations of entities 0
8,433,136 2013 Tagging video using character recognition and propagation 1
8,428,908 2013 Cognitive agent 0
2013/0071,016 2013 SHADOW DETECTION IN A SINGLE IMAGE 1
2013/0051,623 2013 Geo-Relevance for Images 2
2013/0050,472 2013 REMOVAL OF RAYLEIGH SCATTERING FROM IMAGES 0
2013/0009,954 2013 SPATIALLY REGISTERING USER PHOTOGRAPHS 0
2013/0006,517 2013 PROVIDING ROUTES THROUGH INFORMATION COLLECTION AND RETRIEVAL 3
2013/0006,953 2013 SPATIALLY ORGANIZED IMAGE COLLECTIONS ON MOBILE DEVICES 0
2012/0331,009 2012 MAP EDITING WITH LITTLE USER INPUT 0
2012/0331,093 2012 AUDIO PRESENTATION OF CONDENSED SPATIAL CONTEXTUAL INFORMATION 0
8,340,416 2012 Techniques for robust color transfer 0
2012/0322,458 2012 DEPERSONALIZING LOCATION TRACES 0
2012/0315,992 2012 GEOGRAPHIC DATA ACQUISITION BY USER MOTIVATION 2
8,326,048 2012 Geo-relevance for images 2
8,319,796 2012 Generating a texture from multiple images 0
8,311,287 2012 Validation and correction of map data using oblique images 0
2012/0274,642 2012 AUTOMATED FITTING OF INTERIOR MAPS TO GENERAL MAPS 0
8,301,996 2012 Annotating images with instructions 0
8,295,589 2012 Spatially registering user photographs 0
8,296,194 2012 Method, medium, and system for ranking dishes at eating establishments 0
2012/0262,552 2012 City scene video sharing on digital maps 0
2012/0239,513 2012 VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS 1
2012/0233,198 2012 CREATING VIRTUAL WORLDS FROM EXISTING DATA STRUCTURES 0
8,266,132 2012 Map aggregation 0
2012/0207,386 2012 Updating A Low Frame Rate Image Using A High Frame Rate Image Stream 0
2012/0210,171 2012 COGNITIVE AGENT 0
8,237,791 2012 Visualizing camera feeds on a map 3
2012/0155,778 2012 Spatial Image Index and Associated Updating Functionality 2
8,204,338 2012 Factoring repeated content within and among images 0
8,200,037 2012 Importance guided image transformation 0
2012/0141,046 2012 MAP WITH MEDIA ICONS 1
8,195,430 2012 Cognitive agent 0
2012/0133,665 2012 Generating A Texture From Multiple Images 0
8,189,925 2012 Geocoding by image matching 0
8,184,911 2012 Flexible matching with combinational similarity 1
2012/0109,942 2012 IDENTIFYING PHYSICAL LOCATIONS OF ENTITIES 1
2012/0084,323 2012 GEOGRAPHIC TEXT SEARCH USING IMAGE-MINED DATA 7
2012/0072,302 2012 Data-Driven Item Value Estimation 0
2012/0062,748 2012 VISUALIZING VIDEO WITHIN EXISTING STILL IMAGES 0
8,125,493 2012 Generating a texture from multiple images 0
2012/0027,250 2012 DATA DIFFERENCE GUIDED IMAGE CAPTURING 0
8,103,445 2012 Dynamic map rendering as a function of a user parameter 17
2012/0016,586 2012 VISUAL ASSESSMENT OF LANDMARKS 0
8,098,245 2012 Smart navigation for 3D maps 0
2011/0317,914 2011 TECHNIQUES FOR ROBUST COLOR TRANSFER 0
8,085,990 2011 Hybrid maps with embedded street-side images 3
2011/0312,374 2011 MOBILE AND SERVER-SIDE COMPUTATIONAL PHOTOGRAPHY 1
2011/0313,779 2011 AUGMENTATION AND CORRECTION OF LOCATION BASED DATA THROUGH USER FEEDBACK 1
2011/0299,766 2011 DATA DRIVEN INTERPOLATION USING GEODESIC AFFINITY 1
2011/0298,801 2011 VIEW GENERATION USING INTERPOLATED VALUES 0
2011/0298,928 2011 SIMULATED VIDEO WITH EXTRA VIEWPOINTS AND ENHANCED RESOLUTION FOR TRAFFIC CAMERAS 0
2011/0302,527 2011 ADJUSTABLE AND PROGRESSIVE MOBILE DEVICE STREET VIEW 0
2011/0294,515 2011 HYBRID MOBILE PHONE GEOPOSITIONING 2
2011/0286,660 2011 Spatially Registering User Photographs 3
8,060,302 2011 Visual assessment of landmarks 2
8,059,888 2011 Semi-automatic plane extrusion for 3D modeling 1
2011/0221,664 2011 VIEW NAVIGATION ON MOBILE DEVICE 3
8,009,178 2011 Augmenting images for panoramic display 4
2011/0200,252 2011 FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY 0
2011/0195,781 2011 MULTI-TOUCH MOUSE IN GAMING APPLICATIONS 1
2011/0187,716 2011 USER INTERFACES FOR INTERACTING WITH TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES 2
2011/0187,704 2011 GENERATING AND DISPLAYING TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES 0
2011/0187,723 2011 TRANSITIONING BETWEEN TOP-DOWN MAPS AND LOCAL NAVIGATION OF RECONSTRUCTED 3-D SCENES 0
7,991,283 2011 Geotagging photographs using annotations 4
2011/0173,565 2011 VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES 4
7,967,451 2011 Multi-directional image displaying device 7
2011/0137,741 2011 PATH QUERIES 0
7,957,596 2011 Flexible matching with combinational similarity 4
2011/0130,950 2011 TRAVEL DIRECTIONS WITH TRAVEL-TIME ESTIMATES 5
7,946,921 2011 Camera based orientation for mobile devices 1
7,941,271 2011 Displaying images related to a requested path 4
7,912,637 2011 Landmark-based routing 8
7,886,978 2011 Techniques for decoding images of barcodes 5
7,889,947 2011 Image completion 1
2011/0018,902 2011 HYBRID MAPS WITH EMBEDDED STREET-SIDE IMAGES 2
2010/0325,589 2010 BLOCK VIEW FOR GEOGRAPHIC NAVIGATION 4
2010/0316,300 2010 DETECTION OF OBJECTIONABLE VIDEOS 2
2010/0315,411 2010 COMPUTING TRANSITIONS BETWEEN CAPTURED DRIVING RUNS 1
7,852,315 2010 Camera and acceleration based interface for presentations 2
2010/0312,609 2010 Personalizing Selection of Advertisements Utilizing Digital Image Analysis 5
2010/0312,470 2010 Scrubbing Variable Content Paths 0
2010/0313,113 2010 Calibration and Annotation of Video Content 0
2010/0310,182 2010 GEOCODING BY IMAGE MATCHING 0
7,840,032 2010 Street-side maps and paths 7
7,831,089 2010 Modeling and texturing digital surface models in a mapping application 2
2010/0265,178 2010 CAMERA-BASED MULTI-TOUCH MOUSE 7
2010/0246,965 2010 TAGGING VIDEO USING CHARACTER RECOGNITION AND PROPAGATION 6
2010/0250,120 2010 MANAGING STORAGE AND DELIVERY OF NAVIGATION IMAGES 5
2010/0246,890 2010 DETECTION OF OBJECTS IN IMAGES 2
2010/0251,169 2010 AUTOMATIC GENERATION OF MARKERS BASED ON SOCIAL INTERACTION 2
2010/0245,361 2010 CONTEXT-BASED MANAGEMENT OF MARKERS 1
2010/0250,126 2010 VISUAL ASSESSMENT OF LANDMARKS 1
2010/0250,196 2010 COGNITIVE AGENT 1
2010/0245,344 2010 ANNOTATING OR EDITING THREE DIMENSIONAL SPACE 0
2010/0245,376 2010 FILTER AND SURFACING VIRTUAL CONTENT IN VIRTUAL WORLDS 0
2010/0250,366 2010 MERGE REAL-WORLD AND VIRTUAL MARKERS 0
7,805,066 2010 System for guided photography based on image capturing device rendered user recommendations according to embodiments 2
2010/0241,525 2010 IMMERSIVE VIRTUAL COMMERCE 3
2010/0241,514 2010 AUTOMATED PROVISION OF USER FEEDBACK 1
2010/0241,946 2010 ANNOTATING IMAGES WITH INSTRUCTIONS 0
2010/0235,076 2010 ESTIMATION OF FUEL CONSUMPTION FROM GPS TRAILS 0
2010/0235,078 2010 DRIVING DIRECTIONS WITH MAPS AND VIDEOS 0
2010/0235,356 2010 ORGANIZATION OF SPATIAL SENSOR DATA 0
2010/0225,665 2010 MAP AGGREGATION 0
2010/0218,094 2010 SECOND-PERSON AVATARS 0
7,777,648 2010 Mode information displayed in a mapping application 21
2010/0134,484 2010 THREE DIMENSIONAL JOURNALING ENVIRONMENT 2
2010/0118,025 2010 MODE INFORMATION DISPLAYED IN A MAPPING APPLICATION 55
2010/0085,350 2010 OBLIQUE DISPLAY WITH ADDITIONAL DETAIL 4
2010/0085,371 2010 OPTIMAL 2D TEXTURING FROM MULTIPLE IMAGES 0
2010/0080,551 2010 Geotagging Photographs Using Annotations 1
2010/0080,466 2010 Smart Navigation for 3D Maps 0
2010/0080,489 2010 Hybrid Interface for Interactively Registering Images to Digital Models 0
2010/0074,538 2010 VALIDATION AND CORRECTION OF MAP DATA USING OBLIQUE IMAGES 1
2009/0323,029 2009 MULTI-DIRECTIONAL IMAGE DISPLAYING DEVICE 2
2009/0285,482 2009 DETECTING TEXT USING STROKE WIDTH BASED TEXT DETECTION 4
2009/0284,527 2009 VALIDATING USER GENERATED THREE-DIMENSIONAL MODELS 4
2009/0237,510 2009 VISUALIZING CAMERA FEEDS ON A MAP 8
2009/0208,110 2009 FACTORING REPEATED CONTENT WITHIN AND AMONG IMAGES 2
2009/0190,857 2009 IMPORTANCE GUIDED IMAGE TRANSFORMATION 2
2009/0162,042 2009 GUIDED PHOTOGRAPHY BASED ON IMAGE CAPTURING DEVICE RENDERED USER RECOMMENDATIONS 3
7,548,659 2009 Video enhancement 20
2009/0141,966 2009 INTERACTIVE GEO-POSITIONING OF IMAGERY 6
2009/0110,327 2009 Semi-automatic plane extrusion for 3D modeling 0
2009/0092,277 2009 Geo-Relevance for Images 4
2009/0078,772 2009 Techniques for decoding images of barcodes 7
2009/0079,752 2009 Generating a texture from multiple images 2
2009/0058,855 2009 ELECTROMECHANICAL SURFACE OF ROTATIONAL ELEMENTS FOR MOTION COMPENSATION OF A MOVING OBJECT 0
7,477,236 2009 Remote control of on-screen interactions 4
2009/0002,394 2009 AUGMENTING IMAGES FOR PANORAMIC DISPLAY 9
2009/0003,702 2009 IMAGE COMPLETION 3
2008/0319,659 2008 LANDMARK-BASED ROUTING 16
2008/0319,660 2008 LANDMARK-BASED ROUTING 11
2008/0319,658 2008 LANDMARK-BASED ROUTING 7
2008/0273,795 2008 FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY 8
2008/0249,702 2008 Image-based localization for addresses 1
2008/0120,023 2008 DISPLAYING IMAGES RELATED TO A REQUESTED PATH 3
2008/0066,000 2008 PANORAMIC RING USER INTERFACE 2
2008/0050,011 2008 MODELING AND TEXTURING DIGITAL SURFACE MODELS IN A MAPPING APPLICATION 6
2008/0043,020 2008 USER INTERFACE FOR VIEWING STREET SIDE IMAGERY 29
2007/0273,558 2007 DYNAMIC MAP RENDERING AS A FUNCTION OF A USER PARAMETER 64
2007/0236,451 2007 Camera and Acceleration Based Interface for Presentations 3
2007/0210,937 2007 DYNAMIC RENDERING OF MAP INFORMATION 61
2007/0076,920 2007 STREET SIDE MAPS AND PATHS 27
2006/0294,538 2006 Inserting advertising content into video programming 14
2006/0287,083 2006 Camera based orientation for mobile devices 8
2006/0257,042 2006 Video enhancement 28
2006/0248,462 2006 Remote control of on-screen interactions 2