About
I am a senior researcher in at Microsoft Research. My research interests include computer vision for human-computer interaction, Augmented Reality (AR)/Virtual Reality (VR), Haptics and  interactive projection mapping.
I am on the editorial board of IEEE Computer Graphics and Applications, co-chaired the 19th ACM SIGSPATIAL 2011, and on program committee for several leading conferences. I Held a visiting lecture position at the school of computer science, Interdisciplinary Center, Herzelia, Israel.
I obtained my PhD at the Hebrew University of Jerusalem, and founded a couple of companies in the area of computer graphics, including the successful Photon-Paint for the Amiga computer, and managing the software R&D at 3DV Systems, developing the world first active real-time depth cameras, later bought by Microsoft in 2008.
During 2004-2005 I was a researcher at Microsoft Research Asia (MSRA), working on issues such as video completion, reconstruction of hair from images, and camera based interaction, following by…
Research Highlights

Augmented Reality/Virtual Reality
I am interesting in the way that AR & VR effect and being effected by the real environment, and ways in which AR &VR can influence new experiences, for example social interaction.

Haptics
While visual display and 3D audio are progressing at a fast pase, Haptics is still and mostly untapped sense. Many times, due to the severe limitations of haptic rendering capability, there is a need to manipulate our senses to generate a convincing AR & VR experience of touch.

Interactive Projection Mapping
The use of projector enable large area augmentation that can be shared by multiple users.

Camera Based Interaction
Cameras are cheep and powerful sensors. I am looking in ways that we can leverage of cameras capabilities to compensate for the physical limitations of existing displays and input techniques.

Computer Vision
Text detection, Analysis by Synthesis, 2D & 3D completion
Projects
Publications
Videos
Other
Professional Service
Co-Chair
PC Member
- ISMAR
- CVPR
- ACM SIGSPATIAL
- Pacific Graphics
- ISMAR 2015
- ISS 2018
Reviewer
- ACM SIGGRAPH,
- ACM SIGGRAPH Asia,
- ACM SIGSPATIAL,
- CVPR,
- ICCV,
- Pacific Graphics,
- IEEE transcation on PAMI
- CHI
- UIST
- ISMAR
Awards
- Talpiot program member 1984
- Charles Clor Scholarship, 1992
- Best paper, UIST 2009
- Best Paper, CHI 2013
- Best Paper, CSCW 2016
Patents / Publication
PATENTS / PUBLICATION # | YEAR OF PUBLICATION / ISSUED | TITLE | CITATIONS |
---|---|---|---|
9355268 | 2016 | Managing access by applications to perceptual information | |
9298345 | 2016 | Block View For Geographic Navigation | |
9171403 | 2016 | Contour Completion For Augmenting Surface Reconstructions. | |
9167290 | 2015 | City Scene Video Sharing On Digital Maps. | |
9146119 | 2015 | Scrubbing Variable Content Paths | |
9123159 | 2015 | Interactive Geo-Positioning Of Imagery | |
9110921 | 2015 | Map Editing with Little User Input | |
9032042 | 2015 | Audio Presentation Of Condensed Spatial Contextual Information. | |
8970691 | 2015 | Removal Of Rayleigh Scattering From Images | |
9008859 | 2015 | Navigation Instructions Using Low-Bandwidth Signaling | |
8971641 | 2015 | Spatial Image Index And Associated Updating Functionality. | |
8970691 | 2015 | Removal Of Rayleigh Scattering From Images. | |
2014/0267,228 | 2014 | Mapping augmented reality experience to various Environments | 0 |
8,831,380 | 2014 | Viewing media in the context of street-level images | 0 |
8,823,797 | 2014 | Simulated video with extra viewpoints and enhanced resolution for traffic cameras | 0 |
8,817,049 | 2014 | Automated fitting of interior maps to general maps | 0 |
8,773,424 | 2014 | User interfaces for interacting with top-down maps of reconstructed 3-D scences | 0 |
8,774,520 | 2014 | Geo-relevance for images | 0 |
8,769,396 | 2014 | Calibration and annotation of video content | 0 |
2014/0180,695 | 2014 | GENERATION OF CONVERSATION TO ACHIEVE A GOAL | 0 |
2014/0180,833 | 2014 | Path Queries | 0 |
2014/0165,037 | 2014 | REUSABLE APPLICATION USER EXPERIENCE | 0 |
2014/0129,394 | 2014 | VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS | 0 |
2014/0098,107 | 2014 | Transitioning Between Top-Down Maps and Local Navigation of Reconstructed 3-D Scenes | 0 |
8,694,383 | 2014 | Path queries | 0 |
8,688,368 | 2014 | Image-based localization for addresses | 0 |
8,675,018 | 2014 | Electromechanical surface of rotational elements for motion compensation of a moving object | 0 |
2014/0051,510 | 2014 | IMMERSIVE DISPLAY WITH PERIPHERAL ILLUSIONS | 0 |
8,645,230 | 2014 | Virtual closet for storing and accessing virtual representations of items | 0 |
8,640,020 | 2014 | Adjustable and progressive mobile device street view | 0 |
8,633,942 | 2014 | View generation using interpolated values | 0 |
8,624,902 | 2014 | Transitioning between top-down maps and local navigation of reconstructed 3-D scenes | 0 |
8,611,643 | 2013 | Spatially registering user photographs | 0 |
8,581,900 | 2013 | Computing transitions between captured driving runs | 0 |
8,577,170 | 2013 | Shadow detection in a single image | 0 |
2013/0287,307 | 2013 | DATA DIFFERENCE GUIDED IMAGE CAPTURING | 0 |
8,570,325 | 2013 | Filter and surfacing virtual content in virtual worlds | 0 |
8,560,515 | 2013 | Automatic generation of markers based on social interaction | 0 |
8,550,909 | 2013 | Geographic data acquisition by user motivation | 0 |
8,548,725 | 2013 | Visual assessment of landmarks | 0 |
8,549,627 | 2013 | Detection of objectionable videos | 0 |
2013/0246,512 | 2013 | COGNITIVE AGENT | 0 |
2013/0229,434 | 2013 | VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES | 0 |
2013/0212,138 | 2013 | CURVE REPRESENTATIONS | 0 |
8,503,794 | 2013 | Data difference guided image capturing | 0 |
8,494,566 | 2013 | Hybrid mobile phone geopositioning | 0 |
8,487,927 | 2013 | Validating user generated three-dimensional models | 0 |
8,488,040 | 2013 | Mobile and server-side computational photography | 0 |
2013/0173,653 | 2013 | PATH COMPOSITION FOR PLANNING | 0 |
2013/0156,220 | 2013 | SELECTIVE SPATIAL AUDIO COMMUNICATION | 0 |
8,463,289 | 2013 | Depersonalizing location traces | 0 |
2013/0142,384 | 2013 | ENHANCED NAVIGATION THROUGH MULTI-SENSOR POSITIONING | 0 |
8,453,060 | 2013 | Panoramic ring user interface | 0 |
8,447,136 | 2013 | Viewing media in the context of street-level images | 4 |
8,446,367 | 2013 | Camera-based multi-touch mouse | 0 |
8,447,105 | 2013 | Data driven interpolation using geodesic affinity | 0 |
8,442,716 | 2013 | Identifying physical locations of entities | 0 |
8,433,136 | 2013 | Tagging video using character recognition and propagation | 1 |
8,428,908 | 2013 | Cognitive agent | 0 |
2013/0071,016 | 2013 | SHADOW DETECTION IN A SINGLE IMAGE | 1 |
2013/0051,623 | 2013 | Geo-Relevance for Images | 2 |
2013/0050,472 | 2013 | REMOVAL OF RAYLEIGH SCATTERING FROM IMAGES | 0 |
2013/0009,954 | 2013 | SPATIALLY REGISTERING USER PHOTOGRAPHS | 0 |
2013/0006,517 | 2013 | PROVIDING ROUTES THROUGH INFORMATION COLLECTION AND RETRIEVAL | 3 |
2013/0006,953 | 2013 | SPATIALLY ORGANIZED IMAGE COLLECTIONS ON MOBILE DEVICES | 0 |
2012/0331,009 | 2012 | MAP EDITING WITH LITTLE USER INPUT | 0 |
2012/0331,093 | 2012 | AUDIO PRESENTATION OF CONDENSED SPATIAL CONTEXTUAL INFORMATION | 0 |
8,340,416 | 2012 | Techniques for robust color transfer | 0 |
2012/0322,458 | 2012 | DEPERSONALIZING LOCATION TRACES | 0 |
2012/0315,992 | 2012 | GEOGRAPHIC DATA ACQUISITION BY USER MOTIVATION | 2 |
8,326,048 | 2012 | Geo-relevance for images | 2 |
8,319,796 | 2012 | Generating a texture from multiple images | 0 |
8,311,287 | 2012 | Validation and correction of map data using oblique images | 0 |
2012/0274,642 | 2012 | AUTOMATED FITTING OF INTERIOR MAPS TO GENERAL MAPS | 0 |
8,301,996 | 2012 | Annotating images with instructions | 0 |
8,295,589 | 2012 | Spatially registering user photographs | 0 |
8,296,194 | 2012 | Method, medium, and system for ranking dishes at eating establishments | 0 |
2012/0262,552 | 2012 | City scene video sharing on digital maps | 0 |
2012/0239,513 | 2012 | VIRTUAL CLOSET FOR STORING AND ACCESSING VIRTUAL REPRESENTATIONS OF ITEMS | 1 |
2012/0233,198 | 2012 | CREATING VIRTUAL WORLDS FROM EXISTING DATA STRUCTURES | 0 |
8,266,132 | 2012 | Map aggregation | 0 |
2012/0207,386 | 2012 | Updating A Low Frame Rate Image Using A High Frame Rate Image Stream | 0 |
2012/0210,171 | 2012 | COGNITIVE AGENT | 0 |
8,237,791 | 2012 | Visualizing camera feeds on a map | 3 |
2012/0155,778 | 2012 | Spatial Image Index and Associated Updating Functionality | 2 |
8,204,338 | 2012 | Factoring repeated content within and among images | 0 |
8,200,037 | 2012 | Importance guided image transformation | 0 |
2012/0141,046 | 2012 | MAP WITH MEDIA ICONS | 1 |
8,195,430 | 2012 | Cognitive agent | 0 |
2012/0133,665 | 2012 | Generating A Texture From Multiple Images | 0 |
8,189,925 | 2012 | Geocoding by image matching | 0 |
8,184,911 | 2012 | Flexible matching with combinational similarity | 1 |
2012/0109,942 | 2012 | IDENTIFYING PHYSICAL LOCATIONS OF ENTITIES | 1 |
2012/0084,323 | 2012 | GEOGRAPHIC TEXT SEARCH USING IMAGE-MINED DATA | 7 |
2012/0072,302 | 2012 | Data-Driven Item Value Estimation | 0 |
2012/0062,748 | 2012 | VISUALIZING VIDEO WITHIN EXISTING STILL IMAGES | 0 |
8,125,493 | 2012 | Generating a texture from multiple images | 0 |
2012/0027,250 | 2012 | DATA DIFFERENCE GUIDED IMAGE CAPTURING | 0 |
8,103,445 | 2012 | Dynamic map rendering as a function of a user parameter | 17 |
2012/0016,586 | 2012 | VISUAL ASSESSMENT OF LANDMARKS | 0 |
8,098,245 | 2012 | Smart navigation for 3D maps | 0 |
2011/0317,914 | 2011 | TECHNIQUES FOR ROBUST COLOR TRANSFER | 0 |
8,085,990 | 2011 | Hybrid maps with embedded street-side images | 3 |
2011/0312,374 | 2011 | MOBILE AND SERVER-SIDE COMPUTATIONAL PHOTOGRAPHY | 1 |
2011/0313,779 | 2011 | AUGMENTATION AND CORRECTION OF LOCATION BASED DATA THROUGH USER FEEDBACK | 1 |
2011/0299,766 | 2011 | DATA DRIVEN INTERPOLATION USING GEODESIC AFFINITY | 1 |
2011/0298,801 | 2011 | VIEW GENERATION USING INTERPOLATED VALUES | 0 |
2011/0298,928 | 2011 | SIMULATED VIDEO WITH EXTRA VIEWPOINTS AND ENHANCED RESOLUTION FOR TRAFFIC CAMERAS | 0 |
2011/0302,527 | 2011 | ADJUSTABLE AND PROGRESSIVE MOBILE DEVICE STREET VIEW | 0 |
2011/0294,515 | 2011 | HYBRID MOBILE PHONE GEOPOSITIONING | 2 |
2011/0286,660 | 2011 | Spatially Registering User Photographs | 3 |
8,060,302 | 2011 | Visual assessment of landmarks | 2 |
8,059,888 | 2011 | Semi-automatic plane extrusion for 3D modeling | 1 |
2011/0221,664 | 2011 | VIEW NAVIGATION ON MOBILE DEVICE | 3 |
8,009,178 | 2011 | Augmenting images for panoramic display | 4 |
2011/0200,252 | 2011 | FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY | 0 |
2011/0195,781 | 2011 | MULTI-TOUCH MOUSE IN GAMING APPLICATIONS | 1 |
2011/0187,716 | 2011 | USER INTERFACES FOR INTERACTING WITH TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES | 2 |
2011/0187,704 | 2011 | GENERATING AND DISPLAYING TOP-DOWN MAPS OF RECONSTRUCTED 3-D SCENES | 0 |
2011/0187,723 | 2011 | TRANSITIONING BETWEEN TOP-DOWN MAPS AND LOCAL NAVIGATION OF RECONSTRUCTED 3-D SCENES | 0 |
7,991,283 | 2011 | Geotagging photographs using annotations | 4 |
2011/0173,565 | 2011 | VIEWING MEDIA IN THE CONTEXT OF STREET-LEVEL IMAGES | 4 |
7,967,451 | 2011 | Multi-directional image displaying device | 7 |
2011/0137,741 | 2011 | PATH QUERIES | 0 |
7,957,596 | 2011 | Flexible matching with combinational similarity | 4 |
2011/0130,950 | 2011 | TRAVEL DIRECTIONS WITH TRAVEL-TIME ESTIMATES | 5 |
7,946,921 | 2011 | Camera based orientation for mobile devices | 1 |
7,941,271 | 2011 | Displaying images related to a requested path | 4 |
7,912,637 | 2011 | Landmark-based routing | 8 |
7,886,978 | 2011 | Techniques for decoding images of barcodes | 5 |
7,889,947 | 2011 | Image completion | 1 |
2011/0018,902 | 2011 | HYBRID MAPS WITH EMBEDDED STREET-SIDE IMAGES | 2 |
2010/0325,589 | 2010 | BLOCK VIEW FOR GEOGRAPHIC NAVIGATION | 4 |
2010/0316,300 | 2010 | DETECTION OF OBJECTIONABLE VIDEOS | 2 |
2010/0315,411 | 2010 | COMPUTING TRANSITIONS BETWEEN CAPTURED DRIVING RUNS | 1 |
7,852,315 | 2010 | Camera and acceleration based interface for presentations | 2 |
2010/0312,609 | 2010 | Personalizing Selection of Advertisements Utilizing Digital Image Analysis | 5 |
2010/0312,470 | 2010 | Scrubbing Variable Content Paths | 0 |
2010/0313,113 | 2010 | Calibration and Annotation of Video Content | 0 |
2010/0310,182 | 2010 | GEOCODING BY IMAGE MATCHING | 0 |
7,840,032 | 2010 | Street-side maps and paths | 7 |
7,831,089 | 2010 | Modeling and texturing digital surface models in a mapping application | 2 |
2010/0265,178 | 2010 | CAMERA-BASED MULTI-TOUCH MOUSE | 7 |
2010/0246,965 | 2010 | TAGGING VIDEO USING CHARACTER RECOGNITION AND PROPAGATION | 6 |
2010/0250,120 | 2010 | MANAGING STORAGE AND DELIVERY OF NAVIGATION IMAGES | 5 |
2010/0246,890 | 2010 | DETECTION OF OBJECTS IN IMAGES | 2 |
2010/0251,169 | 2010 | AUTOMATIC GENERATION OF MARKERS BASED ON SOCIAL INTERACTION | 2 |
2010/0245,361 | 2010 | CONTEXT-BASED MANAGEMENT OF MARKERS | 1 |
2010/0250,126 | 2010 | VISUAL ASSESSMENT OF LANDMARKS | 1 |
2010/0250,196 | 2010 | COGNITIVE AGENT | 1 |
2010/0245,344 | 2010 | ANNOTATING OR EDITING THREE DIMENSIONAL SPACE | 0 |
2010/0245,376 | 2010 | FILTER AND SURFACING VIRTUAL CONTENT IN VIRTUAL WORLDS | 0 |
2010/0250,366 | 2010 | MERGE REAL-WORLD AND VIRTUAL MARKERS | 0 |
7,805,066 | 2010 | System for guided photography based on image capturing device rendered user recommendations according to embodiments | 2 |
2010/0241,525 | 2010 | IMMERSIVE VIRTUAL COMMERCE | 3 |
2010/0241,514 | 2010 | AUTOMATED PROVISION OF USER FEEDBACK | 1 |
2010/0241,946 | 2010 | ANNOTATING IMAGES WITH INSTRUCTIONS | 0 |
2010/0235,076 | 2010 | ESTIMATION OF FUEL CONSUMPTION FROM GPS TRAILS | 0 |
2010/0235,078 | 2010 | DRIVING DIRECTIONS WITH MAPS AND VIDEOS | 0 |
2010/0235,356 | 2010 | ORGANIZATION OF SPATIAL SENSOR DATA | 0 |
2010/0225,665 | 2010 | MAP AGGREGATION | 0 |
2010/0218,094 | 2010 | SECOND-PERSON AVATARS | 0 |
7,777,648 | 2010 | Mode information displayed in a mapping application | 21 |
2010/0134,484 | 2010 | THREE DIMENSIONAL JOURNALING ENVIRONMENT | 2 |
2010/0118,025 | 2010 | MODE INFORMATION DISPLAYED IN A MAPPING APPLICATION | 55 |
2010/0085,350 | 2010 | OBLIQUE DISPLAY WITH ADDITIONAL DETAIL | 4 |
2010/0085,371 | 2010 | OPTIMAL 2D TEXTURING FROM MULTIPLE IMAGES | 0 |
2010/0080,551 | 2010 | Geotagging Photographs Using Annotations | 1 |
2010/0080,466 | 2010 | Smart Navigation for 3D Maps | 0 |
2010/0080,489 | 2010 | Hybrid Interface for Interactively Registering Images to Digital Models | 0 |
2010/0074,538 | 2010 | VALIDATION AND CORRECTION OF MAP DATA USING OBLIQUE IMAGES | 1 |
2009/0323,029 | 2009 | MULTI-DIRECTIONAL IMAGE DISPLAYING DEVICE | 2 |
2009/0285,482 | 2009 | DETECTING TEXT USING STROKE WIDTH BASED TEXT DETECTION | 4 |
2009/0284,527 | 2009 | VALIDATING USER GENERATED THREE-DIMENSIONAL MODELS | 4 |
2009/0237,510 | 2009 | VISUALIZING CAMERA FEEDS ON A MAP | 8 |
2009/0208,110 | 2009 | FACTORING REPEATED CONTENT WITHIN AND AMONG IMAGES | 2 |
2009/0190,857 | 2009 | IMPORTANCE GUIDED IMAGE TRANSFORMATION | 2 |
2009/0162,042 | 2009 | GUIDED PHOTOGRAPHY BASED ON IMAGE CAPTURING DEVICE RENDERED USER RECOMMENDATIONS | 3 |
7,548,659 | 2009 | Video enhancement | 20 |
2009/0141,966 | 2009 | INTERACTIVE GEO-POSITIONING OF IMAGERY | 6 |
2009/0110,327 | 2009 | Semi-automatic plane extrusion for 3D modeling | 0 |
2009/0092,277 | 2009 | Geo-Relevance for Images | 4 |
2009/0078,772 | 2009 | Techniques for decoding images of barcodes | 7 |
2009/0079,752 | 2009 | Generating a texture from multiple images | 2 |
2009/0058,855 | 2009 | ELECTROMECHANICAL SURFACE OF ROTATIONAL ELEMENTS FOR MOTION COMPENSATION OF A MOVING OBJECT | 0 |
7,477,236 | 2009 | Remote control of on-screen interactions | 4 |
2009/0002,394 | 2009 | AUGMENTING IMAGES FOR PANORAMIC DISPLAY | 9 |
2009/0003,702 | 2009 | IMAGE COMPLETION | 3 |
2008/0319,659 | 2008 | LANDMARK-BASED ROUTING | 16 |
2008/0319,660 | 2008 | LANDMARK-BASED ROUTING | 11 |
2008/0319,658 | 2008 | LANDMARK-BASED ROUTING | 7 |
2008/0273,795 | 2008 | FLEXIBLE MATCHING WITH COMBINATIONAL SIMILARITY | 8 |
2008/0249,702 | 2008 | Image-based localization for addresses | 1 |
2008/0120,023 | 2008 | DISPLAYING IMAGES RELATED TO A REQUESTED PATH | 3 |
2008/0066,000 | 2008 | PANORAMIC RING USER INTERFACE | 2 |
2008/0050,011 | 2008 | MODELING AND TEXTURING DIGITAL SURFACE MODELS IN A MAPPING APPLICATION | 6 |
2008/0043,020 | 2008 | USER INTERFACE FOR VIEWING STREET SIDE IMAGERY | 29 |
2007/0273,558 | 2007 | DYNAMIC MAP RENDERING AS A FUNCTION OF A USER PARAMETER | 64 |
2007/0236,451 | 2007 | Camera and Acceleration Based Interface for Presentations | 3 |
2007/0210,937 | 2007 | DYNAMIC RENDERING OF MAP INFORMATION | 61 |
2007/0076,920 | 2007 | STREET SIDE MAPS AND PATHS | 27 |
2006/0294,538 | 2006 | Inserting advertising content into video programming | 14 |
2006/0287,083 | 2006 | Camera based orientation for mobile devices | 8 |
2006/0257,042 | 2006 | Video enhancement | 28 |
2006/0248,462 | 2006 | Remote control of on-screen interactions | 2 |