Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Computer Vision at Microsoft: Uniting fundamental research and industry-defining products

September 7, 2018 | By Andrew Fitzgibbon, Principal Researcher

mountains at ECCV

Microsoft is very proud to be a diamond sponsor of ECCV 2018 and we’re in Munich, Germany from September 8-14 with the global computer vision community to share our research and to learn from our fellow contributors.

At Microsoft, in parallel with fundamental research, we build products. Our software products, like Visual Studio, PowerPoint and Linux on Windows are used every day by computer vision researchers and engineers. Hardware products, like Surface, Xbox, and HoloLens, contain some of the most advanced computer vision systems in the world (the head tracking in HoloLens is still the industry standard two years after its launch, even when compared to devices with orders of magnitude more computational power). Today, a combination of software and hardware drives Azure and our vision of an intelligent cloud powering devices on the intelligent edge. At ECCV we are demonstrating Project Kinect for Azure, a new standalone depth sensor derived from the sensor in HoloLens. This sensor is smaller than previous Kinect sensors, consuming less power while offering greater accuracy. Designed for use in a huge range of applications, the sensor data can be processed locally, on deep learning hardware, or combined with Azure cloud services to integrate with computer vision and machine learning algorithms at all levels.

For our researchers and engineers, the rich intellectual breadth of our roles is reflected in the range of papers and presentations we are contributing to this conference. We offer state-of-the-art computer vision services in the cloud through our Azure Cognitive Services. We also work on novel hardware such as the HoloLens, with its embedded low-power DNN coprocessor and we’re advancing innovation with our FPGA-enabled Azure nodes. At this conference, we are revealing more details of the process of writing incredibly efficient computer vision algorithms. Indeed, we show that at Microsoft traditional distinctions between “research” and “engineering” roles are increasingly blurred. As one of the senior computer vision people at Microsoft, I work just as happily on topics that could be viewed as research or engineering – exploring new mathematical models of surface geometry, designing software architectures for parallelized nonlinear optimization, or low-level optimization of numerical code. Efficiency is not just needed for low-power devices; by making our code more efficient, we learn about fundamental properties of the machine learning and computer vision algorithms we use. One might argue that a key contributor to the sea change brought about by deep neural networks was Alex Krizhevsky’s combination of engineering and research. In this conference, as we reveal more about the process of building the world’s finest SLAM system, we are sharing how engineering can lead to new research and how new research guides engineering.

If you’ve followed computer vision at Microsoft, you may know that, as in many companies – and indeed many universities – our computer vision activities have historically been distributed across the company. We have had researchers at Microsoft Research and applied scientists in product groups such as Bing, Xbox, and HoloLens. As of this summer, however, we have unified several computer vision teams under the visionary leadership of Alex Kipman, inventor of Kinect and HoloLens, in the fastest-growing sector of the company, Azure. Our new strategic vision gives us incredible power to build great products, where everyone can share expertise and where all our efforts are amplified. This unified team still has the advantage of being geographically diverse. We have computer vision experts in the US on the West Coast, in China and of course in Europe where there are expanding teams in Cambridge and Belgrade and a newly founded laboratory under Marc Pollefeys in Zurich, Switzerland.

If you’re attending ECCV, we encourage you to come to our talks and posters and to drop by the Microsoft exhibit stand to chat with people from across Microsoft – recruiters, scientists, researchers, and engineers. We look forward to hearing your ideas and sharing ours.

Up Next

Artificial intelligence, Computer vision

Holograms, spatial anchors and the future of computer vision with Dr. Marc Pollefeys

On today’s podcast, Dr. Pollefeys brings us up to speed on the latest in computer vision research, including his innovative work with Azure Spatial Anchors, tells us how devices like Kinect and HoloLens may have cut their teeth in gaming, but turned out to be game changers for both research and industrial applications, and explains how, while it’s still early days now, in the future, you’re much more likely to put your computer on your head than on your desk or your lap.

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision […]

Marc Pollefeys

Partner Director of Science

Artificial intelligence, Computer vision, Graphics and multimedia

Second version of HoloLens HPU will incorporate AI coprocessor for implementing DNNs

By Marc Pollefeys, Director of Science, HoloLens It is not an exaggeration to say that deep learning has taken the world of computer vision, and many other recognition tasks, by storm. Many of the most difficult recognition problems have seen gains over the past few years that are astonishing. Although we have seen large improvements […]

Microsoft blog editor