Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

MobileFusion: Research project turns regular mobile phone into 3D scanner

August 24, 2015 | By Microsoft blog editor

By Allison Linn, Senior Writer, Microsoft Research

A new Microsoft Research project lets people to create high-quality 3D images in real time, using a regular mobile phone, with about the same effort it takes to snap a picture or capture a video.

“What this system effectively allows us to do is to take something similar to a picture, but it’s a full 3D object,” said Peter Ondruska, a Ph.D. candidate at Oxford University who worked on the project while he was an intern at Microsoft Research.

The researchers say the system, called MobileFusion, is better than other methods for 3D scanning with a mobile device because it doesn’t need any extra hardware, or even an Internet connection, to work. That means scientists in remote locations or hikers deep in the woods can capture their surroundings using a regular cell phone without a Wi-Fi connection.

“Everything happens on the phone itself,” said Pushmeet Kohli, a principal research scientist with Microsoft Research who also worked on the project.

The scans are high-quality enough to be used for things like 3D printing and augmented reality video games.

Shahram Izadi, a principal researcher who also worked on the project, said he imagines people using a tool like this to take a 3D scan of something they see on vacation, such as the Eiffel Tower, and immediately sharing it with friends or family.

He said a person might also use it to take a 3D scan of something they wanted to sell online, such as a vase or a lamp, and post that, instead of the more conventional picture or video.

“This is really about the accessibility and ubiquity of 3D scanning,” Izadi said.

The researchers will present MobileFusion in early October at the International Symposium on Mixed and Augmented Reality.

Kohli and Izadi had previously worked on a similar project, called Kinect Fusion, which lets people build 3D models of their houses, offices or even themselves. However, that system requires a PC and other gadgets, making it difficult to do any 3D scanning on the go.

When Ondruska came to the lab, Kohli said they were curious to see if they could “go to the next level, in the sense of usability.”

They found most mobile devices had become powerful enough that they could build a 3D-scanning system just using the computational power found on a regular mobile phone.

“The great starting point was to take a sensor that everyone has in their pocket, which is the camera you have on your mobile phone,” Izadi said.

The researchers then developed an algorithm that allowed the camera to act as a 3D scanner, using a technique of taking multiple images that is similar to how the human eye works, Izadi explained.

Currently, the researchers are working on making sure the system works with all types of mobile devices, including Windows Phone, Android and iPhone devices. Izadi said they hope to eventually make it available to the general public in some form, but there are no firm plans right now.


Up Next

Artificial intelligence, Computer vision, Graphics and multimedia

Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision […]

Marc Pollefeys

Partner Director of Science

Computer vision, Graphics and multimedia, Human-computer interaction

Typing in the Virtual Office Environment: Examining the Effects of Keyboard and Hand Representation in VR

Much of the hype around Virtual Reality, or VR, has focused on immersive gaming and entertainment, but the technology also has profound implications for enhancing the way we work. Imagine the perfect office environment: multiple large displays, proper lighting, free from distractions and isolated from outside disturbances. VR can be used to generate the ideal virtual office—even from the confines of an airplane seat or a tiny touchdown space.

Eyal Ofek

Senior Researcher

Microsoft Pix before and after panoramic photo of Miners Landing

Artificial intelligence, Graphics and multimedia

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos. The first of these features, Photosynth, helps create photos that take in more of the perspective or […]

Nicky Budd-Thanos