Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

MobileFusion: Research project turns regular mobile phone into 3D scanner

August 24, 2015 | By Microsoft blog editor

By Allison Linn, Senior Writer, Microsoft Research

A new Microsoft Research project lets people to create high-quality 3D images in real time, using a regular mobile phone, with about the same effort it takes to snap a picture or capture a video.

“What this system effectively allows us to do is to take something similar to a picture, but it’s a full 3D object,” said Peter Ondruska, a Ph.D. candidate at Oxford University who worked on the project while he was an intern at Microsoft Research.

The researchers say the system, called MobileFusion, is better than other methods for 3D scanning with a mobile device because it doesn’t need any extra hardware, or even an Internet connection, to work. That means scientists in remote locations or hikers deep in the woods can capture their surroundings using a regular cell phone without a Wi-Fi connection.

“Everything happens on the phone itself,” said Pushmeet Kohli, a principal research scientist with Microsoft Research who also worked on the project.

The scans are high-quality enough to be used for things like 3D printing and augmented reality video games.

Shahram Izadi, a principal researcher who also worked on the project, said he imagines people using a tool like this to take a 3D scan of something they see on vacation, such as the Eiffel Tower, and immediately sharing it with friends or family.

He said a person might also use it to take a 3D scan of something they wanted to sell online, such as a vase or a lamp, and post that, instead of the more conventional picture or video.

“This is really about the accessibility and ubiquity of 3D scanning,” Izadi said.

The researchers will present MobileFusion in early October at the International Symposium on Mixed and Augmented Reality.

Kohli and Izadi had previously worked on a similar project, called Kinect Fusion, which lets people build 3D models of their houses, offices or even themselves. However, that system requires a PC and other gadgets, making it difficult to do any 3D scanning on the go.

When Ondruska came to the lab, Kohli said they were curious to see if they could “go to the next level, in the sense of usability.”

They found most mobile devices had become powerful enough that they could build a 3D-scanning system just using the computational power found on a regular mobile phone.

“The great starting point was to take a sensor that everyone has in their pocket, which is the camera you have on your mobile phone,” Izadi said.

The researchers then developed an algorithm that allowed the camera to act as a 3D scanner, using a technique of taking multiple images that is similar to how the human eye works, Izadi explained.

Currently, the researchers are working on making sure the system works with all types of mobile devices, including Windows Phone, Android and iPhone devices. Izadi said they hope to eventually make it available to the general public in some form, but there are no firm plans right now.


Up Next

Microsoft Pix before and after panoramic photo of Miners Landing

Artificial intelligence, Graphics and multimedia

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos. The first of these features, Photosynth, helps create photos that take in more of the perspective or […]

Nicky Budd-Thanos

Project Vermont

Graphics and multimedia, Hardware, devices and quantum computing, Human-computer interaction

Project Vermont: Improving camera performance through curved image sensors

By Richard Stoakley, Microsoft Research It’s hard to remember when our world wasn’t full of cameras. They’re in our phones and tablets, posted on street corners and in our buildings and cars. We rely on them not just to capture moments and memories, but for security, navigation, 3D spatial mapping and processing medical data. While […]

Microsoft blog editor

Data management, analysis and visualization, Graphics and multimedia

Partnership yields key breakthroughs in VR’s “grand challenge”

By Noboru Sean Kuno, Research Program Manager, Microsoft Research Asia The potential for virtual reality (VR) to upend industrial design, medicine, and other specialized fields has now vaulted the emerging field into the ranks of what the National Academy of Engineering calls its 14 grand challenges of the 21st century, an eclectic list of endeavors […]

Microsoft blog editor