Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

December 20, 2017 | By Nicky Budd-Thanos

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos.

The first of these features, Photosynth, helps create photos that take in more of the perspective or scene you are standing in front of, whether it is wide, tall, or both. It does this by allowing you to freely pan and capture from side to side, up and down, back and forth, and even go back to the start to include any parts of the scene you may have missed.

“The idea came after some frustrations I had when trying to take a picture of Snoqualmie Falls. I didn’t want to have to choose which part of the scene to capture, and I wanted it all with detail. Photosynth means you no longer have to choose. I can now capture the whole scene in a way that feels natural. As with all Pix features, we have also worked to give the best image quality by introducing more intelligent ways to compute exposure and stitching,” said Josh Weisberg, principal program manager within Microsoft’s AI & Research organization in Redmond, Washington.

Photosynth, originally a popular platform for turning digital pictures into 3D panoramas, was launched by Microsoft in 2008 and decommissioned in 2017. Photosynth in Pix shares similarities with the original, leveraging some of the underlying technology, with more features inspired by the original to come. Photosynth in Pix allows for faster and smoother capture, while also making use of Pix’s auto-enhancement capabilities which improve white balance, tone correction and sharpness.

The second feature, Pix Comix, was developed during Microsoft’s OneWeek hackathon and makes use of the core Pix feature Moment Capture in an innovative new way: identifying the most interesting frames in a video to generate a comic strip. Pix uses its deep learning model to score and select three high-quality frames, searching specifically for non-blurry photos, faces with eyes open and interesting scenes.

Pix Comix performs this AI processing on device, selecting the best frames and formatting them into a comic strip. From there, you can add and edit speech bubbles that can be moved, rotated and resized to tell your own story.

Listening and responding to customer feedback on Pix remains a top priority as the app continues to improve and its AI capabilities are further developed. Today’s release marks another step forward in Pix’s commitment to turning that user feedback into new ways to make your pictures even better.

Related:

Up Next

Artificial intelligence

Making the future of work work for you with Dr. Johannes Gehrke

Episode 83, July 17, 2019- Dr. Johannes Gehrke is a Microsoft Technical Fellow and head of Architecture and Machine Learning for the Intelligent Communications and Conversations Cloud in Microsoft’s Experiences and Devices division. But lest you think his lofty position makes him in any way superior to you, let me tell you, he knows who works for whom, and he’ll be the first to tell you that you are his boss! On today’s podcast, Dr. Gehrke frames the new, cloud-powered work world as a fast paced, widely-distributed workplace that demands real-time decision-making and collaboration – and explains how products like Microsoft Teams are meeting those demands – and tells us, both directly and indirectly, about the future of work, which for Microsoft, involves a pivot from an app-centric approach to a people-centric approach where, by using an AI-infused productivity suite coupled with the power of the cloud, we can essentially “hire Microsoft” to help us get our work done.

Microsoft blog editor

Artificial intelligence, Computer vision

A picture from a dozen words – A drawing bot for realizing everyday scenes—and even stories

If you were asked to draw a picture of several people in ski gear, standing in the snow, chances are you’d start with an outline of three or four people reasonably positioned in the center of the canvas, then sketch in the skis under their feet. Though it was not specified, you might decide to […]

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision […]

Marc Pollefeys

Partner Director of Science