Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

December 20, 2017 | By Nicky Budd-Thanos

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos.

The first of these features, Photosynth, helps create photos that take in more of the perspective or scene you are standing in front of, whether it is wide, tall, or both. It does this by allowing you to freely pan and capture from side to side, up and down, back and forth, and even go back to the start to include any parts of the scene you may have missed.

“The idea came after some frustrations I had when trying to take a picture of Snoqualmie Falls. I didn’t want to have to choose which part of the scene to capture, and I wanted it all with detail. Photosynth means you no longer have to choose. I can now capture the whole scene in a way that feels natural. As with all Pix features, we have also worked to give the best image quality by introducing more intelligent ways to compute exposure and stitching,” said Josh Weisberg, principal program manager within Microsoft’s AI & Research organization in Redmond, Washington.

Photosynth, originally a popular platform for turning digital pictures into 3D panoramas, was launched by Microsoft in 2008 and decommissioned in 2017. Photosynth in Pix shares similarities with the original, leveraging some of the underlying technology, with more features inspired by the original to come. Photosynth in Pix allows for faster and smoother capture, while also making use of Pix’s auto-enhancement capabilities which improve white balance, tone correction and sharpness.

The second feature, Pix Comix, was developed during Microsoft’s OneWeek hackathon and makes use of the core Pix feature Moment Capture in an innovative new way: identifying the most interesting frames in a video to generate a comic strip. Pix uses its deep learning model to score and select three high-quality frames, searching specifically for non-blurry photos, faces with eyes open and interesting scenes.

Pix Comix performs this AI processing on device, selecting the best frames and formatting them into a comic strip. From there, you can add and edit speech bubbles that can be moved, rotated and resized to tell your own story.

Listening and responding to customer feedback on Pix remains a top priority as the app continues to improve and its AI capabilities are further developed. Today’s release marks another step forward in Pix’s commitment to turning that user feedback into new ways to make your pictures even better.

Related:

Up Next

Graphics and multimedia, Human-computer interaction

SwiftKey 7.0: A big change under a small ‘+’

Last week’s rollout of SwiftKey 7.0 is the intelligent keyboard app’s biggest update since joining Microsoft in April 2016. Available on both Android and iOS, SwiftKey 7.0 debuts Toolbar, an expandable menu found under a new ‘+’ sign on the left of your prediction bar that gives you fast and easy access to both new […]

Microsoft blog editor

Computer vision, Graphics and multimedia

Microsoft Pix’s AI gets even smarter, Business Card feature works with LinkedIn to make it easier than ever to manage your contacts

Today's update from Microsoft debuts a new feature for its intelligent camera app, Microsoft Pix, that works with LinkedIn, showcasing the productivity capabilities of the AI-powered camera. The new Business Card feature for iPhones makes it quick and easy for you to add contacts, not only to your iPhone's address book but also to your LinkedIn account. To use, simply open Pix and point your iPhone at a contact's business card. Microsoft Pix automatically detects this and asks if you would like to take action.

Microsoft blog editor

Artificial intelligence, Ecology and environment, Graphics and multimedia

Researchers team up with Chinese botanists on machine learning, flower-recognition project

By Guobin Wu, Senior Research Program Manager, Microsoft Research Asia Has this ever happened to you? You’re out walking with your daughter. She finds a beautiful flower, quizzes you on it, but you’re stumped — you have no idea what it is. Instead of having to admit you don’t know, what if you could quickly […]

Microsoft blog editor