Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

The Friendly Faces of Microsoft Research Asia

June 4, 2012 | By Microsoft blog editor

Posted by Rob Knies

 Face SDK image

The term “natural user interfaces” has been in vogue in recent months, generally invoked to describe different ways that humans can interact with computing devices beyond the longtime pairing of keyboard and mouse.

Surface computing is one example with its roots in Microsoft Research. Kinect functionalities also benefited from work in Microsoft Research labs. Now, scientists at Microsoft Research Asia are examining ways that you can interact with computers using … your face. Qiufeng Yin, a software-development engineer at that Beijing-based lab, explains.

“We envision a world in which mobile devices—phones, tablets, sensors—become more and more ubiquitous,” Yin says. “We hope to make such devices more human-friendly. They can be personalized to a user, and the face is another important, though underutilized, area for interaction, in addition to voice and touch.”

That’s the rationale behind the beta version of the Microsoft Research Face Software Development Kit (SDK), now available for download. Incorporating the latest face technologies from Microsoft Research, the SDK enables engineers to develop face-based applications for Windows Phone.

“Microsoft Research scientists have been working on face-related technologies for the past 10 years,” Yin notes. “We feel it’s the right time to share our research results in an easy-to-use package.”

To demonstrate the potential of the SDK, the team—which includes personnel from the Visual Computing, Web Search and Mining, and Innovation Engineering groups at Microsoft Research Asia, in addition to a couple of contributors from Microsoft Research Redmond—have released three face technologies that have been fashioned into Windows Phone apps: Face Swap, Face Mask, and Face Touch. Now, the researchers are eager to get feedback on the SDK from the research and development communities.

State-of-the-art algorithms included in the SDK process face images using four distinct “modules”:

  • Face detection: This module tries to detect every face that appears in an image. Its analysis returns a set of rectangles, each of which identifies the face position of individuals pictured in the image. The associated algorithm automatically copes with changes in lighting and different face-view angles.
  • Face alignment: This module locates the components of the face in an image—the eyes, the eyebrows, the mouth, the nose. The facial analysis can pinpoint the center of such facial features or provide an outline.
  • Face tracking: This module locates a face position in real time during a live video stream, thus enabling users to use head movement to interact with a Windows Phone.
  • Cartoon generation: You provide the SDK with an image, and this feature creates a personalized cartoon portrait, which can be customized by selecting various styles or applying a selection of templates.

Such features could lead some to wonder about privacy implications, but, Yin notes, the SDK doesn’t support human identification by facial features.

“We believe people can build fun, valuable applications using our SDK,” he says. “We are sure people can find many noble and legal uses for face-recognition technology—and have ways to mitigate the side effects—if such technology becomes widely available.”

That, of course, is the team’s goal.

“Over the long term,” Yin concludes, “we hope to see a wide spectrum of different applications taking advantage of the unique advantage of human face interaction, especially in the area of digital entertainment.”