{"id":993,"date":"2013-10-30T11:00:00","date_gmt":"2013-10-30T11:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/msr_er\/2013\/10\/30\/kinect-sign-language-translator-part-2\/"},"modified":"2016-08-22T20:24:23","modified_gmt":"2016-08-23T03:24:23","slug":"kinect-sign-language-translator-part-2","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/kinect-sign-language-translator-part-2\/","title":{"rendered":"Kinect Sign Language Translator – part 2"},"content":{"rendered":"

Today, we have the second part of a two-part blog posted by program managers in Beijing and Redmond respectively\u2014second up, Stewart Tansley:<\/em><\/p>\n

When Microsoft Research shipped the first official Kinect for Windows software development kit (SDK) beta\u00a0in June 2011, it was both an ending and a beginning for me. The thrilling accomplishment of rapidly and successfully designing and engineering the SDK was behind us, but now the development and supporting teams had returned to their normal research work, and I was left to consider how best to showcase the research potential of Kinect technology beyond gaming.<\/p>\n

Since Kinect\u2019s launch in November 2010, investigators from all quarters had been experimenting with the system in imaginative and diverse applications. There was very little chance of devising some stand-out new application that no one had thought of\u2014since so many ideas were already in play. So I decided to find the best of the current projects and \u201cdouble down\u201d on them.<\/p>\n

\"Sign<\/p>\n

But rather than issuing a public global call\u2014which we didn\u2019t\u00a0do, because so many people were proactively experimenting with Kinect technology\u2014we turned to the Microsoft Research labs around the world and asked them to submit their best Kinect collaborations with the academic world, thus bringing together professors and our best researchers, as we normally do in Microsoft Research Connections.<\/p>\n

We whittled twelve outstanding proposals to five finalists and picked the best three for additional funding and support. One of those three was the Kinect Sign Language Translator, a collaboration among Microsoft Research Asia (opens in new tab)<\/span><\/a>, the Chinese Academy of Sciences (opens in new tab)<\/span><\/a>, and Beijing Union University (opens in new tab)<\/span><\/a>.<\/p>\n

Incredibly, the Beijing-based team delivered a demonstration model in fewer than six months, and I first saw it run in October 2012, in Tianjin. Only hours earlier, I had watched a seminal on-stage demo of simultaneous speech translation, during which Microsoft Research\u2019s then leader, Rick Rashid, spoke English into a machine learning system that produced a pitch-perfect Chinese translation\u2014all in real time, on stage before 2,000 Chinese students. It was a “Star Trek” moment. We are living in the future!<\/p>\n

Equally inspiring though, and far away from the crowds, I watched the diminutive and delightful Dandan Yin gesture to the Kinect device connected to the early sign language translator prototype\u2014and words appeared on the screen! I saw magic that day, and not just on stage.<\/p>\n

\"Sign.Nine months later, in July 2013, we were excited to host Dandan at the annual Microsoft Research Faculty Summit in Redmond\u2014her first trip outside China. We were thrilled with the response by people both attending and watching the Summit. The sign language translator and Dandan made the front page of the Seattle Times and were widely covered by Internet news sites.<\/p>\n

We knew we had to make a full video of the system to share it with others and take the work further. Over a couple of sweltering days in late July (yes, Seattle does get hot sunny days!), we showed the system to Microsoft employees. It continued to capture the imagination, including that of Microsoft employees who are deaf.<\/p>\n

We got the chance to demonstrate the system at the Microsoft annual company meeting in September 2013\u2014center stage, with 18,000 in-person attendees and more than 60,000 watching online worldwide. This allowed us to bring Dandan and the Chinese research team back to Seattle, and it gave us the opportunity to complete our video.<\/p>\n

That week, we all went back into the studio, and through a long hard day, shot the remaining pieces of the story, explaining how the system could one day transform the lives of millions of people who are deaf or hard or hearing\u2014and all of us\u2014around the world.<\/p>\n

I hope you enjoy the video and are inspired by it as much we are.<\/p>\n

We look forward to making this technology a reality for all! We would love to hear your comments.<\/p>\n

\u2014Stewart Tansley (opens in new tab)<\/span><\/a>, Director, Microsoft Research Connections<\/em><\/p>\n

Learn more<\/strong><\/p>\n