Posted by
Eager for foreign travel but worried about foreign languages? Fear no more, thanks to the Translator App for Windows Phone (opens in new tab), updated April 16 and featuring several contributions from Microsoft Research.
Microsoft Research Podcast
AI Frontiers: AI for health and the future of research with Peter Lee
Peter Lee, head of Microsoft Research, and Ashley Llorens, AI scientist and engineer, discuss the future of AI research and the potential for GPT-4 as a medical copilot.
The app, powered by the same state-of-the-art technology used in Bing Translator (opens in new tab), and available for free download (opens in new tab) on Windows Phone Marketplace, enables a new travel experience by offering a variety of machine-translation scenarios certain to please the nascent globetrotter. With the app, users can translate street signs, posters, train schedules, and menus simply by pointing and scanning with their phone’s camera; download highly optimized, compressed language packs to get translation assistance while avoiding huge data-connection bills; use keyboard input to gain instant translations, some of them featuring spoken playback; and, in German, French, Italian, Spanish, and both U.S. and U.K. English, tap an icon, speak, then tap again for a translation.
The translation aspects of the app, notes Vikram Dendi (opens in new tab), director of product strategy for Microsoft Research Redmond (opens in new tab)’s Machine Translation (opens in new tab) team, depend on two things: the Microsoft Translator web service, used by such entities as Bing (opens in new tab), Office (opens in new tab), Internet Explorer (opens in new tab), and Facebook; and a highly compressed and optimized version of the translation engine that uses downloadable language packs.
“The new Translator app,” Dendi says, “takes advantage of Windows Phone’s unique features to deliver a great translation experience, enabling you to go to new places with confidence.”
The research included cross-lab collaboration between researchers from Microsoft Research’s Redmond and Microsoft Research Asia (opens in new tab), including members of the Machine Translation and Natural Language Processing (opens in new tab) groups, as well as Speech groups based in both Redmond (opens in new tab) and Beijing (opens in new tab).
Chris Wendt (opens in new tab), principal group program manager for the Machine Translation group, cites three significant Microsoft Research contributions to the new version of the Translator App:
- The machine-translation service, end to end.
- The machine-translation language pack for download.
- The adaptation and optimization of the machine-translation service for speech input and travel scenarios.
Alex Acero (opens in new tab), a research manager for Microsoft Research Redmond and head of that lab’s Speech group, mentions a couple of technical contributions that merit particular attention.
“Xiaodong He (opens in new tab) on the Speech team worked on machine-translation adaptation algorithms,” Acero says, “and worked with the Machine Translation team to integrate them into the main system.”
That system represented a collaboration with the Microsoft Tellme (opens in new tab) team, with Microsoft Research building customizations atop the TellMe speech engine.
In December, during the International Workshop on Spoken Language Translation (opens in new tab), He’s Chinese-English machine-translation entry took first place in the workshop’s Evaluation Campaign. New technologies pioneered in that entry, including topic adaptation and end-to-end optimization of translation models, will be integrated into the next version of the Translator App.
“There’s also a compact machine-translation engine that runs in the phone,” Acero says. “That is also cool, because it requires no data connection, important when you’re traveling if you don’t want to get hit with a huge bill when you come back home. The server engine is huge, so compressing it to fit in a phone was not an easy feat.”
Chris Quirk (opens in new tab) and Anthony Aue (opens in new tab) of the Natural Language Processing team get the credit for that contribution.
Frank Seide (opens in new tab), senior researcher and research manager for Microsoft Research Asia, was joined by Kit Thambiratnam, now senior development lead for the Beijing-based Search Technology Center, on the Translating Telephone (opens in new tab) project, one of the key steps toward the Translator App. Seide knows how difficult it is to incorporate speech-to-speech translation.
“A big challenge is that machine-translation systems are built for the written form of language,” he says, “while a speech-to-speech system is faced with spoken language. Spoken language not only lacks punctuation and is riddled with ‘ums,’ ‘ahs,’ and folks correcting themselves mid-sentence, but it also differs grammatically quite a bit.”
Conquering such obstacles did not go unnoticed.
“In building this app,” Dendi concludes, “engineers and researchers on our team solved tremendous technical challenges and dealt with the nuances of voice, cameras, data availability, and language complexity. Our focus was on delivering a scenario that is truly useful.”