Young Interns Spend Summer Translating Songs

Posted by Rob Knies

 

So, how was your summer?

For three Microsoft Research interns, that question has them singing a happy tune.

The three—Michelle Agcamaran, Priya Ganesan, and Kat Zhou—spent the summer at Microsoft Research Redmond, working with mentor Alex Cheng. In doing so, they also stamped their names on a cool piece of technology. You can actually download the source code and play with it yourself.

It’s called Song Translator, a Windows 8 app that can translate your favorite songs into a new language. The technology, which showcases Bing Translator capabilities, enables users to record songs in a new language, then share them with friends.

Sound like a productive summer for the interns involved? You bet—and, oh, did I mention that the creators were all of high school age?

The interns, though, might dispute that description, given that they have completed their secondary education and are now ensconced in some of the top universities in the United States—Stanford, USC, Harvard. Still, Song Translator represents an impressive achievement to have under your belt as you begin your undergraduate career. That’s something to contemplate during the first day of the annual Grace Hopper Celebration of Women in Computing.

Cheng, a software-design engineer for the Machine Translation team who mentored the interns along with Abdulaziz Mohammed, can only marvel at what they were able to produce.

“Song Translator brings together a number of technologies—Bing Translator and Windows Azure—to build a fun download,” he says. “The interns were always eager to learn and enthusiastic about solving challenging problems and integrating complex components. I am proud of what they have built and the amount they have learned over the 10 weeks they were here.”

One of the three, Ganesan, from the International Community School in Kirkland, Wash., explains how things got going.

“On the first day of our internship, our mentors first pitched to us the idea of building an app that would translate song lyrics,” says Ganesan, now beginning her studies in computer science, with a minor in economics, at Stanford. “We were very intrigued with the concept and the added challenges of translating songs due to their unique musical characteristics, such as rhythm and syntax.

“The three of us then expanded a lot on the idea of translating songs and came up with new features for our app, such as Auto-Tune, voice recording, and being able to edit song lyrics.”

As she says, as the project they undertook gained momentum, it got quite complex. The process of creating a new song in a different language followed these steps:

• Song choice: This included a file containing just the music track, a transcript of the lyrics, and the complete song. The interns would use these to align the lyrics and the musical accompaniment via a series of time stamps. The music and lyrics were stored in Azure, enabling sharing and preserving the data, and the users could connect via a mobile service.
• Translation: This resulted in two sets of lyrics, one original and one translated into the language of choice.
• Alignment: Using text-to-speech and Auto-Tune, the music-only track was played back with an automated voice singing the translated lyrics. An application-programming interface from the Sonic API music-processing web service was used to alter the pitch and the duration of the text-to-speech recording to match the rhythm of the music-only track.

The result is that users can record their own singing, similar to karaoke, to create a new song sung in a different language. Integrating the individual pieces of the work produces a complete, end-to-end experience.

“The coolest feature we built in the Song Translator would be the record function,” says Agcamaran, from Holy Names Academy in Seattle. “When we first began our project, the recording feature was meant to be more of a bonus feature to add on if we had the time. Our main focus was to achieve the song translation and storage into the cloud before we went on to any extra features.

“Our weeks spent working on the project together proved to be productive, and we soon had our record function for the app,” says Agcamaran, now studying computer science with minors in music production and foreign language, at USC. “We had a lot of fun singing over lines of translated and untranslated songs, and the playback proved to be very satisfying in the end.”

For Zhou, from Interlake High School in Bellevue, Wash., a different part of the project stood out.

“I’m most proud of the implementation of cloud storage using Windows Azure,” says Zhou, now pursuing studies in mathematics and computer science at Harvard. “It was a challenging process due to a lack of documentation, which made the implementation that much more satisfying.”

She also confides that she had a great internship experience.

“I think it was a wonderful internship,” Zhou says. “I was surrounded by a great group of peers, and I made plenty of friends throughout my internship. In addition, I had excellent mentors who were great at guiding me through the processes, and with their guidance, I learned a lot.”

For Agcamaran, the opportunity to view life from within Microsoft Research proved most gratifying.

“For the duration of the internship, I would say that the environment was the most interesting,” she recalls. “Working in Microsoft Research, we were exposed to a wide array of experimental projects. Not only that, but the workers themselves gave off such motivational energy. Everyone on campus was approachable and easy to talk to. There was a fine balance of professional and casual attitude.

“It was inspiring to hear the employees’ individual stories and to hear about some of the projects they’ve worked on in the past. In addition, companywide events occurred so frequently that we always had something new to look forward to. And, of course, meeting my fellow high school interns proved to be very valuable. I made great friendships and had the opportunity to work with some brilliant minds.”

Brilliance aside, the Song Translator interns also had a chance to demonstrate their playful side. You know what they say about all work and no play? Ganesan says she and her fellow interns took that to heart.

“The three of us got to know the other Microsoft Research interns really well in the first week,” she says, “and we decided to start a mini-prank war with the other interns—one that lasted throughout the whole internship. It was a fun way to take a short break in the middle of work every day and ultimately form closer friendships with the other interns.”

Let’s see here: high school graduation, a high-powered internship, a high-tech project, music, activities, pranking, new friendships …

So how, exactly, was your summer?

Opens in a new tab