Microsoft Research https://www.microsoft.com/en-us/research Sat, 03 Dec 2016 03:56:07 +0000 en-US hourly 1 https://wordpress.org/?v=4.5.4 Substance, not hype, powers AI excitement at premier machine learning conference https://www.microsoft.com/en-us/research/blog/substance-not-hype-powers-ai-excitement-premier-machine-learning-conference/ Fri, 02 Dec 2016 06:00:57 +0000 https://www.microsoft.com/en-us/research/?p=328967 By Christopher Bishop, Distinguished Scientist and Director of Microsoft Research Cambridge Lab This month, I will attend the Conference and Workshop on Neural Information Processing Systems (NIPS), the premier gathering in the machine learning field. I’ve participated in this conference most years since it began in 1987 and I’m looking forward once again to catching […]

The post Substance, not hype, powers AI excitement at premier machine learning conference appeared first on Microsoft Research.

]]>
By Christopher Bishop, Distinguished Scientist and Director of Microsoft Research Cambridge Lab

Christopher Bishop, Distinguished Scientist, Managing Director, Microsoft Research Cambridge Lab, Artificial Intelligence

This month, I will attend the Conference and Workshop on Neural Information Processing Systems (NIPS), the premier gathering in the machine learning field. I’ve participated in this conference most years since it began in 1987 and I’m looking forward once again to catching up with colleagues and friends as well as exploring new developments in the field. Until recently, the conference attracted a few hundred attendees. The number of participants has grown rapidly in recent years and this year there are more than 4,500 people registered!

This explosion of activity in machine learning is remarkable and reflects the positive trend of research making its way to the marketplace. The first manifestation was the growing interest in “big data.” More recently, the focus shifted to “artificial intelligence.” I am regularly asked to speak on AI, including our research as well as more generally about the social, economic, business, and government policy implications. Everyone wants to know what AI means for them. And while there may be more to AI than machine learning, the resurgence of interest in building intelligent machines undoubtedly stems from advances in machine learning, including deep neural networks.

Is this excitement about AI just hype, or is there substance too? In my view, computing is undergoing the most substantial transformation since the foundations of the field were laid by Alan Turing some eight decades ago. This revolution has two complementary aspects. One is the shift from software solutions that are hand-crafted to solutions learned from data. The second transformation underway is from a view of computation as logic to one involving uncertainty expressed through probabilities. Learning from data and computing with uncertainty are intimately linked. From a probabilistic perspective, machine learning can be viewed as a reduction in uncertainty as a result of observing new data. This process is intrinsically sequential and open-ended, with the posterior distribution resulting from observations so far acting as the prior distribution for the next round of data.

This reduction in uncertainty is illustrated in the Clutter productivity feature, first designed in our labs and recently introduced into the Microsoft Exchange email system, used regularly by tens of millions of users. The feature employs a hierarchical probabilistic model to classify a user’s email into high and low priority. Since definition of high and low priority varies from one user to another, a hierarchical model is used to enable personalization. At the top level of the hierarchy is a probabilistic model learned across a large population of users. New users experience this prior model. The system continually adapts based on the user’s own email usage, providing each user a personal set of parameters derived from the shared prior. This allows a new user to have a positive initial experience, while also providing a system that continues to learn thereby creating a customized experience for each user.

Writing the software to implement these kinds of probabilistic models can be complex and challenging. In this case, however, the process was streamlined by creating the code automatically using Infer.NET, which provides a programming language and associated compiler. Infer.NET supports probabilistic variables as first class citizens and provides an elegant example of probabilistic programming. While there are numerous probabilistic programming languages under development, the focus of Infer.NET is efficient inference for large-scale applications.

Microsoft is placing machine learning and AI at the core of its strategy and we are looking for exceptionally talented scientists and engineers interested in this field to join us. We recently created a new AI and Research group of more than 5,000 researchers and engineers dedicated to developing advances in AI. There is tremendous breadth and depth of talent across the group and ample opportunities for teams to collaborate on some of the world’s toughest research and engineering challenges, resulting in the ability to positively impact the lives of millions. Microsoft has long been at the forefront of machine learning and AI. Today, Microsoft holds the record error rate for object recognition in images and recently announced the first achievement of human parity in word-error rate for speech recognition, both built on deep-learning technology developed in our research labs. We also recently announced the creation of the world’s first exa-scale AI supercomputer, based on a global deployment of FPGAs (field programmable gate arrays) in our data centers to complement our CPU and GPU capabilities.

At Microsoft, we are democratizing AI to empower every person and every organization on the planet to achieve more. I’m looking forward to NIPS as a superb opportunity to meet and talk to people about how they can join us in achieving this goal.

Related:

The post Substance, not hype, powers AI excitement at premier machine learning conference appeared first on Microsoft Research.

]]>
Making better use of the crowd https://www.microsoft.com/en-us/research/blog/making-better-use-of-the-crowd/ Thu, 01 Dec 2016 16:00:46 +0000 https://www.microsoft.com/en-us/research/?p=328679 By Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research Over the last decade, computer scientists have harnessed crowds of Internet users to solve tasks that are notoriously difficult to crack with computers alone, such as determining whether an image contains a tree, rating the relevance of websites, and verifying phone numbers. The machine learning community was […]

The post Making better use of the crowd appeared first on Microsoft Research.

]]>
By Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research

Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research

Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research
Photography: John Brecher

Over the last decade, computer scientists have harnessed crowds of Internet users to solve tasks that are notoriously difficult to crack with computers alone, such as determining whether an image contains a tree, rating the relevance of websites, and verifying phone numbers.

The machine learning community was early to embrace so-called crowdsourcing to quickly and inexpensively obtain the vast quantities of labeled data needed to train machine learning systems how to classify images or recognize speech, for example. Labeled data are essentially sets of teaching examples, such as pictures of cats that are tagged with the word “cat.”

Usually this handoff of labeled data is where interaction with the crowd ends. Are there better ways to make use of the crowd?

On December 5, I will showcase innovative uses of crowdsourcing that go beyond data collection during a crowdsourcing tutorial at NIPS, the premier international machine learning conference, held this year in Barcelona. The tutorial will also dive into recent research aimed at understanding who crowdworkers are and how they behave, which could inform best practices for interacting with the crowd.

Innovative uses of crowdsourcing that go beyond the collection of data include:

  • Harnessing the crowd to improve machine-learning models including extracting features from labeled data most relevant for model training and evaluation of learned models.
  • Leveraging the complementary strengths of humans and machines to achieve more than either can alone. Potential applications of these so-called hybrid-intelligence systems include real-time on-demand closed captioning of day-to-day conversations and crowd-powered writing and editing.
  • Using crowdsourcing platforms to recruit large pools of subjects for experiments designed to study the effects of human behavior when reasoning about the performance of computer systems, which could lead to better designed algorithms and systems.

Recent research—both qualitative and quantitative—has opened the black box of crowdsourcing to uncover that crowdworkers are not just independent contractors, but rather a network with a rich communication structure.  Meanwhile, experiments have explored how to boost the quality of crowdwork using both well-designed monetary incentives (such as performance-based payments) and intrinsic motivation (such as piqued curiosity).

This research has much to teach us about how to most effectively interact with the crowd. (Hint: Be respectful, be responsive, be clear.)

Crowdsourcing has the potential for major impact on the way we design machine learning and AI systems, but to unleash this potential we need more creative minds exploring novel ways to use it.

Related

The post Making better use of the crowd appeared first on Microsoft Research.

]]>
Top PhD students gather at Microsoft Research Asia PhD Forum 2016 https://www.microsoft.com/en-us/research/blog/top-phd-students-gather-microsoft-research-asia-phd-forum-2016/ Tue, 29 Nov 2016 21:06:06 +0000 https://www.microsoft.com/en-us/research/?p=325277 By Lily Sun, Research Program Manager, Microsoft Research Asia Since its inception in 1998, Microsoft Research Asia (MSRA) has anointed nearly 400 top researchers with the coveted honor of “Microsoft Research Fellow.” Some of them recently returned to Beijing, sharing their insights and best practices at the inaugural Microsoft Research Asia PhD Forum, held earlier […]

The post Top PhD students gather at Microsoft Research Asia PhD Forum 2016 appeared first on Microsoft Research.

]]>
By Lily Sun, Research Program Manager, Microsoft Research Asia

Microsoft Research Asia PhD Forum

Since its inception in 1998, Microsoft Research Asia (MSRA) has anointed nearly 400 top researchers with the coveted honor of “Microsoft Research Fellow.” Some of them recently returned to Beijing, sharing their insights and best practices at the inaugural Microsoft Research Asia PhD Forum, held earlier this year.

Hailing from Australia, Singapore, Korea, Japan, Taiwan, Hong Kong and mainland China, more than 30 past and current fellows convened for two days of talks on artificial intelligence and other advanced research. Also present as guests of this highly select group were the latest junior PhD students interviewing to become MSRA fellows for 2017.

“Not only did we explore state-of-the-art research, I was able to share my own research and make friends with other PhD students from different backgrounds,” noted one attendee. “Getting career advice from such accomplished researchers was very helpful.”

Talks on advanced research and technology

Wei-Ying Ma, Assistant Managing Director of MSRA, welcomed students with a brilliant talk on the “Era of Artificial Intelligence and the Fourth Industrial Revolution.” Wei-Ying pointed out that the rapid development of artificial intelligence has brought about unprecedented opportunities along with profound new challenges.

Following Wei-Ying’s talk, six researchers highlighted the latest breakthroughs underpinning current AI research: speech recognition, urban computing, wireless networks, video processing, deep learning and graphics.

Wei-Ying Ma presents at Microsoft Research Asia PhD Forum

Wei-Ying Ma – Assistant Managing Director of MSRA opens PhD Forum with his talk on the “Era of Artificial Intelligence and the Fourth Industrial Revolution”

Jumpstarting a research career

On Day 2, students received a special invitation to explore the newly released Microsoft Academic Preview along with tips to help jumpstart their research career. Microsoft Academic is an online destination that helps researchers stay abreast of developments, discover new ideas, and collaborate with others in the academic community.

A panel discussion “Turning Ideas into Reality” introduced students to the process of converting research discoveries into marketable products. Creating something new with the potential to improve lives and empower people seemed to resonate strongly with many of the students.

“I am always curious how fundamental research can impact people’s life and what it’s like to work in an industry lab,” said Mengyi Liu from the Chinese Academy of Sciences. “Their insights and sharing opened my eyes and inspired me in my future career planning.”

Tim Pan presents at Microsoft Research Asia PhD Forum

Tim Pan, Senior director of MSRA Outreach (upper-left) chaired the panel discussion; Thomas Moscibroda, Dongmei Zhang, Jonathan Tien (upper-right) were panelists for the session

Students showcase their research

The forum concluded with student-led sessions explaining their own research, prompting many great discussions.

Valerio Terragi presents at Microsoft Research Asia PhD Forum

Valerio Terragi – PhD student from The Hong Kong University of Science and Technology gave a talk on “Effective regression testing of concurrent programs”

What’s most rewarding to me is hearing back from students, who were especially appreciative of the opportunity to learn from and connect with others.  I am very proud to have organized the first MSRA PhD forum for top PhD students and look forward to hosting another one next year.

Learn more

The post Top PhD students gather at Microsoft Research Asia PhD Forum 2016 appeared first on Microsoft Research.

]]>
Interpreting visual art with sound for a more inclusive experience https://www.microsoft.com/en-us/research/blog/keith-salmon-oregon-project/ Fri, 18 Nov 2016 17:00:40 +0000 https://www.microsoft.com/en-us/research/?p=323162 By Vanessa Ho, Microsoft News Center Staff. Follow her on Twitter. A well-known Scottish landscape painter, Keith Salmon creates ethereal, moody abstracts of skylines and mountains inspired by the rugged highlands in his country. Trained in fine arts and sculpture, Salmon has broadened his techniques over the years as his eyesight has diminished, learning how […]

The post Interpreting visual art with sound for a more inclusive experience appeared first on Microsoft Research.

]]>
By Vanessa Ho, Microsoft News Center Staff. Follow her on Twitter.

Oregon Project

Artist Keith Salmon, left, and Microsoft researcher Neel Joshi stand in front of “The Oregon Project” at the King Street Station in Seattle. Photos by Dan DeLong

A well-known Scottish landscape painter, Keith Salmon creates ethereal, moody abstracts of skylines and mountains inspired by the rugged highlands in his country. Trained in fine arts and sculpture, Salmon has broadened his techniques over the years as his eyesight has diminished, learning how to smash paint into texture and scratch pastels for scribbles to evoke less a depiction of place and more an exquisite experience of the wild.

Last month, Salmon, who is legally blind, debuted an installation with innovative research from Microsoft that enriches his art even more, with proxemic audio to interpret two-dimensional images. Called “The Oregon Project,” it uses four Kinects, 15 overhead speakers and 54 soundtracks to produce an acoustic and spatial interpretation of three drawings Salmon did of the beautifully remote Hells Canyon area in Oregon. The installation premiered in Seattle at the 9e2 exhibit of art and technology as a powerful new way for Salmon to create and for people with low vision to experience and enjoy visual art.

“It would be great to think that in a few years, you might go to a national museum or art gallery and this technology would be there as part of the experience,” says Salmon, who began to lose his sight in the ‘80s due to diabetic retinopathy. With his sight limited to a small amount of vision in one eye for many years, he’s been able to adapt, paint and hike with his partner Anita Groves as his guide. You can watch the video with audio-description here.

In front of each drawing is a 3D space containing 18 soundtracks that can be played. The Kinects track movement to trigger different sounds, making the art lively and interactive for people with and without vision. From afar, observers can hear birdsong, rushing water and grass swishing in the wind — recorded in Hells Canyon by Salmon and his collaborators.

Moving closer activates digital tones matched to Salmon’s palette of blue, green, brown and ochre. Stand even closer and you can hear Salmon working in his studio as he scribbles pastels on paper, in an experience that parallels how a sighted person can see details when leaning in. The Kinects also track hand waves to change what you hear, helping observers become part of the piece and make their own acoustic mixes with movement. At the exhibit, adults triggered sounds primarily by walking, while kids jumped and waved their arms to produce a different blend of noise.

“As a way of moving from an outer to inner layer to the drawings, natural sounds to industrial sounds works really well,” Salmon says. “It has a wonderful, atmospheric, spacey, landscape-y feel that pulls the whole thing together.”

The sound system stems from an experimental project called Eyes-Free Art, created by Microsoft researchers Neel Joshi and Meredith Ringel Morris and former Microsoft intern Kyle Rector to explore how technology can help people with low vision access visual art.

Many museums offer experiences for people with visual impairment, from detailed audio descriptions of art to special events where people can touch sculptures with gloved hands. But the researchers wanted to create a more personalized experience that was still communal, and something less directed and curated by docents.

After exploring different ideas, they decided to create a Kinect-enabled sound system based on proxemics, the study of how people use space to define social interactions. And they wanted sound and visuals to be equally aesthetic and important, and rejected anything that felt like a “bolted-on” accessibility tool.

“Visual exploration is a personal experience and is based on distance — as I get closer, I see different types of details — so we wanted to mimic those ideas in audio,” says Joshi, who specializes in computer vision and computational photography. He’s also a painter and interactive-installation artist involved in the local arts community.

Kinect Neel Joshi

Microsoft researcher Neel Joshi works on the Kinect-enabled sound system of “The Oregon Project.”

Last year, researchers invited people with low vision to test a prototype built with programmed music, narration and sounds acoustically interpreting paintings by Frida Kahlo, Henri Matisse and others. The testers overwhelmingly enjoyed the experience, and one woman, an art lover who had stopped going to galleries after losing her sight, cried in joy. But the team knew the system wasn’t “art.” They knew they needed to do more.

Connecting with local artists, Joshi began working with Salmon to create a new piece with Salmon’s art and Microsoft research. They collaborated with Seattle filmmaker and photographer Dan Thornton, Scottish sound engineer Graham Byron and 9e2 organizer John Boylan to display it at the nine-day 9e2 event of installations and performances on art, science and technology. Although this particular project has been experimental and there are no immediate plans to make this specific system available broadly to the public, researchers have filed a patent for Eyes-Free Art and hope to continue building more accessible, artistic experiences.

Amos Miller, a Microsoft researcher with visual impairment who works with the company’s Accessibility group, calls the installation “inherently inclusive,” allowing him and others to experience art personally and socially, and in the way the artist intended.

“It’s evidence that you can create really positive, inclusive experiences,” says Miller, who is part of the Cities Unlocked project, which creates 3D soundscapes to help people with low vision navigate their surroundings.

For Sheri Richardson, president of Seattle advocacy group Vision Loss Connections and blind since infancy, the installation was a vibrant, sensory way to explore art from different perspectives and not rely on someone else’s interpretation. She especially liked being able to visualize the artist in the mountains with sounds from Hells Canyon.

“The technology of the Kinects and the sound moving with you as you move into the paintings really makes it experiential and brings out the emotions,” says Richardson.

Oregon Project Keith Salmon

Artist Keith Salmon in front of his drawings of Hells Canyon in “The Oregon Project”

For Salmon, whose limited vision began to further deteriorate a few years ago, Eyes-Free Art has given him hope to continue his artistic career for many more years.

“It’s been amazing,” he says. “I’ve now got a future. If my sight carries on getting worse, I can continue making art and draw with sound. It’s a case of learning and adapting.”

“The Oregon Project” will be on display at the Tent Gallery at the Edinburgh College of Art in Scotland in April of 2017.

Learn more about Salmon and “The Oregon Project” at Microsoft’s Facebook, Instagram and Twitter pages.

The post Interpreting visual art with sound for a more inclusive experience appeared first on Microsoft Research.

]]>
Events in Seoul foster artificial intelligence discussion and collaboration https://www.microsoft.com/en-us/research/blog/events-in-seoul-foster-artificial-intelligence-discussion-and-collaboration/ Fri, 11 Nov 2016 17:00:23 +0000 https://www.microsoft.com/en-us/research/?p=319232 By Miran Lee, Principal Research Program Manager, Microsoft Research Asia More than 1,700 students and faculty members attended the 18th annual Computing in the 21st Century Conference 2016 last week in Seoul. It was the second time the conference, organized by Microsoft Research Asia, has occurred in South Korea since 2007. The conference’s theme, “Human […]

The post Events in Seoul foster artificial intelligence discussion and collaboration appeared first on Microsoft Research.

]]>
By Miran Lee, Principal Research Program Manager, Microsoft Research Asia

More than 1,700 students and faculty members attended the 18th annual Computing in the 21st Century Conference 2016 last week in Seoul. It was the second time the conference, organized by Microsoft Research Asia, has occurred in South Korea since 2007.

From left to right: Marti A. Hearst, Professor of UC Berkeley; Adi Shamir, 2002 Turing Award Recipient; and Peter Lee, Corporate Vice President, Microsoft Research NExT

From left to right: Marti A. Hearst, Professor of UC Berkeley; Adi Shamir, 2002 Turing Award Recipient; and Peter Lee, Corporate Vice President, Microsoft Research NExT

The conference’s theme, “Human and Machine Working as a Team,” was accentuated in keynote speeches by Peter Lee, corporate vice president of Microsoft Research NExT, and Hsiao-Wuen Hon, corporate vice president, Microsoft’s Asia-Pacific R&D Group and Microsoft Research Asia.

Lee told attendees researchers must work to democratize access to the tremendous power of computing and algorithms. “We must democratize access to AI. We must work together to ensure that humans and machines work together as a team,” Lee said.

Hon emphasized that while computers are reaching human parity in recognizing images and speech, humans still have significant advantages over machines when it comes to creativity and reasoning. Hon emphasized that recognizing these distinctions is important, suggesting that humans can exploit AI by letting machines do what they’re good at, while focusing on the skills that are uniquely human.

Hsiao-Wuen Hon, corporate vice president, Microsoft's Asia-Pacific R&D Group and Microsoft Research Asia.

Hsiao-Wuen Hon, corporate vice president, Microsoft’s Asia-Pacific R&D Group and Microsoft Research Asia.

Other distinguished speakers shared their views on artificial intelligence and computing, as well. Turing award winner Adi Shamir discussed cryptography related to the Internet of Things (IoT); UC Berkeley professor Marti A. Hearst focused on the application of AI in education; Fred Schneider from Cornell University discussed internet security and related research; and Juliana Freire from New York University discussed her analysis of urban data by focusing on work she’s done related to New York City taxi data.

From left to right: Hsiao-Wuen Hon, Corporate Vice President, Microsoft Asia-Pacific R&D Group and Microsoft Research Asia; Adi Shamir, 2002 Turing Award Recipient; Peter Lee, Corporate Vice President, Microsoft Research NExT; Yong-Hak Kim, President, Yonsei University; Fred Schneider, Professor of Cornell University; Juliana Freire, Professor of New York University; Marti A. Hearst, Professor of UC Berkeley

Later in the week, the Microsoft Research Asia Faculty Summit 2016 was held at Yonsei University, bringing together more than 200 leading academics, educators and Microsoft researchers to discuss topics related to the theme “Intelligent and Invisible Computing.”

Yong-Hak Kim, president of Yonsei University, welcomed Summit participants, saying, “we are now living in an era where AI technology intersects with all other technologies, bringing a fundamental transformation to how we think, communicate and collaborate,” Kim said. “I am delighted and humbled to greet the most celebrated artificial intelligence scholars from around the world, gathered here at Yonsei University, for the advancement of AI technology that empowers humans to achieve more, bringing unprecedented progress to industry and academia.”

Welcome speech at Microsoft Research Asia Faculty Summit 2016 by Yong-Hak Kim, President, Yonsei University”.

Welcome speech at Microsoft Research Asia Faculty Summit 2016 by Yong-Hak Kim, President, Yonsei University”.

The summit presented a strong lineup of distinguished speakers from China, Japan, Korea, Singapore and the United States. The distinguished guests from various academic institutions helped set the foundation for additional discussions and sessions on the future of artificial intelligence. Two panel discussions on the “Future AI” and “Future Talent 2040” resulted in active discussions among the participants. In addition, attendees saw 34 technology showcase projects from Korean academics and Microsoft researchers.

The post Events in Seoul foster artificial intelligence discussion and collaboration appeared first on Microsoft Research.

]]>
Sriram Rajamani squashed bugs on road to leadership role in Microsoft’s India research lab https://www.microsoft.com/en-us/research/blog/sriram-rajamani-squashed-bugs-on-road-to-leadership-role-in-microsofts-india-research-lab/ Thu, 03 Nov 2016 13:00:47 +0000 https://www.microsoft.com/en-us/research/?p=312785 By John Roach, Writer, Microsoft Research After a handful of years writing code for the telecommunications and design automation industries, computer bugs got the best of Sriram Rajamani. He witnessed firsthand how poorly constructed code caused programs to crash and swung doors open for hackers. “I saw how difficult it was to build real software […]

The post Sriram Rajamani squashed bugs on road to leadership role in Microsoft’s India research lab appeared first on Microsoft Research.

]]>
By John Roach, Writer, Microsoft Research

After a handful of years writing code for the telecommunications and design automation industries, computer bugs got the best of Sriram Rajamani. He witnessed firsthand how poorly constructed code caused programs to crash and swung doors open for hackers.

Sriram Rajamani

Sriram Rajamani Managing Director, Microsoft Research India Lab

“I saw how difficult it was to build real software and real computer systems that work reliably and perform in a way that helps users and society,” says Rajamani, who in August became the new managing director of Microsoft’s research lab in India.

“And I started to compare that to how people build bridges. Bridges don’t fall down all the time. Why is that?”

The answer, he discovered, involves experience gained over time – the software industry is about 70 years old; people have been building bridges for millennia, long enough for a formal design and engineering process to take root and hold.

For example, civil engineers use mathematical models to verify that bridge designs will perform as advertised. “We know that before we build the bridge,” Rajamani says.

To get rid of the bugs in his code, Rajamani solicited advice from computer scientists at the University of California at Berkeley who specialize in formal methods for programming hardware and software.

“I liked what was being done there so much that I decided I would leave my job and go do a PhD on this topic,” he says.

The switch to a research career focused on building strong and reliable computer systems is emblematic of a trait that observers routinely note about Rajamani: He always picks problems to work on that generate impact.

“He tackles problems that are hard but important and whose solutions are useful and deployable,” says Jeannette Wing, corporate vice president, Microsoft Research. “It is very rare to find a researcher who can span the spectrum from basic research to scalable solutions.”

Device driver bugs

After graduate school, in 1999, Rajamani joined Microsoft’s research lab in Redmond to work on the design of bug-free software, a critical issue for the company. Windows, the flagship operating system, was running more than 90 percent of the world’s computers and it was facing criticism that it crashed too frequently.

Research suggested that many of the crashes stemmed not from the operating system itself, but from bugs in device drivers – snippets of code that control communications between the operating system and devices such as the keyboard, screen and printer.

To squash these bugs, Rajamani joined forces with Thomas Ball, an expert in program analysis who also had recently joined Microsoft’s research lab.

“We worked well together because we had a common goal, which was device driver quality, but we brought to it different perspectives,” recalls Ball.

The collaboration resulted in a so-called device driver verifier, a type of quality assurance program that checks for bugs before the drivers are run, thus heading off a driver-caused system error.

The project substantially improved Windows reliability and brought widespread recognition to the researchers within Microsoft and throughout the industry.

A magnet in India

In 2006, Rajamani was among a handful of rising stars in Microsoft’s research organization recruited to join the company’s newly opened lab in his native India. Mid-career experts in algorithms, machine learning, security, programming languages, systems and human-computer interaction were also lured to the lab.

The strategy, explains Rajamani, was to use star power to attract and retain talented computer scientists in India, bucking the trend of India’s top graduates moving to the U.S. and Europe for jobs.

Aditya Nori, for example, joined Rajamani’s group shortly after the lab opened. He’d just earned a PhD in theoretical computer science in India and read about Microsoft’s new venture in the newspaper.

“The first two years were like a second PhD for me because I switched fields,” he recalls. “And Sriram was a mentor. I learned a lot from him. He led by example and that is very important, especially for people who are young and starting off their careers.”

One of the group’s early projects, Yogi, involved improvements to the device driver verifier that enabled bug checking both before and during operation, which added another layer of protection against crashes.

Another project focused on security and privacy in the cloud, an area that Rajamani was eager to explore in order to stay on top of technology trends and adapt his research accordingly.

To tackle the problem, he collaborated with Manuel Costa, a researcher in Microsoft’s Cambridge, U.K., lab, and Sanjit Seshia, a professor in the department of electrical engineering and computer sciences at the University of California at Berkeley, to design methods for programming new so-called trusted hardware in the cloud, and verify the security of programs that run on this hardware.

As Rajamani’s earlier research did, this latest project is having impact.

“There is a growing community that is starting to work on this topic of verifying programs running on trusted hardware,” Seshia says.

Microsoft, he notes, is interested in the work as a way to provide security guarantees for its cloud service, Azure.

Rajamani’s eye for good problems, Seshia adds, also bodes well for the future of Microsoft’s research lab in India.

“That quality is what you’d like to have in a technical leader,” he says.

Unique opportunities in India

Nearly 1 billion people in India currently lack access to the Internet, but new connections are growing quickly. Researchers expect 250 million people in India to come online in the next few years.

“And then another 250 million after that,” Rajamani says, explaining that a major opportunity for the India lab is to design systems that use technology to improve the lives of these new-comers to the Internet, and build on the experience to design solutions for other markets around the world.

India, he explains, is representative of much of the world outside of the U.S. and, as a result, can serve as a testbed for the services that make technology meaningful and valuable to the next billion people.

Microsoft researchers in the India lab’s Technology for Emerging Markets group, for example, have already spent more than 10 years exploiting the country’s wide adoption of mobile phones to address needs in healthcare, education and agriculture, Wing notes.

“As new lab director, Sriram is enhancing these efforts by bringing in researchers in other areas, such as systems and machine learning, to work with Technology for Emerging Markets to create new technologies to address these concerns,” she says.

Rajamani says he is bullish on finding and tackling problems that will impact markets everywhere around the world.

“Our dream is not to just design stuff for India,” he says. “I think of India as a testbed that looks very different, but a lot of our researchers are very international, they have lived in the U.S. or Europe, and so I hope it gives a different point of view to design systems that are applicable more broadly.”

The post Sriram Rajamani squashed bugs on road to leadership role in Microsoft’s India research lab appeared first on Microsoft Research.

]]>
Bots generate video titles and tags to bring AI researchers one step closer to visual intelligence https://www.microsoft.com/en-us/research/blog/bots-generate-video-titles-and-tags-to-bring-ai-researchers-one-step-closer-to-visual-intelligence/ Mon, 10 Oct 2016 16:00:48 +0000 https://www.microsoft.com/en-us/research/?p=302027 By Winnie Cui, Senior Research Program Manager, Microsoft Research Asia When your grandma posts a video to the cloud, there it lies, lonely and unwatched, unless your grandmother has more tagging and titling skills than mine does. My nana loves taking family videos with her cellphone, but while she’s great at creating content, she’s not […]

The post Bots generate video titles and tags to bring AI researchers one step closer to visual intelligence appeared first on Microsoft Research.

]]>
By Winnie Cui, Senior Research Program Manager, Microsoft Research Asia

When your grandma posts a video to the cloud, there it lies, lonely and unwatched, unless your grandmother has more tagging and titling skills than mine does. My nana loves taking family videos with her cellphone, but while she’s great at creating content, she’s not so good at creating an audience. While my sisters and I might love to watch the clip, she’s made it practically undiscoverable.

I know your nana (and friends, colleagues, and family) is probably like mine, because a high percentage of user-generated videos stored in the cloud have very few views. Well, grandma, help is here from artificial intelligence researchers. Their research will eventually enable you to easily find and watch user-generated content, including that amazing clip of your grandpa losing his teeth while dancing at your cousin’s wedding!

Chia-Wen Lin and Min Sun, professors in the Electrical Engineering department of National Tsinghua University in Taiwan, tackled this issue with machine learning. In effect, they’ve created a system where bots watch a video, determine its highlights, create a relevant title for easy searching, and recommend who might want to be tagged to watch it.

“Our research has taken us one step closer to the holy-grail of visual intelligence, understanding visual content in user-generated videos,” said Professor Sun.

Video Title Generation System

Professor Sun created a video title generation method based on deep learning to automatically find the special moments—or highlights—in videos, and generate an accurate and interesting title for the highlights. In parallel, Professor Lin developed a method to detect and cluster the faces in videos to provide richer summaries of the videos and relevant suggestions about whom to share them with. Working together, their algorithms can detect highlights, generate descriptions of highlights and tag potential viewers of user-generated videos.

Their work was inspired by Microsoft Research’s COCO (Common Objects in Context). COCO is a new image recognition, segmentation, and captioning dataset that recognizes more than 300,000 images, in context, and because videos are essentially a succession of images, this dataset could help with video title generation. Professor Lin and Professor Sun collaborated with Dr. Tao Mei, a lead researcher in multimedia at Microsoft Research Asia in 2015, using COCO captions for sentence augmentation and using captions in MSCOCO to train their system. Their work is published on arxiv.

Currently, Professors Sun and Lin have analyzed 18,000 videos for highlights and generated 44,000 titles/descriptions. To improve the system, Professor Sun and his students participated in the VideoToText challenge sponsored by Microsoft Research, using the data released in the challenge for additional validation. Their work will be published at the ECCV (European Conference on Computer Vision) conference, October 8–16, 2016. Professor Sun and Dr. Tao Mei have started looking into the next phase of their collaboration on storytelling of personal photos.

If you’re also engaged with furthering the state of the art in visual intelligence research, you’ll find our Cognitive Services Computer Vision API useful. It extracts rich information from any image to categorize and process visual data. You can even build an app to caption your own videos using our sample on GitHub. Check it out!

Learn More

The post Bots generate video titles and tags to bring AI researchers one step closer to visual intelligence appeared first on Microsoft Research.

]]>
HoloLens future highlights wide ranging vision research at ECCV https://www.microsoft.com/en-us/research/blog/hololens-future-highlights-eccv/ Fri, 07 Oct 2016 13:00:16 +0000 https://www.microsoft.com/en-us/research/?p=302054 By Marc Pollefeys, Partner Director of Science, HoloLens and Jamie Shotton, Partner Scientist Lead, HoloLens We are pleased to announce Microsoft’s Platinum sponsorship of the 14th European Conference on Computer Vision (ECCV) in Amsterdam from October 8-16. ECCV is one of the top international conferences on computer vision research. Microsoft researchers, scientists, and engineers will be […]

The post HoloLens future highlights wide ranging vision research at ECCV appeared first on Microsoft Research.

]]>
By Marc Pollefeys, Partner Director of Science, HoloLens and Jamie Shotton, Partner Scientist Lead, HoloLens

Microsoft Research's HoloLens Director of Science at ECCVJamie Shotton, Partner Scientist Lead, HoloLensWe are pleased to announce Microsoft’s Platinum sponsorship of the 14th European Conference on Computer Vision (ECCV) in Amsterdam from October 8-16. ECCV is one of the top international conferences on computer vision research. Microsoft researchers, scientists, and engineers will be participating in the discussion with dozens of talks and posters, a workshop co-organized by Zhengyou Zhang on Computer Vision for Audio-Visual Media, and a keynote by Changhu Wang at the first workshop on Visual Analysis of Sketches.

This is a golden age for computer vision. Research breakthroughs are leaving the lab and getting into users’ hands in record time. Computer vision now plays a pivotal role in many advances benefitting society, such as autonomous vehicles, improved biometric security, and medical imaging. But out of all these innovations, one really stands out to us as having the potential to completely upend how we access information and communicate with each other: mixed reality. Spurred by recent developments in SLAM, 3D reconstruction, gesture recognition, and scene understanding, we’re already experiencing it in the form of groundbreaking products including Microsoft HoloLens.

But we’re just at the start of our journey. Many deep research questions and difficult engineering challenges remain if we are to deliver the ultimate promise of mixed reality. And so, to help invent this future, we’ve just announced the formation of a new HoloLens computer vision research team at Microsoft in Cambridge, UK. Poised to expand substantially over the coming months, we’re looking for people who love to build amazing new technology and have the strong blend of research, engineering, and mathematics they’ll need to thrive with us.

If you’re attending ECCV please stop by our booth and talk to us about computer vision at Microsoft, and opportunities in Cambridge, Redmond, and beyond.

We look forward to meeting you!

In addition to the main plenary sessions, the conference will include the following keynotes, workshops, demonstrations and exhibits by Microsoft employees.

Conference organization

  • Gang Hua, Workshop Chair
  • Sebastian Nowozin, Area Chair
  • Jingdong Wang, Area Chair

Keynotes

Workshops

Accepted Papers

Microsoft Attendees at ECCV 2016

The post HoloLens future highlights wide ranging vision research at ECCV appeared first on Microsoft Research.

]]>
Microsoft accelerates data science at The Alan Turing Institute with $5m in cloud computing credits https://www.microsoft.com/en-us/research/blog/microsoft-accelerates-data-science-at-the-alan-turing-institute-with-5m-in-cloud-computing-credits/ Thu, 06 Oct 2016 13:00:58 +0000 https://www.microsoft.com/en-us/research/?p=300734 By Kenji Takeda, Director, Azure for Research, Microsoft Research Microsoft is excited to be empowering researchers at The Alan Turing Institute to achieve more by awarding $5 million in Microsoft Azure cloud computing credits. The Turing is the U.K.’s national center for data science, with its headquarters at the British Library in London. It is […]

The post Microsoft accelerates data science at The Alan Turing Institute with $5m in cloud computing credits appeared first on Microsoft Research.

]]>
msr-allanturing-blog-900x300

By Kenji Takeda, Director, Azure for Research, Microsoft Research

Microsoft is excited to be empowering researchers at The Alan Turing Institute to achieve more by awarding $5 million in Microsoft Azure cloud computing credits.

The Turing is the U.K.’s national center for data science, with its headquarters at the British Library in London. It is a joint venture founded by the universities of Cambridge, Edinburgh, Oxford, University College London and Warwick and the UK Engineering and Physical Sciences Research Council (EPSRC).

“More than 100 research staff — ranging from computer scientists, engineers and experts in machine learning to statisticians, mathematicians and social scientists — will soon begin work at The Alan Turing Institute with the mission to advance the world-changing potential of data science,” said Andrew Blake, the Institute’s Director.

“Azure cloud services will provide our data scientists with an easily accessible platform where they can prototype ideas with a fast turnaround of results, complementing local computing facilities available in the institute’s five founding universities, and national resources such as the supercomputer ARCHER supported by EPSRC,” Blake added. “We are delighted that Microsoft is enabling access to Azure cloud services and supporting this crucial element of our research infrastructure.”

Joseph Sirosh, corporate vice president of the Data Group at Microsoft, commented: “We are proud to be working closely with The Alan Turing Institute to show how AI, machine learning and data science can be applied in novel ways to real-world problems. We are excited to be enabling researchers to do their best work by providing access to the state-of-the-art capabilities in Microsoft Azure.”

Jeannette M. Wing, corporate vice president of Microsoft Research, added: “The Alan Turing Institute is a unique place where researchers from the U.K.’s top universities come together to push the boundaries of data science. This partnership with The Alan Turing Institute is a prime example of how Microsoft is investing in the global data science research ecosystem, and we look forward to seeing the results of this collaboration.”

Turing Research Fellow Chris Russell anticipates that Microsoft Azure will prove especially useful in data science research. “We often spend a lot of time thinking and coding, and then we have a short window where we want to use a lot of computation power to immediately test our ideas, before we go back to thinking again,” Russell said. “This kind of burst in the usage of computers is a great match for the cloud where you can scale up your computational resources quickly and easily.”

Personally, I am honored to be joining The Alan Turing Institute as a Visiting Industry Fellow, to work directly with the researchers there. We will be training every researcher at the Turing in the latest approaches to data science, encouraging them to employ cloud-first thinking about their problems through access to near-infinite computing and data processing capabilities. Our Azure for Research program is seeding hundreds of academic projects in the cloud, enabling better and more reproducible data-intensive research around the world. Particularly exciting is how Microsoft Azure’s unique features are a perfect match for data science researchers, such as true high-performance computing GPUs for deep learning, Cortana Intelligence Suite (Microsoft’s data and analytics platform) with Azure Machine Learning, Jupiter Notebooks with R and Python, Azure Data Lake and Spark on HDInsight.

Suzy Moat, a Faculty Fellow at The Alan Turing Institute and Associate Professor of Behavioral Science at Warwick Business School, explained: “Our everyday use of the internet generates vast quantities of data. In my research, I’m interested in understanding whether such data can help us measure and even predict human behavior in the real world. Excellent access to cloud computing services will be of immense use in helping us turn these huge online datasets into insights that we hope will be of value to scientists and policymakers alike.”

We look forward to telling you more about the exciting cloud-first research we’re helping to enable at The Alan Turing Institute in coming months.

Learn more

The post Microsoft accelerates data science at The Alan Turing Institute with $5m in cloud computing credits appeared first on Microsoft Research.

]]>
Boosting fitness and workplace performance looks to machine learning for answers https://www.microsoft.com/en-us/research/blog/boosting-fitness-and-workplace-performance-looks-to-machine-learning-for-answers/ Fri, 16 Sep 2016 18:32:57 +0000 https://www.microsoft.com/en-us/research/?p=293258 By Miran Lee, Principal Research Program Manager, Microsoft Research A field study exploring the usability of integrating fitness equipment into a workstation environment has researchers looking to tap next-generation machine learning innovations to address the seemingly elusive challenge of burning calories without ever leaving the desk. Elliptical trainers small enough to fit under a desk […]

The post Boosting fitness and workplace performance looks to machine learning for answers appeared first on Microsoft Research.

]]>
By Miran Lee, Principal Research Program Manager, Microsoft Research

A field study exploring the usability of integrating fitness equipment into a workstation environment has researchers looking to tap next-generation machine learning innovations to address the seemingly elusive challenge of burning calories without ever leaving the desk.

Elliptical trainers small enough to fit under a desk may seem like a perfect way to stay fit and be productive at work. But until recently no one had really studied the viability of successfully doing both at the same time.

Suspecting that it’s much harder than it looks, Uichin Lee, an associate professor at the Korea Advanced Institute of Science and Technology (KAIST), teamed up with Microsoft researchers in Asia to gauge how office workers integrated elliptical trainers into their workflow.

The research, Exploring User Experiences of Active Workstations: A Case Study of Under Desk Elliptical Trainers was presented September 15 in a session during the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2016) in Heidelberg Germany.

The user experience (UX) study tracked 13 office workers over the course of a month and found they typically only used the elliptical trainers when performing simple office tasks or taking breaks. Some mostly used them as footrests or just to fidget. Although acknowledging the potential for improved health, participants noted any overall benefit comes at a cost of some leg room and “a fair amount of noise.”

But what if active workstations could be configured to fully track a user’s biometrics, workflow and other data streams to predict optimal times to work on any given task?

“Our next mission is to devise a novel persuasive computing system that can recommend the opportune moments for active workstation usage,” said Lee, who co-authored the paper with KAIST colleagues Woohyeok Choi and Aejin Song in collaboration with Darren Edge and Masaaki Fukumoto of Microsoft Research Asia.

For recommendation algorithm design, Lee and his team will consider multidimensional data, including computer usage, individual physiological/physical states and environmental sensor data. Owing to its wide variety of available sensing data, Microsoft Band 2 will be used to collect physiological/physical data such as heart rate, galvanic skin response, skin temperature and 3-axis linear/rotational acceleration. The team will continue to use Microsoft Azure to manage and analyze the multidimensional data for machine learning.

Lee, whose Interactive Computing Lab at KAIST covers topics like social computing systems and vehicular sensor networking, was surprised by the lack of UX research for active workstations. “This rapidly growing field has somehow received little attention from HCI [human-computer interaction] researchers,” Lee stated. “UX research will be beneficial to a broad range of stakeholders, including end users, researchers and device designers. Moreover, this will open up many other research opportunities in this field, such as active workstation design, workout intervention software design, and interactive workplace design.”

The project was the latest of several collaborative efforts with HCI researchers Edge and Fukumoto including the group swimming “exergame” SwimTrain.

To conduct the field study, researchers modified existing Stamina InMotion elliptical trainers to support usage tracking and goal setting features. A microcontroller was attached to the trainer to sense pedal spinning while an app displayed the total daily steps in real time with options for users to set daily activity goals. Participants were all graduate students employed in office environments with an average daily sedentary working time of 8.7 hours.

During the study, the team collected usage logs that showed “participants performed an average of 2,690.8 strides for an average 38.4 minutes on work days. Five participants failed to complete 3,000 strides per day, and the others took 43.0 minutes to meet their daily goals.”

under desk elliptical trainer

Researchers obtained direct feedback via weekly surveys and a final interview with each participant at the study’s conclusion. The interviews showed that work contexts such as the urgency of requests and their cognitive load were the main factors in choosing whether to work and pedal at the same time.

One participant commented: “I pedaled this machine while surfing the web or watching sports videos on YouTube during a rest […] I may pedal when doing familiar tasks. For example, when reading an academic paper in a familiar domain, pedaling did not disturb me. However, it was difficult to comprehend a novel or an unfamiliar paper while using this machine.”

Using the trainer to combat drowsiness was among the most popular reasons to use the trainer, a majority of participants indicated.

“After eating a meal, I felt sleepy and could not concentrate on work,” a participant indicated. “So I used the machine to stay awake.”

Learn More

The post Boosting fitness and workplace performance looks to machine learning for answers appeared first on Microsoft Research.

]]>