TechFest is an annual event that brings researchers from Microsoft Research’s locations around the world to Redmond to share their latest work with fellow Microsoft employees. Attendees experience some of the freshest, most innovative technologies emerging from Microsoft’s research efforts. The event provides a forum in which product teams and researchers can discuss the novel work occurring in the labs, thereby encouraging effective technology transfer into Microsoft products.
Project Gustav: Immersive Digital Painting | Video
Client + Cloud Computing for Research
Scientific applications have diverse data and computation needs that scale from desktop to supercomputers. Besides the nature of the application and the domain, the resource needs for the applications also vary over time—as the collaboration and the data collections expand, or when seasonal campaigns are undertaken. Cloud computing offers a scalable, economic, on-demand model well-matched to evolving eScience needs. We will present a suite of science applications that leverage the capabilities of Microsoft’s Azure cloud-computing platform. We will show tools and patterns we have developed to use the cloud effectively for solving problems in genomics, environmental science, and oceanography, covering both data and compute-intensive applications.
Learn more about this project
To make cloud computing work, we must make applications run substantially faster, both over the Internet and within data centers. Our measurements of real applications show that today’s protocols fall short, leading to slow page-load times across the Internet and congestion collapses inside the data center. We have developed a new suite of architectures and protocols that boost performance and the robustness of communications to overcome these problems. The results are backed by real measurements and a new theory describing protocol dynamics that enables us to remedy fundamental problems in the Transmission Control Protocol. We will demo the experience users will have with Bing Web sites, both with and without our improvements. The difference is stunning. We also will show visualizations of intra-data-center communication problems and our changes that fix them. This work stems from collaborations with Bing and Windows Core Operating System Networking.
Learn more about this project
Energy-Aware VMs and Cloud Computing
Virtual machines (VMs) become key platform components for data centers and Microsoft products such as Win8, System Center, and Azure. But existing power-management schemes designed at the server level, such as power capping and CPU throttling, do not work with VMs. VMmeter can estimate per-VM power consumption from Hyper-V performance counters, with the assistance of WinServer2008 R2 machine-level power metering, thus enabling power management at VM granularity. For example, we can selectively throttle VMs with the least performance hit for power capping. This demo compares VMmeter-based with hardware-based power-management solutions. We run multiple VMs, one of them being a high-priority video playback on a server. When a user requests power capping with our solution, the video playback will maintain high performance, while with hardware-capping solutions, we see reduced performance. We also will show how VMmeter can be part of System Center management packs.
Learn more about this project
The Future of Looking Back
We will examine the issue of family archiving and present a system designed to enable families to capture, manage, create, and store new kinds of digital memorabilia. The system, using Surface as its hub, shows how families can upload photos and videos quickly and easily, and also scan in physical memorabilia, such as children’s artwork or a child’s first pair of shoes. The system enables families to view these media in many flexible ways and to create new, compelling kinds of digital objects, such as multimedia scrapbooks and even a digital piñata. We further will show how this system would fit into a larger ecosystem of devices in the home and link to new kinds of media displays.
Learn more about this project
Greening Corporate Networks with Sleep Proxy
In a corporate network, most desktop machines always are left on, even when they are not in use for extended periods, such as at night. This is wasteful, bad for the environment, and bad for the corporate treasury. While Win7 provides aggressive sleep functionality, most users override it because they occasionally might want to access their machine remotely. Ideally, a desktop would go to sleep when not in use and awaken seamlessly when the user tries to access it. We have built a system to enable this. Our system consists of a “sleep server” that maintains the network presence of the sleeping machine and seamlessly awakens it on remote access. We do not require special hardware or changes to existing software. Our system is operational in Building 99 and has resulted in substantial savings in terms of money, power consumption, and carbon-dioxide emissions.
Learn more about this project
Inside the Cloud: New Cloud-Computing Interaction
With cloud computing, users can access their personal data anywhere and anytime. Cloud computing also will enable new forms of data to be provided for users, with applications ranging from Web data mining to social networks. But cloud computing necessitates new interaction metaphors and input-output technology. The cloud mouse is one such technology. Every user will have one. It will be a secure key to every user’s cloud data. And, with six degrees of freedom and with tactile feedback, the cloud mouse will enable users to orchestrate, interact with, and engage with their data as if they were inside the cloud.
Mobile Assistance Using Infrastructure (MAUI)
Mobile devices have reached an impasse. Although the resources that can be integrated onto mobile handheld devices will continue to improve, faster CPUs, more RAM, faster wireless NICs, making substantial use of these resources will require a major breakthrough in battery technology. To bypass these limitations, MAUI (Mobile Assistance Using Infrastructure) is a system that enables fine-grained offloading of mobile applications to cloud-based infrastructure. By leveraging nearby infrastructure, MAUI enables a new class of resource-intensive applications, such as augmented reality, to run on mobile handheld devices. With MAUI, we enable resource-intensive .NET applications to run on Windows Mobile smartphones. We will demonstrate: A resource-intensive face-recognition application that consumes an order-of-magnitude less energy. A voice-based translation application that previously could not run using only the limited resources available on today’s smartphones.
Learn more about this project
Mobile-Search and Advertisement-Cache Architecture
We will show how to improve the mobile-search user experience by caching popular search results on mobile devices. First, a community-based cache is created by mining the most popular queries in mobile-search logs. Over time, the cache is personalized by adding all the new user search queries. An analysis of four months of mobile-search logs shows that, on average, 66 percent of the search queries submitted by a user can be answered by caching 2,500 links on a 1MB cache. Our prototype implementation in Windows Mobile demonstrates responses 16 times faster and 23 times more energy-efficient compared with querying through a 3G link. Our prototype also demonstrates how our caching architecture can enable monetization of mobile local search without hurting the mobile user experience. A rich set of ads is first cached on the phone. Because all the ads are locally stored, finding and displaying a mobile local ad is extremely fast, enabling us to display ads instantly to a mobile user as a query is being typed.
Learn more about this project
Mobile Surface is a novel interaction system for mobile computing. Our goal is to bring the Microsoft Surface experience to mobile scenarios and, more importantly, to enable 3-D interaction with mobile devices. We will demonstrate how to transform any surface, such as a coffee table or a piece of paper, into a Mobile Surface by using a mobile device and a camera-projector system. Besides the Surface, we will show 3-D object imaging, augmented reality, and multiple-layer 3-D information visualization. In particular, we have developed a system with the camera-projector component to scan 3-D objects in real time while doing normal projection. To visualize, 3-D data can be projected onto a surface formed by a piece of paper while maintaining the original scale as if it were printed on that paper, and a user can interact with the projected content with a hand. Mobile Surface enables you to interact with digital contents and information around you from anywhere.
Learn more about this project
Mobile-to-Mobile Networking in 3G Networks
Mobile devices increasingly are first-class computing devices, generating large amounts of data. Searching and sharing data securely across multiple devices can be a significant challenge. We have built Contrail, a communication abstraction for P2P communication on mobile phones. Communication in Contrail is purely asynchronous, coping with the fact that phones temporarily are disconnected from the network. Phones set up filters with other phones expressing their interest set. We will demonstrate the usage of Contrail with three applications: P2P content distribution, P2P search, and location-based group communication.
Learn more about this project
Natural User Interfaces with Physiological Sensing
Microsoft has innovated continually in developing novel interaction modalities, or natural user interfaces. Surface and Project Natal are two examples. While these modalities rely on sensors and devices situated in the environment, we believe there is a need for new modalities that enhance the mobile experience. We take advantage of sensing technologies that enable us to decode the signals generated by the body. We will demo muscle-computer interfaces, electromyography-based armbands that sense muscular activation directly to infer finger gestures on surfaces and in free space, and bio-acoustic interfaces, mechanical sensors on the body that enable us to turn the entire body into a tap-based input device.
Learn more about the Skinput project
Learn more about the Muscle-Computer Interfaces project
New Technologies for Multi-Image Fusion
As video and still cameras have become almost ubiquitous, people are taking increasingly more photographs and videos of the world around them. Often, the photographer’s intent is to capture more than what can be seen in a single photograph, and he or she instead takes a large set of images or a video clip to capture a large scene or a moment that extends over time. One can combine these images to produce an output that improves the input images, such as creating an image with a large field of view, a panorama, or a composite image that takes the best parts of the image, a photo montage. But creating these results is still non-trivial for many users. One challenge is in creating large-scale panoramas, for which the capture and stitching times can be long. In addition, when using consumer-level point-and-shoot cameras and camera phones, artifacts such as motion blur appear. Another challenge is combining large image sets from photos or videos to produce results that use the best parts of the images to create an enhanced photograph. We will present several new technologies that advance the state of the art in these areas and create improved user experiences. For panorama generation, we will demonstrate: ICE 2.0. Stitching of panoramas from video. Generating sharp panoramas from blurry videos. For generating composites, we will demonstrate: Video to snapshots. De-noising and sharpening using lucky imaging.
Learn more about this project
OneAlbum-Find Your Photos in Your Friends' Albums
Today, my album refers to a collection of photos I have taken. But many photos relevant to me—such as photos of me or my children—are in my friends’ albums. OneAlbum automatically finds relevant photos in my friends’ albums on social networks or in shared albums, brings them to my album, and shows them side-by-side with the photos I’ve taken. For example, if I was at a party, I’ll see all photos from that party—those I’ve taken and those my friends took. The technology behind OneAlbum is a novel, unsupervised face-recognition algorithm. It analyzes the photos in my album to find automatically the faces of people I most care about, based on frequency of their appearance; no tagging is required. Then, using the social-network graph and other information, OneAlbum crawls my friend’s albums looking for photos of people that interest me. The algorithm was tested on real large-scale albums including tens of thousands of photos and achieved accuracy rates as high as 90 percent.
Project Gustav: Immersive Digital Painting
Project posterGustav is a realistic painting-system prototype that enables artists to become immersed in the digital painting experience. The natural interface makes Gustav ideal for hobbyists and professional artists alike. Gustav achieves a high level of interactivity and realism by leveraging the computing power of modern GPUs, taking full advantage of multitouch and tablet input technology and our novel, natural media-modeling and brush-simulation algorithms. Our prototype provides convincingly realistic models for pastel and oil media, with more to come.
Learn more about this project
Saving Desktop Energy: WakeOnLAN & Virtualization
Saving desktop energy has been an area of focus at Microsoft Research India over the past year. WARP and LiteGreen are complementary projects under this theme. WARP is a composition of platform components that facilitates remote power-state management of a PC using the Wake-on-LAN and other system-management mechanisms. This system is virtually stateless and provides user interfaces for system management. Features include remote peer-to-peer wakeups and remote sleep, hibernate, or shutdown of a desktop and asynchronous machine-state transitions based on events published from user interfaces such as the Web, e-mail, SMS, and location-based services. Automatic desktop upgrades, centralized system control, delegation of power management, and auditing are also components of this system. LiteGreen is a system for saving energy from idle PCs in enterprises by exploiting short idle periods as well as long ones. To avoid user disruption, LiteGreen virtualizes the desktop computing environment and migrates it between the physical PC and a virtual-machine server, depending on whether the desktop computing environment is being used. Based on usage analysis of 120 desktops at Microsoft Research India, LiteGreen was able to deliver energy savings of 72 to 74 percent. When a user steps away from a PC, the desktop is migrated to a server and the PC is put to sleep. When the user returns, he or she is able to start using the desktop immediately.
Learn more about this project
Technologies for Rural Financial-Service Delivery
We will show two solutions that enable financial-service delivery to low-income individuals in developing countries. Both integrate the use of portable digital devices with paper-based tools that cost little and are widely used in developing countries. The first solution focuses on improving microfinance-record management through the use of pen-and-paper-based input on a low-cost digital slate device. Handwritten data on paper is digitized and processed simultaneously to provide instant user feedback, delivering gains in data quality and process efficiency. The second solution addresses a security concern in mobile-phone-based banking transactions. We use paper to facilitate secure PIN entry on mobile phones and to achieve a suitable tradeoff between security and usability in phone-assisted banking.
The Translating! Telephone
We will demonstrate a system for live speech-to-text and speech-to-speech translation of telephone calls. Douglas Adams’ Babelfish inspired dreams of unfettered universal communication. Though we are still far from achieving that goal, there are scenarios in which today’s limited accuracy can create value. Our goal in the telephone-call scenario is to provide an aid for cross-language communication in the event that no other means of communication exists. The system we will show makes extensive use of speaker-adaptation technologies to achieve reasonable, real-time speech-to-text transcription accuracy. This is then translated live using machine translation to provide speech-to-text translation and further fed into a text-to-speech system to realize speech-to-speech translation. The speech-to-text transcript and the translated transcript are shown to the users to enable validation of their intentions. This system will be demonstrated by a live conversation between German and English speakers.
Learn more about this project
Inventing the Future of Computing
Since its founding in 1991, Microsoft Research has grown into one of the largest, and most respected software research organizations in the world. Today, more than 1,100 brilliant scientists and engineers — among them the brightest minds in computer science — work in 13 labs, across four continents, on a mission that has stayed the same for over 20 years: to advance the state of the art in computing through a combination of basic and applied research.The reason is simple: innovation is the lifeblood of Microsoft’s products and services. As our researchers solve the toughest problems in computing, share their ideas with the greater research community, and collaborate with Microsoft product groups, they’re ensuring that the company — and the industry — will continue to find new ways for technology to enhance our world for years to come.
To be successful, basic and applied research depends on three practices essential for any venture: building mutually beneficial relationships, maintaining effective two-way communication, and nurturing bonds of trust. We carry out these practices through collaboration with internal product teams, and external researchers.
Our labs are unique among corporate research facilities in that they balance an open academic model with an effective process for building research results into product development. This approach, unique among corporate research facilities, pays off for Microsoft as enhancements to virtually every product and service it offers and as entirely new lines of business for the company. At the same time, it produces significant technological and intellectual advances across the spectrum of computer science.
Our contributions to Microsoft are reflected in the company’s sustained investment in research and development. Unlike other companies, where research projects must be funded by individual product groups, Microsoft funds its research at the corporate level to ensure that we are looking beyond immediate product considerations to problems further in the future.
We are excited to be developing technologies that weave software, hardware, and services into every aspect of our lives, thereby advancing toward the goal of making computers genuinely useful — We see a future in which computers interact with people the way we interact with each other, think ahead to anticipate and meet our needs, and support us as we make the most of our ideas, efforts, and intellect. Developing next-generation software that improves people’s lives is stimulating work, and we look forward to the breakthroughs we’ll make in the years ahead.
Accelerating Research and Discovery
Microsoft Research is dedicated to pursuing innovation through basic and applied research in computer science and software engineering. Basic long-term research, unconstrained by the demands of product cycles, leads to new discoveries and lays the foundation for future technology breakthroughs that can define new paradigms, such as the current move toward cloud computing and software-plus-services. Applied research focuses on the near-term goal of improving products by transferring research findings and innovative technology to development teams. By balancing basic and applied research, and by maintaining an effective bridge between the two, Microsoft Research continually advances the state of the art in computer science and redefines the computing experience for millions of people worldwide.
Microsoft Research has more than 1,100 brilliant scientists and engineers, including some of the world’s finest computer scientists, sociologists, psychologists, mathematicians, physicists, and engineers, working in our worldwide locations. Microsoft researchers work across more than 60 disciplines within the following areas:
Microsoft Research collaborates with the world’s foremost researchers in academia, industry, and government to move research in new directions across nearly every field of computer science, engineering, and general science. As part of that effort, Microsoft builds deep relationships with key universities worldwide to enhance teaching and learning experiences, to inspire technological innovation, and to strengthen computer-science and engineering education. Through global and regional initiatives, Microsoft aims to accelerate research and discovery and, ultimately, to help researchers and scientists address some of the most urgent and challenging societal and technological issues facing humankind.
Microsoft researchers make breakthroughs, large and small, that enable people to use computing to do more things in more places throughout their lives and to interact with technology as naturally as they do with each other. Put more simply, Microsoft Research is building the future by turning ideas into reality.
By its nature, research is an incremental, step-by-step process. Innovations can take years to evolve. The impact of today’s idea might not be fully realized until years later, but the implications of any new technology can be far-reaching. This is why Microsoft researchers focus on advancing the state of the art in a broad range of areas, including many that don’t have commercial applications in today’s software. Researchers also put Microsoft innovation to work to advance other areas of science. Such efforts have included:
WorldWide Telescope. WorldWide Telescope is a collaborative effort between Microsoft Research and a variety of academic and governmental agencies, such as NASA, the Harvard-Smithsonian Center for Astrophysics, and the Adler Planetarium. It is available as a free resource to the astronomy and education communities with the hope that it will inspire and empower people to explore and understand the universe as never before.
Genetic Research. Along with several other research groups, the eScience Research group is collaborating with biologists to unravel some of the gene-splicing mechanisms in higher organisms and to create an improved model of evolution. Additionally, the group is helping to analyze associations between diseases and genetic variations in humans, and working with biologists to develop languages that will better describe biological systems.
Imagining What Comes Next
Microsoft Researchers are innovators whose jobs are never finished: to continue to advance the state of the art in computer science. With the freedom to pursue the subjects they are passionate about, they look for gaps in current technologies and dedicate themselves to filling them. They envision what comes next, without worrying about product-delivery deadlines. When their ideas lead to real-world applications, the researchers work with a team of technology-transfer agents or directly with product teams to weave them into Microsoft software and services. Occasionally, researchers involved with a project transfer to a product-development group where their initial ideas will take shape. Nearly every Microsoft product in the marketplace today has been influenced by this approach, including the Windows Server® System, SQL Server®, the Xbox® gaming system and Kinect game controller, MSN®, Bing™, and Windows Media®. The projects described in this section represent just a fraction of the many ways that Microsoft researchers are advancing the state of the art in computing.
Making Interactions with Computers Easier and More Natural
The Sensors and Devices Group is exploring how new forms of hardware can be combined with innovative software to enrich the user experience when interacting with computer systems of all types. The group’s projects include SenseCam, a badge-like wearable device that automatically captures images and sensor data as people move through their day, creating a “visual blog” that can then be played back at high speed to share particular experiences with friends or as a way to reflect upon a day’s activities.
As the devices that surround us become more powerful and connected — from mobile phones to in-car computers — there is greater demand for user interfaces that transcend the keyboard and the mouse. Microsoft Research has made significant advances in speech and handwriting recognition that help us interact with today’s PCs and mobile devices as we interact with each other. But the work doesn’t end there.
“Solving today’s toughest problems takes more than a good idea. It also requires sharing ideas and thinking about solutions by interacting with people at all levels to understand the problems they have and how we can solve those problems with technology.”
— Susan Dumais, Principal Researcher, Adaptive Systems and Interaction Group
The Commute UX project aims to make interacting with in-car computers easier by moving beyond today’s complex menu structures and limited voice commands to adopting a “say anything, anytime” approach to handling user input. The result is a system that is safer and more responsive in situations when drivers’ hands and eyes should be focused on the road.
The Virtual Receptionist is a “situated conversational agent” that can act as a front-desk receptionist, with a lifelike avatar interacting with visitors, giving directions, or welcoming and registering guests. This project incorporates key advances that can be applied to many other areas of computing, such as the ability to notice, recognize, and interact with multiple individuals, to understand the user’s intentions and offer helpful information, or to sense when users are frustrated or have been waiting too long.
To make teleconferencing a more realistic, comfortable experience, Microsoft researchers are developing technologies that can identify different speakers and “place” voices to make it appear they are in different parts of a room. By exploiting humans’ ability to hear voices as positioned in space, the technology makes remote communication feel natural — a significant improvement to technology with the potential to enhance collaboration and reduce the need for business travel.
Since its 2010 debut, the Kinect for Xbox 360 has been wildly successful. Advances in skeletal tracking, facial-recognition and audio technology have come together to create the fastest selling consumer electronic device to date according to Guinness World Records. The software development kit (SDK) released has offered new opportunities across widely different industries, from health care and education, and to potentially new manufacturing techniques using the advanced modeling features.
Improving Software Quality and Reliability
As computing makes its way into more aspects of our lives — and spreads across more devices and the Internet cloud — maintaining security, performance, quality, manageability, and reliability becomes an increasingly important requirement. Microsoft Research is inventing new programming tools, methodologies, and techniques to help developers build software that better meets these needs.
To help administrators quickly identify and resolve the myriad problems that can occur in today’s data centers — which can include tens of thousands of individual servers that could fail for any number of reasons — Microsoft researchers are using advanced visualization and machine-learning techniques to predict and anticipate problems well in advance. Additionally, the Microsoft Research Extreme Computing Group (XCG) is looking to reinvent hardware and software technologies for the future of Cloud Computing. This work will greatly improve energy efficiencies, quality control processes, plus datacenter startup and operating costs.
The Rigorous Software Engineering team is working to improve the quality of software tools by combining traditional program analysis with statistical methods. The approach is able to identify security vulnerabilities, generate specifications for APIs, point to potential root causes of bugs, and find relevant information from bug repositories and version control during debugging. In initial testing, one tool built using the approach has demonstrated an ability to identify more potential vulnerabilities than existing tools and processes.
Other Microsoft Research projects advance design, development, debugging, develop formal techniques and models to better understand programs, programming abstractions, and languages; and improve the methods, notation, and tool support for high-level system design and analysis. A number of Microsoft researchers are conducting security-related work that includes researching new encoding methods and applications to enhance privacy and security, collaborating with standards bodies to develop security protocols, and providing internal security consulting on Microsoft products.
Improving How Systems Store, Retrieve, and Present Information
Our lives are more data-driven than ever, yet computing hasn’t achieved its full potential to help us manage, protect, visualize, and understand that data. For businesses, it is important to identify quickly the information that matters and make the right decisions in real time. Individuals want to manage their photos, contacts, and communications easily and intuitively. Microsoft researchers are exploring ways to design systems, architectures, and components that can help computers manage and present data in more useful ways, while also preparing us for an era when everything will be digital.
“Unlike other industrial research labs, Microsoft Research conducts a lot of external research programs. I wished to keep publishing papers, going to conferences to meet with people, and visiting universities. Microsoft Research Asia allows all those things, and that provides me with an excellent research environment.”
— Yasuyuki Matsushita, Lead Researcher, Microsoft Research Asia
The Web Search & Data Mining Group is dedicated to improving how people locate, analyze, organize, retrieve, and visualize information. Although most Web-search methods simply rank and retrieve data from documents, this group uses methods that delve much deeper, searching at the object level to transform that data into knowledge and useful insights.
The Socio-Digital Systems Group combines psychology, sociology, computer science, and hardware engineering to design technology capable of supporting people throughout their lives — such as by building digital “timelines” to capture memories and experiences, extending the reach of today’s social networks to people who can’t or don’t use a PC and helping people create rich digitalmedia collections.
OCRLess is technology that improves the reach and quality of multilingual search by converting user queries into images and then matching them against scanned documents. By eliminating the scanning and recognition of source documents, this technology increases the volume of searchable information in many different languages, including Arabic, Chinese, Hebrew, and even hieroglyphics.
Exploring Tomorrow’s Problems
Thirty years ago — when mainframe computers cost millions of dollars and filled entire rooms — only a few dreamers imagined that computers several orders of magnitude more powerful would fit into our pockets. As technology continues to evolve — including the advent of multicore and many-core processors, and continued progress toward quantum computing — the computers we’ll use decades from now will be just as different. Microsoft researchers are engaged in some of the tough theoretical problems that need to be solved before we can take advantage of tomorrow’s technology and make it truly useful.
The Theory group at Microsoft Research is engaged in fundamental problems in mathematics and theoretical computer science, including probability theory, combinatorics, statistical physics, metric geometry, fractals, algorithms, and optimization — all of which have the potential to help computers work faster, solve problems, and make decisions.
The Algorithms Research Group is working in advanced areas such as streaming algorithms, directed data sampling, machine learning, approximation algorithms, and integration of numerical and combinatorial methods. Its aim is to be at the forefront of conceptual breakthroughs in the way computers can work with massive amounts of data.
Microsoft Research XCG’s research spans several critical areas in computing research including cloud computing hardware and software, topological quantum computing, operating system and application software focused on heterogeneous many-core computers and all aspects of computer security from the mathematics of cryptographic primitives and protocols through system design supporting practical privacy, integrity and confidentiality in heterogeneous trust environments. A hallmark of the group is its ability to employ broad investigative techniques from the development of fundamental new theory through development of practical prototype systems that concretely demonstrate potential technology impact in Microsoft products.
Breakthroughs, Large and Small
Microsoft Research takes a long-term view on innovation. No matter how far-reaching their projects might be, Microsoft researchers consistently strive for results that will solve real-world technology challenges.
Supporting Advances Throughout the Sciences
Advances in computer science don’t just benefit computing. They play a growing role in helping scientists in all fields make new contributions, facilitating discovery and advancing the state of the art in everything from health care to education to the environment, while also addressing some of society’s most pressing challenges. Since 1998, Microsoft Research Connections has led the organization’s collaborative projects with leading scientists and researchers in academia, government, and industry to apply the latest innovations in software to scientific challenges in other fields. The division also creates open tools, technologies, and services to support every stage of the research process. It also supports the aspirations and potential of early-career researchers and promising young scientists around the world through internships, scholarships, and awards. Below is a sampling of how Microsoft researchers and groups are engaging with the broader scientific community:
Microsoft Research collaborates with medical researchers at the Ragon Institute of MGH, MIT, and Harvard; the Fred Hutchinson Cancer Research Center; the University of Washington; and the University of Oxford to design an effective vaccine for HIV/AIDS, in which techniques for machine learning are used to identify the most vulnerable parts within the immune system. Researchers from the Pontificia Universidad Católica de Chile and the Instituto Politécnico Nacional in Mexico collaborate with Microsoft Research on a project to use computer-vision technology to improve quality control in the food industry — a key part of the Latin American economy.
In the Swiss Experiment, an international collaboration, scientists are using Web-based tools developed by Microsoft researchers to view huge quantities of environmental-sensor data more efficiently and in richer detail than ever.
Microsoft Research Asia’s eHeritage program works with academic partners throughout the region to apply the latest technologies to aid the preservation, interpretation, and dissemination of cultural and natural heritage for research, education and protection. The program has funded more than 10 research proposals over the past several years, from universities such as the University of Queensland in Australia and the University of Tokyo.
In India, Microsoft teams with the Molecular Biophysics Unit at the Indian Institute of Science on a project called From Genomics to Function. Identifying a number of target organisms, such as pathogens, that play a key role in the lives of humans, researchers are using computers to learn more about how the organisms’ genes relate to their functions.
Researchers at the University of Queensland and a consortium of government, academic, community, and environmental groups collaborate with Microsoft Research on the Health-e-Waterways project, which is developing data-management tools to help water-resource managers to react to changes in water quality, quantity, and aquatic species.
Microsoft has also established more than 30 joint research institutes to support innovative research projects. For example, Microsoft Research opened three in Europe: the University of Trento Centre for Computational and Systems Biology, which uses programming language theory to design new computational tools for biology; the Microsoft Research-INRIA Joint Centre, which focuses on longterm research into formal methods and computing security; and the BSC-Microsoft Research Centre in Barcelona, which focuses on the design and interaction of next-generation processors.
Other global joint research institutes include the Games for Learning Institute at New York University, which seeks ways to use computer gaming to improve instruction in science, math, literacy, and other academic disciplines. We also spearhead Microsoft’s academic multi-core relations through the Universal Parallel Computing Research Centers, a joint activity between Microsoft and Intel that supports client multi-core research at the University of Illinois at Urbana-Champaign and the University of California at Berkeley.
The Microsoft Institute for Japanese Academic Research Collaboration is enabling faculty and students in one of the world’s leading economies to combine their tradition of technological development and product innovation with Microsoft’s computer-science expertise.
Investing in People
Microsoft operates the largest Ph.D. internship program in the information technology industry. Each year, nearly 1,000 top computer-science students have the opportunity to work at one of Microsoft Research’s locations around the world. A variety of Ph.D. fellowship and scholarship programs have benefited hundreds of students worldwide over the past decade. In 2008, Microsoft awarded scholarships to nearly 80 Ph.D. students, including 10 through our Graduate Women’s Scholarship program. We support young computer scientists through an assortment of summer schools, student clubs, visiting professorships and awards. Microsoft Research runs numerous academic conferences and workshops where students and university faculty members can interact and exchange ideas with top computer-science researchers in academia, government, and industry. In 2000, the first Microsoft Research Faculty Summit drew 150 participants; since then, more than 25,000 scientists, academic researchers, faculty members, and students have attended Microsoft Research-sponsored summits, conferences, and workshops held in Asia, India, the United Kingdom, and Latin America.
Bringing Together the Greatest Minds in Computer Science
Walk through the halls of Microsoft’s research labs, and you’ll find yourself rubbing shoulders with many of the world’s finest computer scientists, sociologists, psychologists, mathematicians, physicists, anthropologists, medical doctors, and engineers. They represent one of the most diverse, influential brain trusts ever assembled, and their contributions are making an indelible mark on the world’s computing research community. They chair major conferences and lead research committees. Academic peers regularly select more of their research papers than those of any other organization for publication at major conferences, such as the Association for Computing Machinery’s (ACM’s) SIGCHI, SIGIR, SIGMOD, and SIGGRAPH.
Among Microsoft Research’s great minds are the co-creators of the laser printer, the Ethernet local-area network, and the Mach operating system. There are winners of the Kyoto Prize in Advanced Technology and the A.M. Turing award — two of the most prestigious awards presented to computer scientists. There are fellows of the Royal Society and winners of other prestigious prizes, including the National Medal of Technology and Innovation, the Charles Stark Draper Prize, the Fields Medal given by the International Mathematical Union, the IEEE John von Neumann Medal, the IEEE Emanuel R. Piore Award, the Gödel Prize, the Computer Graphics Achievement Awards from SIGGRAPH, the SIGCHI Academy Award, and even Academy of Motion Picture Arts and Sciences awards. One Microsoft researcher has been knighted by the Queen of the United Kingdom.
As diverse as their accomplishments and areas of research are, virtually all Microsoft researchers share a passion to see their innovations improve the lives of people around the world. Ask them why they joined Microsoft Research, and most will point to the widespread use of Microsoft products and the company’s dedication to basic research, which enables them to pursue their interests and helps their ideas reach more people in more places than any other company or research organization.
A sampling of the all-star team of researchers Microsoft has assembled includes:
Chuck Thacker, a technical fellow at Microsoft Research Silicon Valley. Thacker has been an innovator in computing since the 1960s. Before joining Microsoft in 1997 as director of Advanced Systems in the Microsoft Research lab in Cambridge, England, Thacker worked for the Xerox Palo Alto Research Center, and the Digital Equipment Corp. (DEC) Systems Research Center. He served as project leader of the MAXC time-sharing system and as chief designer on Alto, the first personal computer to use a bit-mapped display and a mouse for its user interface. Thacker is also co-inventor of the Ethernet local-area network, the first laser printer, the DEC Firefly multiprocessor workstation, and the AN1 and AN2 networks. Thacker has received many honors for his work, including the 2004 Charles Stark Draper Prize, for his contribution to development of the first networked distributed personal-computer system, and the 2007 John von Neumann Medal, for his central role in the creation of the personal computer and the development of networked computer systems.
Yi Ma, research manager of Microsoft Research Asia’s Visual Computing Group, which focuses on research into computer vision, pattern recognition, machine learning, and image and video editing. Ma and his team are advancing the state of the art in areas such as highly accurate face detection, tracking, alignment, and recognition. By designing systems that can adapt to different lighting conditions and variations in poses and expressions, Microsoft researchers can contribute to improvements in areas such as human-computer interaction, graphics and animation, security and surveillance, and desktop photo management. Ma says: “My main research interest is in finding the most pertinent mathematical principles for analyzing and understanding high-dimensional sensorial data such as images so that machines and humans can make more intelligent decisions.”
Dan Morris, a researcher in the Computational User Experiences group at Microsoft Research Redmond. An accomplished musician, Morris has a special interest in creativity and the arts and is part of a team that created Songsmith, a Microsoft application that enables novice songwriters to create original songs with full musical accompaniment simply by singing into a PC microphone. With a background in surgical simulation, cyber kinetics, and clinical neural prosthetics, Morris also is researching physiological signal analysis, as well as alternative input devices and interfaces for improved ergonomics.
Venkat Padmanabhan, a principal researcher at Microsoft Research India who leads the Mobility, Networks, and Systems group. Padmanabhan conducts a broad range of research on network systems, including the intersection of sensor systems and mobile computing, wireless networking, RFID, geolocation, network tomography, and peer-to-peer networking. He developed the ProbeGap bandwidth-probing tool and the GeoCluster technique for IP-location mapping, both of which have been incorporated into Microsoft products. He also helped develop the RADAR WLAN-based user localization system, which was made available through the company’s technology-licensing program.
“Computer science is all about possibilities. I chose this field because I wanted to do things that haven’t been done before—to solve life’s really big challenges. Microsoft Research gives me the opportunity to accomplish this goal.”
— Mary Czerwinski, Research Area Manager, Visualization and Interaction for Business and Entertainment Research Group
Steve Hodges, manager of Microsoft Research Cambridge’s Sensors and Devices Group, a cross-disciplinary team working to deliver compelling new user experiences in computing. Hodges and his team have made significant contributions to surface computing, which turns an ordinary tabletop into a vibrant, interactive surface and provides effortless access to digital content through natural gestures, touch, and interaction with physical objects. Hodges says the ultimate goal of his research is “to better understand how advances in technology will impact traditional computing and the ways in which people use and interact with computing devices.”
Wei-Ying Ma, assistant managing director at Microsoft Research Asia. Ma’s team of researchers has become a widely recognized global powerhouse in search, data mining, and multimedia information retrieval. His team has transferred key technologies into Microsoft’s search and online-service products. In the mid-1990s, Ma developed Netra, one of the first image-retrieval systems and still regarded as one of the most influential in the industry.
Merrie Morris, a researcher in the Adaptive Systems and Interaction group at Microsoft Research Redmond, focusing on human-computer interaction and computer-supported cooperative work. Morris’s research interests include facilitating small-group collaboration on Web-search tasks and exploring the use of horizontal interactive displays for a variety of entertainment, educational, and productivity applications. Morris is also an affiliate faculty member in Computer Science and Engineering at the University of Washington and has authored or co-authored numerous publications on topics such as collaborative research and surface computing.
Leslie Lamport, principal researcher at Microsoft Research Silicon Valley. Lamport is internationally recognized as an expert in distributed algorithms, and Lamport’s research contributions over the years have laid the foundation for distributed systems. During the 1980s, he created LaTex, a document-markup language and document-preparation system widely used by mathematicians, scientists, engineers, philosophers, economists, and other scholars. Lamport, a member of the National Academy of Engineering, is the recipient of the Edsger W. Dijkstra Prize in Distributed Computing, the IEEE John von Neumann Medal, the ACM SIGOPS Hall of Fame Award, and the IEEE Piore Award, among many other honors.
danah boyd, a senior researcher at Microsoft Research New England and a fellow at the Harvard Berkman Center for Internet and Society. One of the world’s leading experts on social networking, especially among young people, boyd is doing breakthrough work on the social and cultural effects of these technologies. Shortly after joining Microsoft in January 2009, boyd wrote about the balance between basic and applied research at the company: “Microsoft Research is hands down the most impressive research institution I’ve seen. Even though my research has product implications, I’m not a product person, but I love being in a place where my work funnels into products.”
Luca Cardelli, principal researcher, joined Microsoft Research, Cambridge, in 1997 as an expert in the design and properties of programming languages. He built the first compiler for ML which is the direct forerunner of the groundbreaking F# language that has been developed by Microsoft for Visual Studio 2010. His work on ML and Modula-3 also laid the foundations for the near-universal acceptance today of type-safe programming languages like C# and Java. His visionary language-based approach to Systems Biology has positioned Microsoft Research as a pioneer in this booming field, in which insights from programming languages are used to help tame the prodigious complexities of modelling biological systems.
Sriram Rajamani, assistant managing director of Microsoft Research India and the leader of the Rigorous Software Engineering group. Rajamani’s group works to improve programmer productivity by bringing rigor to all aspects of software development. He uses his firsthand experience in the realities of commercial software development to guide his choice of problems and approaches to research in software productivity. Before joining Microsoft Research, Rajamani managed the company’s Software Productivity Tools group, and he and his team developed projects such as SLAM, Static Driver Verifier, Behave!, and Zing.
Michael Cohen, a principal researcher in the Interactive Visual Media Group. Before joining Microsoft Research Redmond, Cohen served on the Computer Science faculty at Princeton University and the Architecture faculty at Cornell University, where he did research on radiosity, which enables more realistic rendering of light and shadows by tracing the energy emitted from light sources. Cohen was also an adjunct faculty member at the University of Utah, focusing on space-time control for linked figure animation. The co-author of a book, Radiosity and Image Synthesis, Cohen won the 1998 SIGGRAPH Computer Graphics Achievement Award for his work on radiosity. At Microsoft, Cohen’s research on image-based rendering includes the Lumigraph, which captures the flow of light and enables the re-creation of an image from multiple angles, and computer applications for photography.
Rakesh Agrawal, a Microsoft technical fellow at Search Labs in Silicon Valley. Search Labs is developing new technologies that enhance the user experience in Internet search and services. Agrawal is well known for developing fundamental data-mining concepts and technologies and for pioneering key concepts in data privacy, including the Hippocratic Database, Sovereign Information Sharing, and Privacy-Preserving Data Mining. Agrawal has been granted more than 55 patents and has published more than 150 research papers, many of them considered seminal. His work has been honored with the ACM SIGKDD First Innovation Award, the ACM SIGMOD Edgar F. Codd Innovations Award, the ACM SIGMOD Test of Time Award, and many others. In 2003, Scientific American named him one of the top 50 scientists and technologists.
Dongmei Zhang, senior researcher and research manager of the Software Analytics Group at Microsoft Research Asia. Zhang has done pivotal work in digital ink and digital media. Her Handwritten Math Equation Recognizer makes it easy to convert a complicated handwritten mathematical equation into an accurate, digitally rendered representation that can be used in a report or a presentation. She and her team also developed East Asian Language Recognition technology, which improves digital recognition of handwritten text in traditional Chinese, simplified Chinese, Korean, and Japanese.
Cynthia Dwork, distinguished scientist at Microsoft Research Silicon Valley. Dwork has made fundamental contributions to complexity theory, distributed computing, and cryptography. She has developed powerful new privacy definitions and applied them to a wide range of data analysis problems. A recipient of the 2007 Edsger W. Dijkstra Prize in Distributed Computing and the 2009 PET Award for Outstanding Research in Privacy Enhancing Technologies, Dwork is a fellow of the American Academy of Arts and Sciences and a member of the National Academy of Engineering.
A.J. Brush, a senior researcher in the Visualization and Interaction for Business and Entertainment group at Microsoft Research Redmond. Brush’s work focuses on how technology can help people with everyday problems, such as reducing e-mail overload and enabling families to coordinate their schedules and stay connected. Her research interests include human-computer interaction, with a focus on ubiquitous computing and computer-supported cooperative work. She builds technology to support asynchronous collaboration in small groups and explores non-traditional ways for people to interact with technology — to move beyond the mouse and keyboard.
Bringing Innovations to Life
Innovations from Microsoft Research make their way into the marketplace through a variety of mechanisms, including a dedicated technology-transfer team, product incubations, IP licensing, and the sale of Microsoft Research products through the online Microsoft Store.
“It’s all about exchanging ideas and solving problems. with the cinematic quality of a movie, the control of a video game, and the detail of the real world. The best technology transfer occurs when the problems can meet the solutions.”
— Philip Fawcett, Principal Research Program Manager, Microsoft
We believe that successful technology transfer is a social process, not a mechanical or logistical one. It depends on people — relationships, communication, and trust. With established relationships and ongoing communication and collaboration between researchers and product teams, technology transfer can happen naturally. New ideas arise during integration meetings, and technologies born in Microsoft Research move to the company’s product and incubation teams. Once the technology-transfer process begins, close collaboration between research and product teams ensures that those ideas effectively translate into products and services that can make computers more useful, reliable, and fun.
This process delivered fundamental advances underlying many of Microsoft’s current products and services, including the natural input capabilities of the Tablet PC, graphics and artificialintelligence technology that makes Xbox games more challenging and realistic, data-mining capabilities within SQL Server that businesses can use to transform data into insight, and media-compression technologies that help Microsoft products deliver rich Internet audio and video experiences.
Recent examples of the contributions Microsoft Research has made to the company’s products and services include:
Kinect for Xbox 360. Microsoft Research work played a significant role in the success of Kinect, working on solving the complicated task of teaching a machine how to recognize people when they change poses, frown or smile, have shadows across their face, or are brightly lit. Kinect Identity, as the device’s player-recognition tool set is called, recognizes people by looking for three visual cues: the player’s height, color of their clothing and their faces. By any standard, Kinect for Xbox 360 has proved to be a technological sensation. Kinect, the controller-free interface that enables users to interact with the Xbox 360 with the wave of your hand or the sound of your voice, sold 8 million units in its first 60 days on the market, a figure that makes it the fastest-selling consumer-electronics device in history, as confirmed by Guinness World Records.
Windows 7. The latest version of Microsoft’s flagship operating system incorporates a number of “under the hood” technologies from Microsoft Research that contribute to the product’s improved performance, reliability and usability. These include enhancements to handwriting recognition, graphics performance, networking and security. Additionally, the product development team made use of a number of Microsoft Research technologies to write and test code efficiently and more thoroughly identify potential security issues.
Maps. The traffic functionality in Microsoft’s Bing™ Maps uses a Microsoft Research technology called ClearFlow, which employs statistical methods to infer surface-street speeds by considering real-time speeds on highways, the properties of surface streets, and detailed geometric relationships between them. Additionally, MapCruncher technology enables users to build interactive mashups of different maps by establishing correspondences between key features in user-provided maps with their equivalents on Bing Maps.
Search and online services. Microsoft Research contributes advances that make the company’s search engine and online services more efficient, responsive, and useful. This includes new algorithms that vastly improve the relevance of search results; opinion-mining technology that crawls the Internet to detect, summarize, and present user reviews in a structured way; fast and space-efficient algorithms that cluster users and documents into buckets to provide personalized recommendations for MSN users; query categorization technology that helps content creators research the keywords users type on Bing™ Search; and the cashback strategy, a new business model that lowers prices for products users buy through Bing Search and creates new revenue opportunities for merchants.
Photosynth®. Microsoft Photosynth automatically reconstructs a three-dimensional space from a collection of photos of a place or object. Built on groundbreaking research conducted in collaboration with the University of Washington, this technology helps people share the places and things they love with the cinematic quality of a movie, the control of a video game, and the detail of the real world.
The Best Minds in the Right Places
Microsoft Research has expanded over the years to eleven labs worldwide and a number of collaborative projects that bring together the best minds in computer science to advance a research agenda based on their unique talents and interests:Microsoft Research’s original lab at the company’s headquarters in Redmond, Wash., now houses more than 300 researchers and staff working across every discipline in computer science. The lab also includes product-management and technology-transfer teams that ensure that the innovation and ingenuity from each worldwide lab is reflected in the company’s products and services.
Microsoft Research Cambridge was established in 1997 with three researchers. The city of Cambridge, England, was the clear choice for the location of Microsoft’s first overseas lab, because of its world-renowned reputation and rich history as a center of learning together with its more recent fame as the heart of “Silicon Fen”. Today, more than 100 researchers from throughout Europe work here on programming languages, machine learning, computer vision, human computer interaction, operating systems, networking, and computational science.
Microsoft Research Asia, Microsoft’s fundamental research arm in the Asia Pacific region, conducts curiosity-driven research in areas such as natural user interfaces, next-generation multimedia, data intensive computing and search and online ads. By attracting the best talent from Asia and the world, Microsoft Research Asia, created in 1998, has grown into a world-class research lab with over 250 researchers and developers.
Established in 2001 on the Microsoft campus in Mountain View, Calif., Microsoft Research Silicon Valley employs more than 70 researchers who focus on fields that drive the state of the art in distributed computing, including privacy, security, protocols, fault tolerance, large-scale systems, the Web, concurrency, computer architecture, and related theory.
Since 2005, Microsoft Research India has been engaged in cutting-edge basic and applied research in algorithms, cryptography, systems, natural language processing, software engineering and the role of technology in socio-economic development. In addition to innovating and contributing key technologies to Microsoft products, the lab collaborates with a wide range of scientific and academic institutions to advance the state of the art in computing research in India.
Opened in 2008, Microsoft Research New England is pursuing new, interdisciplinary areas of research that bring together computer scientists and social scientists to understand, model, and enable the computing and online experiences of the future. The group’s work includes applied projects in areas as diverse as economics, social media, and health care, as well as theoretical projects in areas such as mathematics and cryptography. Additionally, the lab’s proximity to MIT, Harvard and other institutions creates further opportunities for collaboration within and outside the field of computer science.
Based in Redmond, Microsoft Research Extreme Computing Group (XCG) is tasked to go beyond incremental innovation to advance the state of the art in computing. Collaborating cross-company to solve the most challenging, strategic problems, the team accomplishes this goal through basic and intentional research, advanced prototyping and industry partnerships. Microsoft Research XCG works to accelerate change in hardware, systems, platforms and applications. By drawing on a broad base of explorations, both internal to Microsoft and through external collaborations with the government, academia, and private companies, the group targets strategic opportunities for innovation and translates ideas into actions by prototyping as well as discovering and inventing.
Also based in Redmond, Washington, Future Social Experiences (FUSE) Labs is an applied research team that ideates, develops, and delivers new social, real-time, media-rich experiences for home and work. FUSE Labs experiences give users new ways to create, connect and collaborate with the people, information and ideas that matter to them.
To help promote collaborative research and technology development, Microsoft established the Advanced Technology Labs (ATL) Europe in Munich, Germany, in 2003. Researchers at the facility work on projects in complex event processing and streaming data, and focus on the boundary between hardware and software, especially as it relates to the ARM architecture. As an applied research laboratory, ATL Europe makes significant contributions to both research and engineering groups at Microsoft. At the same time, ATL Europe continues to engage with industrial partners to ensure its innovations satisfy proven needs.
Advanced Technology Labs (ATL) Cairo, established in 2006, houses more than two dozen researchers, developers, and staff working on applied research and development projects that cover information retrieval (particularly in the Arabic language), collaboration technologies, and browsing and searching multimedia content on PCs and mobile devices. The center not only develops innovations suited to the Middle East and Africa, but also fosters and nurtures a strong local talent base for the region.
Advanced Technology Labs (ATL) Israel is a unique group of engineers, researchers, and user experience (UX) designers who focus on providing new ways to interact, explore, and enhance the online experience. ATL Israel works with product and research teams to ideate, develop, and implement new technologies and solutions for online services, computer vision, natural user interfaces, and social data mining.
Making Computers More Capable and Useful
The distinguished researchers and scientists at Microsoft Research help to shape the computing experience of millions of people worldwide. The innovations they create enhance virtually every product Microsoft releases and influence the work of many other companies.
Thanks to Microsoft Research advances …
Computers are learning to adapt to user behavior and to anticipate human needs and preferences.
E-mail services can delete as much as 90 percent of spam before it hits users’ inboxes.
Computers automatically can reconstruct a 3-D image from a collection of 2-D photos, and it is easy to combine individual photos to create seamless collages or striking panoramas.
Hours of digital home movies can be edited quickly and easily into professional-looking episodes, with tools that automatically group together related scenes, extract key frames, and synchronize images to music.
Individuals can use their voices to search the Web on their phones and use text commands to search through audio and video.
Internet searches provide answers to questions, not just links to Web sites.
VirtualWiFi software in Windows 7 allows users to connect to multiple wireless networks at the same time, enabling them to surf the Web, play games, or perform other online tasks simultaneously.
Medical researchers are using computational biology tools and high-performance computing to accelerate their work to develop effective vaccines for complex infections and diseases such as HIV, malaria and hepatitis C.
Programmers have tools that enable them to ensure the accuracy and enhance the security of the millions of lines of code in today’s increasingly complex software.
Databases can employ probability and pattern detection to help retailers identify customer preferences and sales trends — or to enable scientists to detect the faintest undiscovered stars within millions of astronomical images.
Users can take a break from typing and instead use speech, touch, or handwriting in digital ink.
On-screen graphics are becoming so realistic that it will soon be hard for gamers to distinguish between the images on their Xbox video games and those they see outside their windows.
TechFest 2010 Keynote Address
Inside the Cloud: New Cloud-Computing Interaction
Natural User Interfaces with Physiological Sensing
New Technologies for Multi-Image Fusion
Project Gustav: Immersive Digital Painting
Rick Rashid introduces TechFest 2010
TechFest 2010: Project Gustav
The Translating Telephone