2014 Outstanding Technical Leadership
With the presentation of the Outstanding Technical Leadership Award, the Microsoft Technical Community Network recognizes Harry Shum for his transformative impact on Search Development at Bing in OSD.
When Steve Ballmer handpicked Harry Shum to lead the then-struggling Bing search quality team in 2007, Shum considered himself a “surprising choice.” As we shall see, this faith in Shum was well founded: In relatively short order, Shum righted the ship and spearheaded a series of science and engineering enhancements that would turn Bing into a viable consumer alternative with a rapidly increasing search query share in the U.S.
Nevertheless, when asked about his achievements, Shum downplays his role. In typical leadership fashion, he would much prefer the spotlight shine on his team. “I think the Bing search quality team has done a fabulous job putting search quality on a par with Google over the past few years—actually beating Google on a number of measures. This achievement really belongs to the entire team.”
By his own admission, Shum had very little experience in running a large-scale engineering team at the time Ballmer reached out to him. Having earned a Ph.D. in robotics from Carnegie Mellon University, Shum joined Microsoft Research in 1996. After two years in Redmond, he moved to Beijing as a founding member of Microsoft Research China. Over the span of nine years he would climb the ranks, starting as a researcher and ultimately rising to the role of managing director for the by-then renamed Microsoft Research Asia (MSR Asia)—not to mention Distinguished Engineer of Microsoft Corporation.
During his 3½-year tenure as managing director, Shum became increasingly cognizant of the looming importance of search and online advertising. This ultimately led Shum to make an interesting proposal to Yusuf Mehdi, who was then leading MSN search and ads team. “I suggested to Yusuf that I would like to build a search team for him,” recalls Shum. “We would essentially work for the search team in Redmond, but I would run it out of Beijing by leveraging MSR Asia.” It was an unusual proposition, and Mehdi boldly decided to go for it.
Shum’s resulting Search Technology Team (STC) set the tone for joint collaboration between Search product team and MSR that Shum would continue in the future. “It probably had some influence on Steve’s decision to have me take on the search job,” says Shum.
The Bing Search Development team in Redmond numbered roughly 300 when Shum assumed responsibility in 2007 as corporate vice president. Shum laughingly notes that his lack of traditional software engineering experience probably made him “more bold” than other colleagues, which was maybe both good and bad. “It took me a couple of years to really learn a lot of the basics,” he says. “And I got very lucky because I had great support from my manager Satya [Nadella] and had some very strong direct reports.” In particular, he calls out Technical Fellow Darren Shakib, instrumental in creating elements of the cloud infrastructure used by Bing and, later, the entire company. Also Gaurav Sareen, who ran the index serving team and is now one of Shum’s successors as corporate vice president in Search Development.
Shum knew that getting things on track at Bing would mean reformulating their approach. “We realized that search is as much a science problem as an engineering problem,” he explains. “You have to reformulate the search quality problem as a science problem or research problem.”
Identifying search ranking as the crux of the search quality challenge, the team collaborated with MSR in pursuing a solution based in machine learning—essentially applying the problem to a sort of machine-based neural network. This was different from the rule-based methodologies used by others in the industry at the time.
The ultimate result was FastRank, initially proposed by Chris Burgess and his colleagues at MSR Redmond, a machine-learning algorithm using boosted decision trees that reduced training time exponentially (from a hundred hours to one) and could be tuned for rapid experimentation. Another advancement was AEther, an experimental platform that enabled engineers to combine existing models and build on each other’s work. More recently, the Bing team and MSR Asia together are bringing forth a new index serving system called Tiger, providing significant index serving performance improvement.
The overall list of collaboration examples between Bing and MSR implemented under Shum’s leadership is too broad to describe in detail. It includes the Kirin store, mobile search improvements, smart motion thumbnail in Bing Video Search, active answers, relevance measurement (pSkip), web N-gram services for broad research community, and many more.
The results have been impressive. Using internal metrics and blind tests, Shum’s team came up with a formula for measuring and scoring search results. According to Shum, their scores finally caught up with Google about three years ago.
Shum says he is now confident in describing Bing as a credible consumer alternative to Google. It seems consumers increasingly agree: according to ComScore, Bing’s search query share is now 18.2 percent in the U.S., more than doubled since its launch. Evidence can be found at Bing It On (bingiton.com), a site offering an impartial comparison of search results between Bing and Google, akin to the Pepsi Challenge blind taste tests.
In addition to these performance metrics, Shum points to his team itself as further evidence of success. “I feel incredibly proud that we have built this very strong Bing engineering team,” says Shum. “Over six years we have grown one technical fellow, two corporate vice presidents, three distinguished engineers, and 20 partner engineers.”
Although the TCN Technical Leadership Award is a fitting and well-deserved testimonial to Shum’s leadership, he recently earned an even bigger endorsement. At the conclusion of 2013, Steve Ballmer once again tapped Shum for a critical leadership role, this time as executive vice president of Microsoft’s Technology and Research Group. Shum now oversees Microsoft Research, as well as the company Trustworthy Computing Group and Technology Policy Group.
“My new charter is to drive the company’s overall technical direction with SLT, especially for the mid-term and long-term,” says Shum. “We are already building products in a 12- to 18-month time frame. So we want to look beyond 18 months into the future. At MSR we have the charter to look ahead even five or ten years to where all the technology is headed.”
Shum says he is comfortable leaving his position at Bing, knowing his team is positioned to continue its great work. And he leaves with a great deal of satisfaction in what was accomplished. “I am proud of what we achieved as a team,” he concludes. “I am proud that we have built a very high-quality search engine comparable to Google and with differentiating features. We have provided to society, even to humanity, a different voice than Google.”