Posted by
PODCAST SERIES
AI Testing and Evaluation: Learnings from Science and Industry
Discover how Microsoft is learning from other domains to advance evaluation and testing as a pillar of AI governance.
Some people, it seems, are simply born to be computer-science researchers.
Take Youssef Hamadi (opens in new tab), for example. Earlier this week, he was asked a question about challenges he had encountered during a research project. His response began by sharing this revealing nugget:
“As a kid, I remember reading that 70 percent of the code written every day is not new,” he recalled. “This is one of the reasons why I decided to become a researcher: to write innovative code every day.”
Youssef Hamadi was born for the Bing Code Search (opens in new tab) project.
Bing Code Search for Visual Studio 2013 (opens in new tab) is a unique partnership between groups within and outside Microsoft, including such websites as StackOverflow (opens in new tab), to deliver new tools to developers that will save them time and make software development easier.
Hamadi and Yi Wei (opens in new tab) of Microsoft Research Cambridge (opens in new tab)’s Constraint Reasoning Group (opens in new tab) work with colleagues such as Nirupama Chandrasekaran and Sumit Gulwani (opens in new tab) of Microsoft Research Redmond (opens in new tab), the Visual Studio platform team, represented by Ala Shiban (opens in new tab), and the Bing (opens in new tab) tech segment, including Shabbar Husain.
The result of all this collaboration is a free add-in, which became available for download (opens in new tab) on Feb. 17, that makes it easier for .NET developers to search for and reuse code samples from across the coding community. The news about Bing Code Search also appears on the Bing (opens in new tab) and Visual Studio (opens in new tab) blogs.
Software developers routinely rely on the Internet to find and reuse code samples that pertain to their current projects. Sites such as the Microsoft Developer Network (opens in new tab) (MSDN) and StackOverflow provide a rich collection of code samples to address many of the needs programmers face.
The process for doing so, though, is not particularly streamlined. The developer has to exit the programming environment, switch to a browser, enter a search query, sift through the search results for useful code snippets, copy and paste a promising snippet back into the programming environment, and adapt the pasted snippet to the programming context at hand.
It works, but it’s not optimal.
Bing Code Search has been designed to help. Developers can perform a code search directly from within Visual Studio, type queries in plain English, and capture useful code samples within Visual Studio’s integrated development environment. The add-in automatically adapts the snippets to match the developer’s context, making the suggested snippets easier to understand and better prepared for reuse.
The add-in got an early tryout with a low-key debut on Feb. 10. The sneak preview enabled the engineers involved to collect telemetry data and fine-tune the tool—and, along the way, obtain some gratifying feedback.
A few days after the initial release, the collaborating teams were getting 40,000 requests per day from developers worldwide. And the comments engendered by this trial were quite encouraging:
- “This is very interesting and looks useful … Availability?”
- “You guys are amazing! This thing is just wonderful!”
- “Can’t wait to see this in Visual Studio. Microsoft rocks.”
- “One of the best ideas I have seen so far …”
Such plaudits should come as little surprise, though, because the project was devised by Wei—then an intern, now a postdoc at the Cambridge lab, who was looking for ways to help his wife, who happens to be a software developer.
Hamadi recently took a few moments to discuss the origins of the Bing Code Search project.
“As a group leader in Constraint Reasoning, I am always interested by new application domains, for two reasons,” he says. “First, I believe that those domains are important to push the boundaries of constraint solvers. Second, I like to escape the aridity of automatic theorem proving and have some fun!
“Two years ago, after working on a project to help architects synthesize efficient buildings (opens in new tab), I decided to consider program synthesis, an old and important problem. We picked up Yi Wei for an internship on this subject, and two years later, the result is Bing Code Search.”
One interesting aspect of the Bing Code Search project is its integration with Bing and its partnership with popular providers to the coding community, both within Microsoft—MSDN—and sites not affiliated with Microsoft, such as dotnetperls (opens in new tab), C# 411 (opens in new tab), and StackOverflow.
“This project involves Microsoft Research’s Cambridge and Redmond labs, Visual Studio, Bing, and Azure,” Hamadi says. “Having a research team on such a problem is the only way to make it happen, because we are prepared to take such risks. Having a research team was also important to create a bridge between Visual Studio and Bing, and as a result, they are now exploring their own opportunities.”
Bing Code Search also should provide a boon to apprentice programmers coming to C# from a different language. The new download will help them learn about the APIs at their disposal.
A recent study indicated that Bing Code Search provides to programmers a time improvement of more than 60 percent, compared with the browser-search-copy-and-paste scenario.
“The research challenges looked feasible, but difficult,” Hamadi says. “The main problem was our capacity to correctly identify relevant snippets and to judge them with regard to the programming context and the user query. We had to invent new features—and ranking techniques largely inspired by static analysis.
“One major challenge was the automated adaptation of retrieved snippets. For that, we had to invent mapping techniques able to match programmers’ variables and retrieved ones.”
Having achieved the feat of providing the development community with the luxury of increased productivity, what’s next for Hamadi in this domain?
“We are working on very surprising things in this area,” he smiles slyly. “I cannot comment about them.”