Testing Tools at a Conference Week in Zurich
The beautiful interior of the Eidgenössische Technische Hochschule’s (ETH, the Swiss Federal Institute of Technology) main building in Zurich is always abuzz with conferences at the end of June, including TOOLS—and this year was no exception. Now in its forty-ninth iteration, TOOLS was a week-long event that brought together four major conferences, eight workshops, and a tutorial on the subject of programming languages, models, components, and proofs.
ETH is a world-famous institute for science and technology; it has produced 21 Nobel Prize winners since its inception in 1855—with seven since 1975—a top score among European universities. Perhaps ETH is more widely known as the place where Albert Einstein began his studies. The chair of Software Engineering is held by Bertrand Meyer, who started the TOOLS Conference Series in 1989 with conferences held in different continents: TOOLS Europe, TOOLS USA, TOOLS Pacific, and TOOLS China. Eight people from Microsoft were right at the center of the event this year.
With program committees spread across the world, holding a program committee meeting to discuss the selection of the papers for the conference is a challenge—but one that TOOLS has always met. This year, I had the honor of serving as program co-chair of one of the conferences, and Ethan Jackson was program co-chair of another. The collected papers from TOOLS are available as a volume entitled Objects, Components, Models, Patterns in the prestigious Lecture Notes in Computer Science series, published by Springer.
The program committee meeting for 2011 was held in Zurich in March with 16 members present and 16 listening in from various regions ranging from India to the United States’ west coast. With very careful planning, papers were scheduled for discussion according to the time zone in their reviewers’ regions. Thus, Aditya Nori from Microsoft Research India was brought in first, and Nikolai Tillmann from Microsoft Research Redmond, Washington, had to rise very early to discuss his papers at the end of the meeting.
Other Microsoft researchers who participated in the conference week included Yuri Gurevich, the conference co-chair of TAP (Tests and Proofs), and program committee members of the various conferences: Nikolaj Bjorner, Clemens Syzperski, Margus Veanes, and Madhu Sudan. (I should note that Nikolaj Bjorner and his team are well known for having popularized the use of testers through RiSE4fun, which enables the tools to be run in browsers.)
Patrice Godefroid presented the keynote address for one of the conferences: TAP 2011. He discussed his work with the SAGE tool for white-box testing technology. Test generation has recently become the largest application of SMT solvers as measured by computational usage. Satisfiability Modulo Theories (SMT) are concerned with checking the satisfiability of logical formulas over one or more theories. At Microsoft, the Z3 SMT solver has solved more than 2 billion constraints in the past two years as a sub-component of SAGE, the first white-box fuzzer. Fuzz testing is an effective technique for finding security vulnerabilities in software. Traditionally, fuzz-testing tools apply random mutations to well-formed inputs of a program and test the resulting values.
Patrice Godefroid – Automated Whitebox Fuzz Testing with SAGE
Since 2009, SAGE has been running non-stop on more than 100 (on average) machines, automatically “fuzzing” hundreds of applications in a dedicated lab that is owned by the Windows security test team. In the process, SAGE found many new expensive security vulnerabilities (which were missed by black box testing and static program analysis).
—Judith Bishop, Director of Computer Science, Microsoft Research Connections