Candidate Talk: Fault Localization in Large-Scale Computing Systems

February 12, 2008
Naoya Maruyama | Tokyo Institute of Technology, Japan

We describe a new fault localization technique for software bugs in large-scale computing systems. Our technique always collects per-process function call traces of a target system, and derives a concise execution model that reflects its normal function calling behaviors using the traces. To find the cause of a failure, we compare the derived model with the traces collected when the system failed, and compute a suspect score that quantifies how likely a particular part of call traces explains the failure. The execution model consists of a call probability of each function in the system that we estimate using the normal traces. Functions with low probabilities in the model give high anomaly scores when called upon a failure. Frequently-called functions in the model also give high scores when not called. Finally, we report the function call sequences ranked with the suspect scores to aid the human analyst in further localizing the fault to a small part of the overall system. We have applied our proposed method to fault localization of a known non-deterministic bug in a distributed parallel job manager. Experimental results on a three-site, 78-node distributed environment demonstrate that our method quickly locates an anomalous event that is highly correlated with the bug, indicating the effectiveness of our approach.

Speaker Details

Naoya Maruyama is a Ph.D. candidate graduate student at Tokyo Institute of Technology, Japan, expected to graduate in March 2008. He received his Master’s degree at Tokyo Institute of Technology in 2003. His research interests include cluster and Grid computing, statistical techniques for system management, and program analysis. His recent research focuses on fault detection and localization through online system monitoring and modeling.