Measuring Reliability of Software Products

Current methods to measure the reliability of software are usually focused on large server based products. In these approaches, the product reliability is traditionally measured in terms of catastrophic failures, as the failure data is generally collected manually through service organizations which filter out data on many types of operational failures. These method and metrics are not applicable for mass market products that run in multiple operational profiles, where other types of failures might be equally important, and where manual data collection is inadequate. For such products, unique issues arise in obtaining the failure and population data, and in analyzing this data to determine reliability. In this paper we first discuss some of the key issues in determining reliability of such software products, and then discuss two systems being used for measuring reliability of commercial software products.