There are many ways to measure quality before and after software is released. But for commercial and internal-use-only products the most important measurement is the user’s perception of product quality. Unfortunately perception is a very difficult thing to measure so companies attempt to quantify it through running customer satisfaction surveys and through the collection of failure/behavioural data from its customer base. This article focuses in on the problems of capturing failure data from customer sites. And while we’ll be using the experienced gained from collecting failure data from Windows XP systems to explore the pertinent issues, the problems one is likely to face when developing internal (non-commercial) software should not be dissimilar.