Probabilistic Approach For Assessing Quality of Label Generation Techniques In Machine Learning

Microsoft Defender for Office protects enterprise users from cyber-attacks via email. The intelligence powering the protection comes from ML models, third-party intelligence sources as well as human graders who play an important role in generating ground truth data (labels) for emails that are submitted by end-users. These emails are generally categorized into Good, Phish, Spam, Bulk or else unknown in cases when it is difficult to label it to the known four classes. Historical data indicates that human graders’ labels are not perfect, and segmentations suffer from inter-grader and intra-grader variability. To quantify the underlying variability, the Simultaneous Truth And Performance Level Estimation (STAPLE) algorithm is used, which mathematically computes a square matrix of probabilities of estimated true segment corresponding to a graded segment. With hundreds of graders present in the system, grading thousands of emails each day across multiple segments, the STAPLE algorithm results are non-actionable by business stakeholders. In addition, STAPLE only provides performance at grader level but not at item (emails in this case) level.

In this paper, we make the following three contributions. Firstly, we incorporated a post-processing step that provides interpretability to STAPLE’s results as applied to the generation of reliable ground truth for emails. Secondly, we propose an algorithm that consumes STAPLE results and estimates the true segmentation of an item. Thirdly, we propose a computation methodology of the conflict rate that is easily applicable to the proposed formulation when multiple graders are grading the same item and we will provide a use case of the approach on a real example. These contributions will help identify graders areas of weakness to facilitate personalized training for an overall improvement of effectiveness throughout our systems and to contribute to the ultimate protection to the customer from email based phishing attacks.