![\"Example](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2022\/11\/UHBFig1.jpg\")
<\/figure>\n\n\n\nCOVID-19 X-rays have been a recommended procedure for patient triaging and resource management in intensive care units (ICUs) throughout the COVID-19 pandemic. The machine learning efforts to augment this workflow have, however, been long challenged due to deficiencies in reporting, model evaluation, and failure mode analysis. A recent study (opens in new tab)<\/span><\/a> showed that 415 COVID-19 medical imaging projects had deficiencies that limited their use outside of the research lab. To address some of those shortcomings, we worked closely with our clinical partners to model radiological features with a human-interpretable class hierarchy that aligns with the radiological decision process. A DenseNet-121 backbone was first pre-trained with BYOL self-supervision on the public NIH-CXR dataset (opens in new tab)<\/span><\/a> \u2013 technical details and how to do this using InnerEye-DeepLearning are here (opens in new tab)<\/span><\/a>. This was then fine-tuned with cross-validation using a private COVID-19 training dataset from four hospitals of the UHB group collected during the first COVID-19 wave (March to June 2020). The UHB team used the Azure Machine Learning DICOM image labelling tool to significantly speed up a labelling study involving several clinical annotators, who were asked to classify 400 chest X-rays. The collected labels were then used to benchmark the model’s performance.<\/p>\n\n\n\n The developed model outperforms the clinicians across all defined sub-tasks with respect to the reference labels. To better understand the model\u2019s failure patterns, we employed an error analysis tool in Azure Machine Learning (opens in new tab)<\/span><\/a>. This tool trains a decision tree to identify partitions of data on which the model underperforms, according to attributes that are most predictive of mistakes. It is based on work by our MSR colleagues Besmira Nushi, Ece Kamar, and Eric Horvitz: Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure – Microsoft Research<\/a>.<\/p>\n\n\n\n This kind of error analysis is not often found in healthcare-related ML studies, we believe it is crucial for providing transparency and actionable insights about a model\u2019s behavior. The analysis may also be useful after deployment if presented as reliability information alongside the model\u2019s predictions.<\/p>\n\n\n\n You can read the full paper – \u201cHierarchical Analysis of Visual COVID-19 Features from Chest Radiographs<\/em><\/a>\u201d that was presented as part of ICML\u2019s 1st<\/sup> workshop on \u201cInterpretable Machine Learning in Healthcare (opens in new tab)<\/span><\/a>\u201d.<\/p>\n","protected":false},"excerpt":{"rendered":" Overview Microsoft Research\u2019s Project InnerEye team in Cambridge (UK) worked with University Hospitals Birmingham NHS Foundation Trust to develop deep learning models to analyze anonymized chest X-Rays and chest computed tomography (CT) scans to assist clinicians in determining disease severity, aid decision making, and improve our understanding of the disease. This collaborative project in Microsoft\u2019s Studies In […]<\/p>\n","protected":false},"author":32522,"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-content-parent":740356,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[],"msr-locale":[268875],"msr-post-option":[],"class_list":["post-897876","msr-blog-post","type-msr-blog-post","status-publish","hentry","msr-locale-en_us"],"msr_assoc_parent":{"id":740356,"type":"project"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/897876","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-blog-post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/32522"}],"version-history":[{"count":8,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/897876\/revisions"}],"predecessor-version":[{"id":899244,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/897876\/revisions\/899244"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=897876"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=897876"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=897876"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=897876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"Error](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2022\/11\/UHBFig4-1024x624.png\")
<\/figure>\n\n\n\n