{"id":1142540,"date":"2025-06-26T09:08:25","date_gmt":"2025-06-26T16:08:25","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=1142540"},"modified":"2025-07-18T13:18:15","modified_gmt":"2025-07-18T20:18:15","slug":"padchest-gr-a-bilingual-grounded-radiology-reporting-benchmark-for-chest-x-rays","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/padchest-gr-a-bilingual-grounded-radiology-reporting-benchmark-for-chest-x-rays\/","title":{"rendered":"PadChest-GR: A bilingual grounded radiology reporting benchmark for chest X-rays"},"content":{"rendered":"\n
\"Alt<\/figure>\n\n\n\n

In our ever-evolving journey to enhance healthcare through technology, we\u2019re announcing a unique new benchmark for grounded radiology report generation\u2014PadChest-GR (opens in new tab)<\/span><\/a><\/strong>. The world\u2019s first multimodal, bilingual sentence-level radiology report dataset, developed by the University of Alicante with Microsoft Research, University Hospital Sant Joan d\u2019Alacant and MedBravo, is set to redefine how AI and radiologists interpret radiological images. Our work demonstrates how collaboration between humans and AI can create powerful feedback loops\u2014where new datasets drive better AI models, and those models, in turn, inspire richer datasets. We’re excited to share this progress in\u202fNEJM AI, highlighting both the clinical relevance and research excellence of this initiative. <\/p>\n\n\n\n

A new frontier in radiology report generation <\/h2>\n\n\n\n

It is estimated that over half of people visiting hospitals have radiology scans that must be interpreted by a clinical professional. Traditional radiology reports often condense multiple findings into unstructured narratives. In contrast, grounded radiology reporting demands that each finding be described and localized individually.<\/p>\n\n\n\n

This can mitigate the risk of AI fabrications and enable new interactive capabilities that enhance clinical and patient interpretability. PadChest-GR is the first bilingual dataset to address this need with 4,555 chest X-ray studies complete with Spanish and English sentence-level descriptions and precise spatial (bounding box) annotations for both positive and negative findings. It is the first public benchmark that enables us to evaluate generation of fully grounded radiology reports in chest X-rays. <\/p>\n\n\n\n

\"Figure
Figure 1. Example of a grounded report from PadChest-GR. The original free-text report in Spanish was \u201dMotivo de consulta: Preoperatorio. Rx PA t\u00f3rax: Impresi\u00f3n diagn\u00f3stica: Ateromatosis a\u00f3rtica calcificada. Engrosamiento pleural biapical. Atelectasia laminar basal izquierda. Elongaci\u00f3n a\u00f3rtica. Sin otros hallazgos radiol\u00f3gicos significativos.\u201d<\/em><\/figcaption><\/figure>\n\n\n\n

This benchmark isn\u2019t standing alone\u2014it plays a critical role in powering our state-of-the-art multimodal report generation model, MAIRA-2<\/strong>. Leveraging the detailed annotations of PadChest-GR, MAIRA-2 represents our commitment to building more interpretable and clinically useful AI systems. You can explore our work on MAIRA-2 on our project web page<\/a>, including recent user research conducted with clinicians in healthcare settings<\/a>.<\/p>\n\n\n\n

PadChest-GR is a testament to the power of collaboration. Aurelia Bustos at MedBravo and Antonio Pertusa at the University of Alicante published the original PadChest dataset (opens in new tab)<\/span><\/a> in 2020, with the help of Jose Mar\u00eda Salinas from Hospital San Juan de Alicante and Mar\u00eda de la Iglesia Vay\u00e1 from the Center of Excellence in Biomedical Imaging at the Ministry of Health in Valencia, Spain. We started to look at PadChest and were deeply impressed by the scale, depth, and diversity of the data.<\/p>\n\n\n\n

As we worked more closely with the dataset, we realized the opportunity to develop this for grounded radiology reporting research and worked with the team at the University of Alicante to determine how to approach this together. Our complementary expertise was a nice fit. At Microsoft Research, our mission is to push the boundaries of medical AI through innovative, data-driven solutions. The University of Alicante, with its deep clinical expertise, provided critical insights that greatly enriched the dataset\u2019s relevance and utility. The result of this collaboration is the PadChest-GR dataset.<\/p>\n\n\n\n

A significant enabler of our annotation process was Centaur Labs<\/strong>. The team of senior and junior radiologists from the University Hospital Sant Joan d\u2019Alacant, coordinated by Joaquin Galant, used this HIPAA-compliant labeling platform to perform rigorous study-level quality control and bounding box annotations. The annotation protocol implemented ensured that each annotation was accurate and consistent, forming the backbone of a dataset designed for the next generation of grounded radiology report generation models. <\/p>\n\n\n\n

Accelerating PadChest-GR dataset annotation with AI <\/h2>\n\n\n\n

Our approach integrates advanced large language models with comprehensive manual annotation: <\/p>\n\n\n\n

Data Selection & Processing:<\/strong> Leveraging Microsoft Azure OpenAI Service (opens in new tab)<\/span><\/a> with GPT-4, we extracted sentences describing individual positive and negative findings from raw radiology reports, translated them from Spanish to English, and linked each sentence to the existing expert labels from PadChest. This was done for a selected subset of the full PadChest dataset, carefully curated to reflect a realistic distribution of clinically relevant findings. <\/p>\n\n\n\n

Manual Quality Control & Annotation:<\/strong> The processed studies underwent meticulous quality checks on the Centaur Labs platform by radiologist from Hospital San Juan de Alicante. Each positive finding was then annotated with bounding boxes to capture critical spatial information. <\/p>\n\n\n\n

Standardization & Integration:<\/strong> All annotations were harmonized into coherent grounded reports, preserving the structure and context of the original findings while enhancing interpretability. <\/p>\n\n\n\n

\"Figure
Figure 2. Overview of the data curation pipeline.<\/figcaption><\/figure>\n\n\n\n

Impact and future directions <\/h2>\n\n\n\n

PadChest-GR not only sets a new benchmark for grounded radiology reporting, but also serves as the foundation for our MAIRA-2 model, which already showcases the potential of highly interpretable AI in clinical settings<\/a>. While we developed PadChest-GR to help train and validate our own models, we believe the research community will greatly benefit from this dataset for many years to come. We look forward to seeing the broader research community build on this\u2014improving grounded reporting AI models and using PadChest-GR as a standard for evaluation. We believe that by fostering open collaboration and sharing our resources, we can accelerate progress in medical imaging AI and ultimately improve patient care together with the community.<\/p>\n\n\n\n

The collaboration between Microsoft Research and the University of Alicante highlights the transformative power of working together across disciplines. With our publication in NEJM-AI and the integral role of PadChest-GR in the development of MAIRA-2<\/a> and RadFact (opens in new tab)<\/span><\/a>, we are excited about the future of AI-empowered radiology. We invite researchers and industry experts to explore PadChest-GR and MAIRA-2, contribute innovative ideas, and join us in advancing the field of grounded radiology reporting. <\/p>\n\n\n\n

Papers already using PadChest-GR:<\/p>\n\n\n\n