Journal Article
Machine Learning

Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?

Olivier Bernard(Lyon 1 Université), Alain Lalande(Centre National de la Recherche Scientifique), Clément Zotti(Université de Sherbrooke), Frédéric Cervenansky(Lyon 1 Université), Xin Yang(Chinese University of Hong Kong), Pheng‐Ann Heng(Chinese University of Hong Kong), Irem Cetin(Universitat Pompeu Fabra), Karim Lekadir(Universitat Pompeu Fabra), Óscar Cámara(Universitat Pompeu Fabra), Miguel Á. González Ballester(Institució Catalana de Recerca i Estudis Avançats), Gerard Sanromà(Universitat Pompeu Fabra), Sandy A. Napel(Stanford Medicine), Steffen Erhard Petersen(Queen Mary University of London), Georgios Tziritas(University of Crete), Elias Grinias(University of Crete), Mahendra Khened(Indian Institute of Technology Madras), Varghese Alex Kollerathu(Indian Institute of Technology Madras), Ganapathy Krishnamurthi(Indian Institute of Technology Madras), Marc-Michel Rohé(Institut national de recherche en sciences et technologies du numérique), Xavier Pennec(Institut national de recherche en sciences et technologies du numérique), Maxime Sermesant B(Institut national de recherche en sciences et technologies du numérique), Fabian Isensee(German Cancer Research Center), Paul F. Jäger(German Cancer Research Center), Klaus Hermann Maier-Hein(German Cancer Research Center), Peter M. Full(Heidelberg University), Ivo Wolf A(Technische Hochschule Mannheim), Sandy Engelhardt(Technische Hochschule Mannheim), Christian Frederik Baumgartner(ETH Zurich), Lisa Margret Koch(ETH Zurich), Jelmer M. Wolterink(Utrecht University), Ivana Išgum(Utrecht University), Yeonggul Jang(Yonsei University), Yoonmi Hong(Yonsei University), Jay Patravali(Qure.ai Technologies Private Limited (India)), Shubham Jain(Qure.ai Technologies Private Limited (India)), Olivier Humbert(PHEN X - Physiopathologie, Environnement et Exposome), Pierre‐Marc Jodoin(Université de Sherbrooke)
May 17, 2018IEEE Transactions on Medical Imaging2,416 citations
PDFDOI

2.4k

Citations

0

Influential Citations

IEEE Transactions on Medical Imaging

Venue

2018

Year

Abstract

Delineation of the left ventricular cavity, myocardium, and right ventricle from cardiac magnetic resonance images (multi-slice 2-D cine MRI) is a common clinical task to establish diagnosis. The automation of the corresponding tasks has thus been the subject of intense research over the past decades. In this paper, we introduce the "Automatic Cardiac Diagnosis Challenge" dataset (ACDC), the largest publicly available and fully annotated dataset for the purpose of cardiac MRI (CMR) assessment. The dataset contains data from 150 multi-equipments CMRI recordings with reference measurements and classification from two medical experts. The overarching objective of this paper is to measure how far state-of-the-art deep learning methods can go at assessing CMRI, i.e., segmenting the myocardium and the two ventricles as well as classifying pathologies. In the wake of the 2017 MICCAI-ACDC challenge, we report results from deep learning methods provided by nine research groups for the segmentation task and four groups for the classification task. Results show that the best methods faithfully reproduce the expert analysis, leading to a mean value of 0.97 correlation score for the automatic extraction of clinical indices and an accuracy of 0.96 for automatic diagnosis. These results clearly open the door to highly accurate and fully automatic analysis of cardiac CMRI. We also identify scenarios for which deep learning methods are still failing. Both the dataset and detailed results are publicly available online, while the platform will remain open for new submissions.

Analysis

Why This Paper Matters

This paper is pivotal because it introduces the ACDC dataset, which has become a cornerstone for cardiac MRI analysis. By providing a large, fully annotated, multi-center dataset, it enables reproducible research and fair comparison of deep learning methods. The challenge format fosters collaboration and accelerates progress in the field.

The paper's significance is further amplified by its comprehensive benchmarking of deep learning techniques for both segmentation and diagnosis. It demonstrates that deep learning can achieve near-expert performance, which is a critical step toward clinical adoption. The high citation count (over 2400) reflects its impact on the medical imaging community.

Moreover, the paper addresses the question "Is the problem solved?" by showing that while performance is high, there are still failure cases. This nuanced view prevents overconfidence and guides future research toward robust solutions.

Technical Contributions

  • ACDC Dataset: The largest publicly available fully annotated cardiac MRI dataset with 150 patients, including reference measurements and classifications from two experts.
  • Benchmarking Framework: A standardized evaluation protocol for segmentation and classification tasks, with results from multiple research groups.
  • State-of-the-Art Performance: Demonstrates that deep learning methods can achieve high correlation (0.97) for clinical indices and high accuracy (0.96) for diagnosis.
  • Identification of Failure Modes: Highlights scenarios where deep learning methods still struggle, providing direction for future work.

Results

The best segmentation methods achieved a mean correlation score of 0.97 for automatic extraction of clinical indices such as ejection fraction and ventricular volumes. For diagnosis, the best method achieved an accuracy of 0.96, closely matching expert consensus. These results were obtained from nine groups for segmentation and four for classification, showing the robustness of deep learning across different implementations. The paper also reports that some methods failed in specific cases, such as patients with abnormal anatomy or poor image quality.

Significance

This work has significantly advanced the field of medical image analysis by providing a benchmark dataset and demonstrating the feasibility of fully automatic cardiac MRI assessment. It has spurred further research in deep learning for cardiac imaging and has been widely cited, influencing both academic and clinical communities. The dataset and platform remain open for new submissions, continuing to drive innovation. The paper's findings suggest that deep learning can be a reliable tool for clinical diagnosis, potentially reducing workload and improving patient care.

deep-learningmedical-imagingcardiac-mrisegmentationdiagnosisbenchmarkdataset

Related Papers