Preprint
Machine Learning

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

Julia Amann(ETH Zurich), Alessandro Blasimme(ETH Zurich), Effy Vayena(ETH Zurich), Dietmar Frey(Charité – Universitätsmedizin Berlin), Vince I. Madai(Birmingham City University)
November 30, 2020BMC Medical Informatics and Decision Making2,184 citations

2.2k

Citations

0

Influential Citations

BMC Medical Informatics and Decision Making

Venue

2020

Year

Abstract

BACKGROUND: Explainability is one of the most heavily debated topics when it comes to the application of artificial intelligence (AI) in healthcare. Even though AI-driven systems have been shown to outperform humans in certain analytical tasks, the lack of explainability continues to spark criticism. Yet, explainability is not a purely technological issue, instead it invokes a host of medical, legal, ethical, and societal questions that require thorough exploration. This paper provides a comprehensive assessment of the role of explainability in medical AI and makes an ethical evaluation of what explainability means for the adoption of AI-driven tools into clinical practice. METHODS: Taking AI-based clinical decision support systems as a case in point, we adopted a multidisciplinary approach to analyze the relevance of explainability for medical AI from the technological, legal, medical, and patient perspectives. Drawing on the findings of this conceptual analysis, we then conducted an ethical assessment using the "Principles of Biomedical Ethics" by Beauchamp and Childress (autonomy, beneficence, nonmaleficence, and justice) as an analytical framework to determine the need for explainability in medical AI. RESULTS: Each of the domains highlights a different set of core considerations and values that are relevant for understanding the role of explainability in clinical practice. From the technological point of view, explainability has to be considered both in terms how it can be achieved and what is beneficial from a development perspective. When looking at the legal perspective we identified informed consent, certification and approval as medical devices, and liability as core touchpoints for explainability. Both the medical and patient perspectives emphasize the importance of considering the interplay between human actors and medical AI. We conclude that omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individual and public health. CONCLUSIONS: To ensure that medical AI lives up to its promises, there is a need to sensitize developers, healthcare professionals, and legislators to the challenges and limitations of opaque algorithms in medical AI and to foster multidisciplinary collaboration moving forward.

Analysis

Why This Paper Matters

Explainability in artificial intelligence is often framed as a purely technical challenge, but this paper argues convincingly that in healthcare it is fundamentally a multidisciplinary concern. As AI-driven clinical decision support systems become more capable—sometimes surpassing human performance in analytical tasks—the lack of transparency sparks criticism from clinicians, patients, regulators, and ethicists alike. This paper matters because it moves the conversation beyond algorithmic techniques and situates explainability within the broader context of medical practice, law, and ethics.

By adopting a multidisciplinary approach, the authors show that explainability is not just about opening the black box; it is about ensuring that AI tools can be responsibly integrated into clinical workflows. The paper's ethical assessment, grounded in the four principles of biomedical ethics—autonomy, beneficence, nonmaleficence, and justice—provides a normative framework for why explainability is not optional but essential. This is particularly relevant as healthcare systems increasingly experiment with AI, often without clear guidelines for transparency and accountability.

The paper's impact is amplified by its timing and venue. Published in BMC Medical Informatics and Decision Making, it bridges the gap between technical AI research and clinical practice, reaching an audience of healthcare professionals, policymakers, and developers. With over 2,000 citations, it has become a foundational reference for discussions on trustworthy medical AI.

Technical Contributions

  • Multidisciplinary framework: The paper systematically analyzes explainability from technological, legal, medical, and patient perspectives, identifying core considerations and values in each domain.
  • Ethical assessment: It applies the Principles of Biomedical Ethics to evaluate the need for explainability, showing how opacity threatens autonomy (informed consent), beneficence (trust and adoption), nonmaleficence (safety), and justice (equitable access).
  • Legal touchpoints: It pinpoints informed consent, medical device certification and approval, and liability as key areas where explainability is legally relevant.
  • Human-AI interplay: It emphasizes that both medical and patient perspectives require understanding the interaction between human actors and AI systems, rather than treating AI in isolation.
  • Call for collaboration: It advocates for sensitizing stakeholders and fostering multidisciplinary collaboration to address the challenges of opaque algorithms.

Results

As a conceptual analysis, the paper does not report quantitative metrics or experimental comparisons. Instead, its "results" are the synthesized insights from each domain. Technologically, explainability must be considered both in terms of how it can be achieved and what benefits it brings to development. Legally, the paper identifies informed consent, certification, and liability as core touchpoints. Medically and from the patient perspective, the interplay between human actors and AI is crucial. The ethical assessment concludes that omitting explainability poses a threat to core ethical values and may have detrimental consequences for individual and public health. These findings are qualitative but carry significant normative weight.

Significance

The paper's significance lies in its framing of explainability as an ethical imperative rather than a technical luxury. It provides a common language for developers, clinicians, lawyers, and ethicists to discuss the challenges of opaque AI in healthcare. By linking explainability to established biomedical principles, it offers a robust justification for regulatory and design requirements. Moreover, it highlights the need for multidisciplinary collaboration, which has become a cornerstone of responsible AI development. The paper's high citation count attests to its influence in shaping the discourse on trustworthy medical AI and its role in guiding future research and policy.