Preprint
Machine Learning

Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions

Luca Longo(Intel (Ireland)), Mario Brčić(University of Zagreb), Federico Cabitza(Istituto Clinico Sant'Ambrogio), Jaesik Choi(Korea Advanced Institute of Science and Technology), Roberto Confalonieri(University of Padua), Javier Del Ser(University of the Basque Country), Riccardo Guidotti(University of Pisa), Yoichi Hayashi(Meiji University), Francisco Herrera(Universidad de Granada), Andreas Holzinger(BOKU University), Richard Jiang(Lancaster University), Hassan Khosravi(The University of Queensland), Freddy Lécué(Institut national de recherche en sciences et technologies du numérique), Gianclaudio Malgieri(Leiden University), Andrés Páez(Universidad de Los Andes), Wojciech Samek(Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute), Johannes Schneider(University of Liechtenstein), Timo Speith(University of Bayreuth), Simone Stumpf(University of Glasgow)
February 15, 2024Information Fusion551 citations
arXivPDFDOI

551

Citations

29

Influential Citations

Information Fusion

Venue

2024

Year

Abstract

Understanding black box models has become paramount as systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper highlights the advancements in XAI and its application in real-world scenarios and addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. We aim to develop a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 28 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders.

Analysis

Why This Paper Matters

Explainable AI (XAI) has become critical as opaque AI systems proliferate in high-stakes domains like healthcare, finance, and autonomous systems. This paper stands out because it moves beyond individual technical contributions to present a holistic, community-driven agenda. By assembling 20 experts from diverse fields, the authors address the fragmentation that has hindered XAI progress. The manifesto format is particularly valuable: it not only lists problems but also suggests concrete research directions, making it a practical guide for researchers and practitioners.

The paper's timing is significant. With over 550 citations already, it reflects a growing consensus that XAI needs interdisciplinary approaches—combining computer science, cognitive science, law, and ethics. This aligns with Neura Market's focus on actionable AI insights, as the paper directly addresses pain points like user trust, regulatory compliance, and model debugging.

Technical Contributions

The paper's primary technical contribution is the systematic categorization of 28 open problems into nine categories:

  • Foundational issues: Definitions, evaluation metrics, and theoretical underpinnings of XAI.
  • Human-centered XAI: User studies, cognitive biases, and interface design.
  • Model-specific vs. model-agnostic methods: Trade-offs in interpretability and fidelity.
  • Robustness and security: Adversarial attacks on explanations and reliability.
  • Fairness and ethics: Bias detection and mitigation through explanations.
  • Regulatory and legal aspects: Compliance with GDPR and other regulations.
  • Scalability and efficiency: Real-time explanations for large models.
  • Interdisciplinary integration: Combining insights from psychology, sociology, and law.
  • Evaluation and benchmarking: Standardized protocols for comparing XAI methods.

Each problem is accompanied by specific research directions, such as developing causal explanation frameworks or creating user-centric evaluation metrics.

Results

The paper does not present experimental results but rather a structured taxonomy of challenges. The key output is the list of 28 problems, which serves as a benchmark for the field. For example, Problem 1 addresses the lack of a unified definition of explainability, while Problem 15 focuses on ensuring explanations are robust to input perturbations. The authors also highlight the need for interdisciplinary collaboration, citing examples where legal requirements (e.g., right to explanation) intersect with technical feasibility.

Significance

This manifesto has the potential to reshape XAI research by providing a common language and priority list. For practitioners, it offers a checklist of issues to consider when deploying XAI systems. For researchers, it identifies underexplored areas like explanation stability and user-specific customization. The paper's emphasis on interdisciplinary work is particularly timely, as AI regulation (e.g., EU AI Act) demands explainability. By aligning research agendas, this work could accelerate the transition from academic XAI to robust, real-world applications.

explainable-aixaiinterdisciplinaryopen-challengesblack-box-modelsmachine-learningresearch-agenda

Related Papers