Open Access
ENHANCING TRUST AND CLINICAL ADOPTION: A SYSTEMATIC LITERATURE REVIEW OF EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI) APPLICATIONS IN HEALTHCARE
4
Department of Health Informatics, Biomedical Technology Research Center, London, United Kingdom
4
Department of Health Informatics, Biomedical Technology Research Center, London, United Kingdom
Abstract
Background: The transformative potential of Artificial Intelligence (AI) in healthcare is hampered by the "black box" problem, where a lack of transparency in decision-making fundamentally undermines clinician trust and creates barriers to clinical adoption. Explainable Artificial Intelligence (XAI) is proposed as a necessary solution to bridge the gap between high-performance AI models and the critical need for justification and accountability in patient care.
Methods: This systematic literature review was conducted in adherence to PRISMA guidelines, analyzing literature published between January 2020 and early 2024. A rigorous search across major databases identified 50 relevant primary studies on XAI applications in clinical and biomedical contexts. Data extracted included the medical domain, AI model, XAI technique, and reported impact on trust and accuracy.
Results: Analysis of the 50 studies demonstrated a wide application of XAI across diverse medical fields, including diagnostics, medical imaging, and disease prediction. XAI—especially methods like SHAP, LIME, and GRAD-CAM—was found to significantly enhance interpretability, transparency, and diagnostic accuracy in these applications, successfully building clinician confidence in AI systems. The primary applications were observed in areas like chronic wound classification, cancer diagnosis, and cardiovascular risk prediction.
Conclusion: XAI is paramount for the safe and effective integration of AI into clinical practice. However, real-world integration is associated with persistent technical and data-quality challenges, including inconsistent validation and biased datasets. Future efforts must prioritize the development of standardized frameworks and regulatory compliance to ensure safe, ethical, and fully explainable AI use in healthcare.
Keywords
Explainable Artificial Intelligence (XAI),
Healthcare AI,
Systematic Review,
Clinical Adoption
References
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler, A Highly Transparent and Explainable Artificial Intelligence Tool for Chronic Wound Classification: XAI-CWC, Jan. 2021.
M. Merry, P. Riddle, and J. Warren, “A mental models approach for defining explainable artificial intelligence,” BMC Medical Informatics and Decision Making, vol. 21, no. 1, Dec. 2021.
J. A. Yeung, Y. Y. Wang, Z. Kraljevic, and J. T. H. Teo, Artificial intelligence (AI) for neurologists: do digital neurones dream of electric sheep?, Practical Neurology, vol. 23, no. 6, pp. 476–488, Dec. 2023.
Srilatha, S. (2025). Integrating AI into enterprise content management systems: A roadmap for intelligent automation. Journal of Information Systems Engineering and Management, 10(45s), 672–688. https://doi.org/10.52783/jisem.v10i45s.8904
Bohr and K. Memarzadeh, The rise of artificial intelligence in healthcare applications, Artificial Intelligence in Healthcare, vol. 1, no. 1, pp. 25–60, Jun. 2020.
T. Davenport and R. Kalakota, The Potential for Artificial Intelligence in Healthcare, Future Healthcare Journal, vol. 6, no. 2, pp. 94–98, Jun. 2019.
K. B. Johnson et al., Precision Medicine, AI, and the Future of Personalized Health Care, Clinical and Translational Science, vol. 14, no. 1, Oct. 2020, Available: https/www.ncbi.nlm.nih.gov/pmc/articles/PMC7877825/.
C. J. Kelly, A. Karthikesalingam, M. Suleyman, G. Corrado, and D. King, Key challenges for delivering clinical impact with artificial intelligence, BMC Medicine, vol. 17, no. 1, Oct. 2019.
J. Bajwa, U. Munir, A. Nori, and B. Williams, Artificial intelligence in healthcare: transforming the practice of medicine, Future Healthcare Journal, vol. 8, no. 2, pp. e188–e194, 2021.
S. A. Alowais et al., Revolutionizing healthcare: the role of artificial intelligence in clinical practice, BMC Medical Education, vol. 23, no. 1, Sep. 2023.
H. Alami et al., Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity, Journal of Medical Internet Research, vol. 22, no. 7, p. e17707, Jul. 2020.
T. Raclin et al., Combining Machine Learning, Patient-Reported Outcomes, and Value-Based Health Care: Protocol for Scoping Reviews,” JMIR Research Protocols, vol. 11, no. 7, p. e36395, Jul. 2022.
Rangu, S. (2025). Analyzing the impact of AI-powered call center automation on operational efficiency in healthcare. Journal of Information Systems Engineering and Management, 10(45s), 666–689. https://doi.org/10.55278/jisem.2025.10.45s.666
B. Vasey et al., DECIDE-AI: new reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence, Nature Medicine, vol. 27, no. 2, pp. 186–187, Feb. 2021.
Moher, D., Liberati, A., Tetzlaff, J., and Altman, D. G. (2009). Preferred reporting items for systematic reviews and metaanalyses: the PRISMA statement. Ann. Inter. Med. 151, 264–269.
Kwang Sig Lee and Eun Sun Kim, Explainable artificial intelligence in the early diagnosis of gastrointestinal disease, Diagnostics, vol. 12, no. 11, Nov. 2022.
A.-D. Samaras et al., Explainable classification of patients with primary hyperparathyroidism using highly imbalanced clinical data derived from imaging and biochemical procedures, Applied Sciences, vol. 14, no. 5, p. 2171, Mar. 2024.
U. Pawar, S. Rea, Ruairi O'reilly, and D. O’shea, Incorporating explainable artificial intelligence (XAI) to aid the understanding of machine learning in the healthcare domain, 2020. Available: .
Sergiusz Wesołowski et al., An explainable artificial intelligence approach for predicting cardiovascular outcomes using electronic health records, PLOS Digital Health, vol. 1, no. 1, p. e0000004, Jan. 2022.
Chadha, K. S. (2025). Zero-Trust Data Architecture for Multi-Hospital Research: HIPAA-Compliant Unification of EHRs, Wearable Streams, and Clinical Trial Analytics. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3477
Das and P. Rad, Opportunities and challenges in explainable artificial intelligence (XAI): A survey, Jun. 2020. Available: .
V. Sharma, Samarth Chhatwal, and B. Singh, An explainable artificial intelligence based prospective framework for COVID-19 risk prediction.
José Jiménez-Luna, F. Grisoni, and G. Schneider, “Drug discovery with explainable artificial intelligence,” Nature Machine Intelligence, vol. 2, no. 10, pp. 573–584, Oct. 2020.
D. Dave, H. Naik, S. Singhal, and P. Patel, Explainable AI meets healthcare: A study on heart disease dataset, Nov. 2020. Available: .
J. Hoffmann et al., Prediction of clinical outcomes with explainable artificial intelligence in patients with chronic lymphocytic leukemia, Current Oncology, vol. 30, no. 2, pp. 1903–1915, Feb. 2023.
Khishigsuren Davagdorj, Jang Whan Bae, Van Huy Pham, Nipon Theera-Umpon, and Keun Ho Ryu, Explainable artificial intelligence based framework for non-communicable diseases prediction, IEEE Access, vol. 9, pp. 123672–123688, 2021.
Augusto Anguita-Ruiz, A. Segura-Delgado, R. Alcalá, C. M.1 Aguilera, and Jesús Alcalá-Fdez, EXplainable Artificial2 Intelligence (XAI) for the identification of biologically relevant3 gene expression patterns in longitudinal human studies, insights4 from obesity res5earch, PLoS Computational Biology, vol. 16, no. 4, Apr. 2020.
V. Roessner, J. Rothe, G. Kohls, Georg Schomerus, S. Ehrlich, andC. Beste, Taming the chaos?! Using eXplainable Artificial Intelligence (XAI) to tackle the complexity in mental health research, European Child and Adolescent Psychiatry, vol. 30, no. 8, pp. 1143–1146, Aug. 2021.
Jiten Sardana. (2025). Secure Messaging Protocols for Transactional Health Notifications. Utilitas Mathematica, 122(2), 267–290. Retrieved from https://utilitasmathematica.com/index.php/Index/article/view/2707
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler,A highly transparent and explainable artificial intelligencetool for 6 Journal of Computer Sciences and Applications chronic wound classification: XAI-CWC, 2021.
Salman Muneer et al., An IoMT enabled smart healthcare model to monitor elderly people using Explainable Artificial Intelligence (EAI)., Journal of NCBAE, Vol 1.
Shaker El-Sappagh, J. M. Alonso, S. M.Riazul Islam, A.M. Sultan, and Kyung Sup Kwak, A multilayermultimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Raza, Kim Phuc Tran, L. Koehl, and S. Li, “Designing ECG monitoring healthcare system with federated transfer learning and explainable AI,” Knowl. Based Syst., vol. 236, p. 107763, 2021, Available: .
S. El-Sappagh, J. M. Alonso, S. M. R. Islam, A. M. Sultan, and K. S. Kwak, A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease, Scientific Reports, vol. 11, no. 1, p. 2660, Jan. 2021.
J. Peng et al., An explainable artificial intelligence framework for the deterioration risk prediction of hepatitis patients, Journal of Medical Systems, vol. 45, no. 5, May 2021.
L. Lindsay, S. Coleman, D. Kerr, B. Taylor, and A. Moorhead, Explainable artificial intelligence for falls prediction, in Communications in Computer and Information Science, Springer, 2020, pp. 76–84.
Y. Jia, J. McDermid, T. Lawton, and Ibrahim Habli,“The role of explainability in assuring safety of machine learning in healthcare, IEEE Transactions on Emerging Topics in Computing, vol. 10, no. 4, pp. 1746–1760, Oct. 2022.
F. Vaquerizo-Villar et al., An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea, Computers in Biology and Medicine, vol. 165, Oct. 2023.
F. Xu et al., The clinical value of explainable deep learning for diagnosing fungal keratitis using in vivo confocal microscopy images, Frontiers in Medicine, vol. 8, Dec. 2021.
Z. Naz, Muhammad, T. Saba, A. Rehman, Haitham Nobanee, and Saeed Ali Bahaj, An explainable AI-Enabled framework for interpreting pulmonary diseases from chest radiographs, Cancers, vol. 15, no. 1, Jan. 2023.
Belal Alsinglawi et al., An explainable machine learningframework for lung cancer hospital length of stay prediction, Scientific Reports, vol. 12, no. 1, Dec. 2022.
Esma Cerekci et al., Quantitative evaluation of saliency-based explainable artificial intelligence (XAI) methods in deep learning- based mammogram analysis, European Journal of Radiology, vol. 173, Apr. 2024.
Mohammed Saidul Islam, I. Hussain, Md Mezbaur Rahman, Se Jin Park, and Md Azam Hossain, Explainable artificial intelligence model for stroke prediction using EEG signal, Sensors, vol. 22, no. 24, Dec. 2022.
Z. U. Ahmed, K. Sun, M. Shelly, and L. Mu, Explainable artificial intelligence (XAI) for exploring spatial variability of lung and bronchus cancer (LBC) mortality rates in the contiguous USA, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Samantapudi, R. K. R. (2025). Enhancing search and recommendation personalization through user modeling and representation. International Journal of Computational and Experimental Science and Engineering, 11(3), 6246–6265. https://doi.org/10.22399/ijcesen.3784
F. Ullah, J. Moon, H. Naeem, and S. Jabbar, Explainable artificial intelligence approach in combating real-time surveillance of COVID19 pandemic from CT scan and X-ray images using ensemble model, Journal of Supercomputing, vol. 78, no. 17, pp. 19246–19271, Nov. 2022.
F. Ahmed, M. Asif, M. Saleem, U. F. Mushtaq, and M. Imran, Identification and Prediction of Brain Tumor Using VGG-16 Empowered with Explainable Artificial Intelligence, International Journal of Computational and Innovative Sciences, vol. 2, no. 2, pp. 24–33, Jun. 2023, Available: ttps:// index.php/IJCIS/article/view/69.
Hussain and R. Jany, Interpreting stroke-impaired electromyography patterns through explainable artificial intelligence, Sensors, vol. 24, no. 5, Mar. 2024.
M. Westerlund, J. S. Hawe, M. Heinig, and Heribert Schunkert,Risk prediction of cardiovascular events by exploration of molecular data with explainable artificial intelligence, International Journal of Molecular Sciences, vol. 22, no. 19, Oct. 2021.
S. I. Nafisah and G. Muhammad, Tuberculosis detection in chest radiograph using convolutional neural network architecture and explainable artificial intelligence, Neural Computing and Applications, vol. 36, no. 1, pp. 111–131, Jan. 2024.
Sanjana, V. Sowmya, E. A. Gopalakrishnan, and K. P. Soman, Explainable artificial intelligence for heart rate variability in ECG signal, Healthcare Technology Letters, vol. 7, no. 6, pp. 146–154, Dec. 2020.
Schweizer et al., Analysing cerebrospinal fluid with explainable deep learning: From diagnostics to insights, Neuropathology and Applied Neurobiology, vol. 49, no. 1, Feb. 2023.
Gimeno et al., Explainable artificial intelligence for precision medicine in acute myeloid leukemia, Frontiers in Immunology, vol.13, Sep. 2022.
Anwer Mustafa Hilal et al., Modeling of explainable artificial intelligence for biomedical mental disorder diagnosis, Computers, Materials and Continua, vol. 71, no. 2, pp. 3853–3867, 2022.
Samanta Knapič, A. Malhi, R. Saluja, and K. Främling, Explainable artificial intelligence for human decision support system in the medical domain, Machine Learning and Knowledge Extraction, vol. 3, no. 3, pp. 740–770, Sep. 2021.
Q. Hu et al., Explainable artificial intelligence-based edge fuzzy images for COVID-19 detection and identification Applied Soft Computing, vol. 123, Jul. 2022.
Bader Aldughayfiq, F. Ashfaq, N. Z. Jhanjhi, and M. Humayun, Explainable AI for retinoblastoma diagnosis: Interpreting deep learning models with LIME and SHAP, Diagnostics, vol. 13, no. 11, Jun. 2023.
Jeong Kyun Kim, Myung Nam Bae, K. Lee, Jae Chul Kim, and Sang Gi Hong, Explainable artificial intelligence and wearable sensor-based gait analysis to identify patients with osteopenia and sarcopenia in daily life, Biosensors, vol. 12, no. 3, Mar. 2022.
T. Mahmud, K. Barua, Sultana Umme Habiba, Nahed Sharmen, Mohammad Shahadat Hossain, and K. Andersson, An explainable AI paradigm for alzheimer’s diagnosis using deep transfer learning, Diagnostics, vol. 14, no. 3, Feb. 2024.
S. D. Mohanty, D. Lekan, T. P. McCoy, M. Jenkins, and P. Manda, Machine learning for predicting readmission risk among the frail: Explainable AI for healthcare, Patterns, vol. 3, no. 1, Jan. 2022.
J. Ma et al., Towards trustworthy AI in dentistry, Journal of Dental Research, vol. 101, no. 11, pp. 1263–1268, Oct. 2022.
C. Duckworth et al., Using explainable machine learning to characterize data drift and detect emergent health risks for emergency department admissions during COVID-19, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Merry, P. Riddle, and J. Warren, A mental models approach for defining explainable artificial intelligence, BMC Medical Informatics and Decision Making, vol. 21, no. 1, Dec. 2021.
Aslam, Explainable artificial intelligence approach for the early prediction of ventilator support and mortality in COVID-19 patients, Computation, vol. 10, no. 3, Mar. 2022.
L. M. Thimoteo, M. M. Vellasco, J. Amaral, K. Figueiredo, Cátia Lie Yokoyama, and E. Marques, Explainable artificial intelligence for COVID-19 diagnosis through blood test variables, Journal of Control, Automation and Electrical Systems, vol. 33, no. 2, pp. 625–644, Apr. 2022.
Moreno-Sánchez, Improvement of a prediction model for heart failure survival through explainable artificial intelligence, Frontiers in Cardiovascular Medicine, vol. 10, 2023.
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler, The enlightening role of explainable artificial intelligence in chronic wound classification, Electronics (Switzerland), vol. 10, no. 12, Jun. 2021.
Most read articles by the same author(s)
- Sashank Siwakoti, Bhaskar Chaganti, Human-in-the-Loop Control Planes for Cortex Agents: Policy-Driven Escalation, Approval, and Evidence Capture , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Mohammed Imran Choudhary, AI-Augmented Network-Forensics: Leveraging LLMs for Real-Time Threat Detection and Automated Response in Enterprise Environments , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Priya Sharma, A Data-Centric Approach to Transforming Digital Retail Through Artificial Intelligence-Based Shopping Systems , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Suprajyotsna Dasari , Automated Testing Techniques for Enterprise Software Systems with GenAI Integration , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Sonam Kumari, Enhancing Clinical Decision-Making Using Generative AI-Powered Knowledge Retrieval Systems: A Review of Emerging Approaches and Challenges , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 08 (2026): Volume 03 Issue 08
- Myroslav Mishov, Autonomous Threat Remediation in Localized AI Environments: A Review of Security-as-Code Execution Models , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 06 (2026): Volume 03 Issue 06
- Nimal Perera, Anjali Fernando, Robust Browser Fingerprinting Under Adversarial Conditions: An AI-Driven Detection and Defense Architecture , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Nguyen Minh Anh, Tran Quoc Bao, Unsupervised Learning Framework for Country Clustering Based on Agricultural Import Patterns , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 09 (2026): Volume 03 Issue 09
- Dr. Eleni Markou, Narrative Intelligence In The Age Of Generative Ai: Integrating Computational Storytelling, Transformer Architectures, Ethical Governance, And Consumer Impact , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 03 (2026): Volume 03 Issue 03
- Amit Kumar Dhariwal, Comprehensive Study on the Use of Artificial Intelligence to Minimize Bias in Healthcare Succession Management , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 08 (2026): Volume 03 Issue 08
Similar Articles
- Angelo soriano, Sheila Ann Mercado, The Convergence of AI And UVM: Advanced Methodologies for the Verification of Complex Low-Power Semiconductor Architectures , International Journal of Advanced Artificial Intelligence Research: Vol. 2 No. 11 (2025): Volume 02 Issue 11
- Dr. Alejandro Moreno, An Explainable, Context-Aware Zero-Trust Identity Architecture for Continuous Authentication in Hybrid Device Ecosystems , International Journal of Advanced Artificial Intelligence Research: Vol. 2 No. 11 (2025): Volume 02 Issue 11
- Kwame Asare, AI-Powered Predictive Exception Management and Supply Network Coordination in SAP S/4HANA Manufacturing , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 10 (2026): Volume 03 Issue 10
- Mason Johnson, Forging Rich Multimodal Representations: A Survey of Contrastive Self-Supervised Learning , International Journal of Advanced Artificial Intelligence Research: Vol. 2 No. 11 (2025): Volume 02 Issue 11
- Dr. Janis Ozols, Dr. Elina Berzina, Intelligent Local Learning Architecture for Efficient Kernel-Based Data Analytics and Predictive Modeling , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 08 (2026): Volume 03 Issue 08
- Dr. Liu Wei, Zhang Yiming, Chen Xiaorui, E-COMMERCE RECOMMENDATIONS THROUGH GEOGRAPHIC CONTEXT AND POPULATION CHARACTERISTICS , International Journal of Advanced Artificial Intelligence Research: Vol. 1 No. 01 (2024): Volume 01 Issue 01
- Dr. Michael Lawson, Dr. Victor Almeida, Securing Deep Neural Networks: A Life-Cycle Perspective On Trojan Attacks And Defensive Measures , International Journal of Advanced Artificial Intelligence Research: Vol. 1 No. 01 (2024): Volume 01 Issue 01
- Dr. Matteo Rossi, Dr. Aisha El-Sayed, META-LEARNING DRIVEN FEW-SHOT DIAGNOSTICS: ADDRESSING RARE DISEASE CLASSIFICATION IN MEDICAL AI , International Journal of Advanced Artificial Intelligence Research: Vol. 2 No. 05 (2025): Volume 02 Issue 05
- Dr. Aarav Sharma, Dr. Meera Kulkarni, An Integrated NDVI-Driven Predictive Model for Assessing Protein Concentration in Rice Crops and Nitrogen Status in Rice Leaves Through Aerial Imaging and Regression Analysis , International Journal of Advanced Artificial Intelligence Research: Vol. 3 No. 07 (2026): Volume 03 Issue 07
- Jeyakumar Ramachandran, Confidential AI Cloud Architecture for Secure Enterprise Data Processing and Intelligent Workloads , International Journal of Advanced Artificial Intelligence Research: Vol. 2 No. 12 (2025): Volume 02 Issue 12
You may also start an advanced similarity search for this article.