ENHANCING TRUST AND CLINICAL ADOPTION: A SYSTEMATIC LITERATURE REVIEW OF EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI) APPLICATIONS IN HEALTHCARE
Keywords:
Explainable Artificial Intelligence (XAI), Healthcare AI, Systematic Review, Clinical AdoptionAbstract
Background: The transformative potential of Artificial Intelligence (AI) in healthcare is hampered by the "black box" problem, where a lack of transparency in decision-making fundamentally undermines clinician trust and creates barriers to clinical adoption. Explainable Artificial Intelligence (XAI) is proposed as a necessary solution to bridge the gap between high-performance AI models and the critical need for justification and accountability in patient care.
Methods: This systematic literature review was conducted in adherence to PRISMA guidelines, analyzing literature published between January 2020 and early 2024. A rigorous search across major databases identified 50 relevant primary studies on XAI applications in clinical and biomedical contexts. Data extracted included the medical domain, AI model, XAI technique, and reported impact on trust and accuracy.
Results: Analysis of the 50 studies demonstrated a wide application of XAI across diverse medical fields, including diagnostics, medical imaging, and disease prediction. XAI—especially methods like SHAP, LIME, and GRAD-CAM—was found to significantly enhance interpretability, transparency, and diagnostic accuracy in these applications, successfully building clinician confidence in AI systems. The primary applications were observed in areas like chronic wound classification, cancer diagnosis, and cardiovascular risk prediction.
Conclusion: XAI is paramount for the safe and effective integration of AI into clinical practice. However, real-world integration is associated with persistent technical and data-quality challenges, including inconsistent validation and biased datasets. Future efforts must prioritize the development of standardized frameworks and regulatory compliance to ensure safe, ethical, and fully explainable AI use in healthcare.
References
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler, A Highly Transparent and Explainable Artificial Intelligence Tool for Chronic Wound Classification: XAI-CWC, Jan. 2021.
M. Merry, P. Riddle, and J. Warren, “A mental models approach for defining explainable artificial intelligence,” BMC Medical Informatics and Decision Making, vol. 21, no. 1, Dec. 2021.
J. A. Yeung, Y. Y. Wang, Z. Kraljevic, and J. T. H. Teo, Artificial intelligence (AI) for neurologists: do digital neurones dream of electric sheep?, Practical Neurology, vol. 23, no. 6, pp. 476–488, Dec. 2023.
Srilatha, S. (2025). Integrating AI into enterprise content management systems: A roadmap for intelligent automation. Journal of Information Systems Engineering and Management, 10(45s), 672–688. https://doi.org/10.52783/jisem.v10i45s.8904
Bohr and K. Memarzadeh, The rise of artificial intelligence in healthcare applications, Artificial Intelligence in Healthcare, vol. 1, no. 1, pp. 25–60, Jun. 2020.
T. Davenport and R. Kalakota, The Potential for Artificial Intelligence in Healthcare, Future Healthcare Journal, vol. 6, no. 2, pp. 94–98, Jun. 2019.
K. B. Johnson et al., Precision Medicine, AI, and the Future of Personalized Health Care, Clinical and Translational Science, vol. 14, no. 1, Oct. 2020, Available: https/www.ncbi.nlm.nih.gov/pmc/articles/PMC7877825/.
C. J. Kelly, A. Karthikesalingam, M. Suleyman, G. Corrado, and D. King, Key challenges for delivering clinical impact with artificial intelligence, BMC Medicine, vol. 17, no. 1, Oct. 2019.
J. Bajwa, U. Munir, A. Nori, and B. Williams, Artificial intelligence in healthcare: transforming the practice of medicine, Future Healthcare Journal, vol. 8, no. 2, pp. e188–e194, 2021.
S. A. Alowais et al., Revolutionizing healthcare: the role of artificial intelligence in clinical practice, BMC Medical Education, vol. 23, no. 1, Sep. 2023.
H. Alami et al., Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity, Journal of Medical Internet Research, vol. 22, no. 7, p. e17707, Jul. 2020.
T. Raclin et al., Combining Machine Learning, Patient-Reported Outcomes, and Value-Based Health Care: Protocol for Scoping Reviews,” JMIR Research Protocols, vol. 11, no. 7, p. e36395, Jul. 2022.
Rangu, S. (2025). Analyzing the impact of AI-powered call center automation on operational efficiency in healthcare. Journal of Information Systems Engineering and Management, 10(45s), 666–689. https://doi.org/10.55278/jisem.2025.10.45s.666
B. Vasey et al., DECIDE-AI: new reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence, Nature Medicine, vol. 27, no. 2, pp. 186–187, Feb. 2021.
Moher, D., Liberati, A., Tetzlaff, J., and Altman, D. G. (2009). Preferred reporting items for systematic reviews and metaanalyses: the PRISMA statement. Ann. Inter. Med. 151, 264–269.
Kwang Sig Lee and Eun Sun Kim, Explainable artificial intelligence in the early diagnosis of gastrointestinal disease, Diagnostics, vol. 12, no. 11, Nov. 2022.
A.-D. Samaras et al., Explainable classification of patients with primary hyperparathyroidism using highly imbalanced clinical data derived from imaging and biochemical procedures, Applied Sciences, vol. 14, no. 5, p. 2171, Mar. 2024.
U. Pawar, S. Rea, Ruairi O'reilly, and D. O’shea, Incorporating explainable artificial intelligence (XAI) to aid the understanding of machine learning in the healthcare domain, 2020. Available: .
Sergiusz Wesołowski et al., An explainable artificial intelligence approach for predicting cardiovascular outcomes using electronic health records, PLOS Digital Health, vol. 1, no. 1, p. e0000004, Jan. 2022.
Chadha, K. S. (2025). Zero-Trust Data Architecture for Multi-Hospital Research: HIPAA-Compliant Unification of EHRs, Wearable Streams, and Clinical Trial Analytics. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3477
Das and P. Rad, Opportunities and challenges in explainable artificial intelligence (XAI): A survey, Jun. 2020. Available: .
V. Sharma, Samarth Chhatwal, and B. Singh, An explainable artificial intelligence based prospective framework for COVID-19 risk prediction.
José Jiménez-Luna, F. Grisoni, and G. Schneider, “Drug discovery with explainable artificial intelligence,” Nature Machine Intelligence, vol. 2, no. 10, pp. 573–584, Oct. 2020.
D. Dave, H. Naik, S. Singhal, and P. Patel, Explainable AI meets healthcare: A study on heart disease dataset, Nov. 2020. Available: .
J. Hoffmann et al., Prediction of clinical outcomes with explainable artificial intelligence in patients with chronic lymphocytic leukemia, Current Oncology, vol. 30, no. 2, pp. 1903–1915, Feb. 2023.
Khishigsuren Davagdorj, Jang Whan Bae, Van Huy Pham, Nipon Theera-Umpon, and Keun Ho Ryu, Explainable artificial intelligence based framework for non-communicable diseases prediction, IEEE Access, vol. 9, pp. 123672–123688, 2021.
Augusto Anguita-Ruiz, A. Segura-Delgado, R. Alcalá, C. M.1 Aguilera, and Jesús Alcalá-Fdez, EXplainable Artificial2 Intelligence (XAI) for the identification of biologically relevant3 gene expression patterns in longitudinal human studies, insights4 from obesity res5earch, PLoS Computational Biology, vol. 16, no. 4, Apr. 2020.
V. Roessner, J. Rothe, G. Kohls, Georg Schomerus, S. Ehrlich, andC. Beste, Taming the chaos?! Using eXplainable Artificial Intelligence (XAI) to tackle the complexity in mental health research, European Child and Adolescent Psychiatry, vol. 30, no. 8, pp. 1143–1146, Aug. 2021.
Jiten Sardana. (2025). Secure Messaging Protocols for Transactional Health Notifications. Utilitas Mathematica, 122(2), 267–290. Retrieved from https://utilitasmathematica.com/index.php/Index/article/view/2707
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler,A highly transparent and explainable artificial intelligencetool for 6 Journal of Computer Sciences and Applications chronic wound classification: XAI-CWC, 2021.
Salman Muneer et al., An IoMT enabled smart healthcare model to monitor elderly people using Explainable Artificial Intelligence (EAI)., Journal of NCBAE, Vol 1.
Shaker El-Sappagh, J. M. Alonso, S. M.Riazul Islam, A.M. Sultan, and Kyung Sup Kwak, A multilayermultimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Raza, Kim Phuc Tran, L. Koehl, and S. Li, “Designing ECG monitoring healthcare system with federated transfer learning and explainable AI,” Knowl. Based Syst., vol. 236, p. 107763, 2021, Available: .
S. El-Sappagh, J. M. Alonso, S. M. R. Islam, A. M. Sultan, and K. S. Kwak, A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease, Scientific Reports, vol. 11, no. 1, p. 2660, Jan. 2021.
J. Peng et al., An explainable artificial intelligence framework for the deterioration risk prediction of hepatitis patients, Journal of Medical Systems, vol. 45, no. 5, May 2021.
L. Lindsay, S. Coleman, D. Kerr, B. Taylor, and A. Moorhead, Explainable artificial intelligence for falls prediction, in Communications in Computer and Information Science, Springer, 2020, pp. 76–84.
Y. Jia, J. McDermid, T. Lawton, and Ibrahim Habli,“The role of explainability in assuring safety of machine learning in healthcare, IEEE Transactions on Emerging Topics in Computing, vol. 10, no. 4, pp. 1746–1760, Oct. 2022.
F. Vaquerizo-Villar et al., An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea, Computers in Biology and Medicine, vol. 165, Oct. 2023.
F. Xu et al., The clinical value of explainable deep learning for diagnosing fungal keratitis using in vivo confocal microscopy images, Frontiers in Medicine, vol. 8, Dec. 2021.
Z. Naz, Muhammad, T. Saba, A. Rehman, Haitham Nobanee, and Saeed Ali Bahaj, An explainable AI-Enabled framework for interpreting pulmonary diseases from chest radiographs, Cancers, vol. 15, no. 1, Jan. 2023.
Belal Alsinglawi et al., An explainable machine learningframework for lung cancer hospital length of stay prediction, Scientific Reports, vol. 12, no. 1, Dec. 2022.
Esma Cerekci et al., Quantitative evaluation of saliency-based explainable artificial intelligence (XAI) methods in deep learning- based mammogram analysis, European Journal of Radiology, vol. 173, Apr. 2024.
Mohammed Saidul Islam, I. Hussain, Md Mezbaur Rahman, Se Jin Park, and Md Azam Hossain, Explainable artificial intelligence model for stroke prediction using EEG signal, Sensors, vol. 22, no. 24, Dec. 2022.
Z. U. Ahmed, K. Sun, M. Shelly, and L. Mu, Explainable artificial intelligence (XAI) for exploring spatial variability of lung and bronchus cancer (LBC) mortality rates in the contiguous USA, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Samantapudi, R. K. R. (2025). Enhancing search and recommendation personalization through user modeling and representation. International Journal of Computational and Experimental Science and Engineering, 11(3), 6246–6265. https://doi.org/10.22399/ijcesen.3784
F. Ullah, J. Moon, H. Naeem, and S. Jabbar, Explainable artificial intelligence approach in combating real-time surveillance of COVID19 pandemic from CT scan and X-ray images using ensemble model, Journal of Supercomputing, vol. 78, no. 17, pp. 19246–19271, Nov. 2022.
F. Ahmed, M. Asif, M. Saleem, U. F. Mushtaq, and M. Imran, Identification and Prediction of Brain Tumor Using VGG-16 Empowered with Explainable Artificial Intelligence, International Journal of Computational and Innovative Sciences, vol. 2, no. 2, pp. 24–33, Jun. 2023, Available: ttps:// index.php/IJCIS/article/view/69.
Hussain and R. Jany, Interpreting stroke-impaired electromyography patterns through explainable artificial intelligence, Sensors, vol. 24, no. 5, Mar. 2024.
M. Westerlund, J. S. Hawe, M. Heinig, and Heribert Schunkert,Risk prediction of cardiovascular events by exploration of molecular data with explainable artificial intelligence, International Journal of Molecular Sciences, vol. 22, no. 19, Oct. 2021.
S. I. Nafisah and G. Muhammad, Tuberculosis detection in chest radiograph using convolutional neural network architecture and explainable artificial intelligence, Neural Computing and Applications, vol. 36, no. 1, pp. 111–131, Jan. 2024.
Sanjana, V. Sowmya, E. A. Gopalakrishnan, and K. P. Soman, Explainable artificial intelligence for heart rate variability in ECG signal, Healthcare Technology Letters, vol. 7, no. 6, pp. 146–154, Dec. 2020.
Schweizer et al., Analysing cerebrospinal fluid with explainable deep learning: From diagnostics to insights, Neuropathology and Applied Neurobiology, vol. 49, no. 1, Feb. 2023.
Gimeno et al., Explainable artificial intelligence for precision medicine in acute myeloid leukemia, Frontiers in Immunology, vol.13, Sep. 2022.
Anwer Mustafa Hilal et al., Modeling of explainable artificial intelligence for biomedical mental disorder diagnosis, Computers, Materials and Continua, vol. 71, no. 2, pp. 3853–3867, 2022.
Samanta Knapič, A. Malhi, R. Saluja, and K. Främling, Explainable artificial intelligence for human decision support system in the medical domain, Machine Learning and Knowledge Extraction, vol. 3, no. 3, pp. 740–770, Sep. 2021.
Q. Hu et al., Explainable artificial intelligence-based edge fuzzy images for COVID-19 detection and identification Applied Soft Computing, vol. 123, Jul. 2022.
Bader Aldughayfiq, F. Ashfaq, N. Z. Jhanjhi, and M. Humayun, Explainable AI for retinoblastoma diagnosis: Interpreting deep learning models with LIME and SHAP, Diagnostics, vol. 13, no. 11, Jun. 2023.
Jeong Kyun Kim, Myung Nam Bae, K. Lee, Jae Chul Kim, and Sang Gi Hong, Explainable artificial intelligence and wearable sensor-based gait analysis to identify patients with osteopenia and sarcopenia in daily life, Biosensors, vol. 12, no. 3, Mar. 2022.
T. Mahmud, K. Barua, Sultana Umme Habiba, Nahed Sharmen, Mohammad Shahadat Hossain, and K. Andersson, An explainable AI paradigm for alzheimer’s diagnosis using deep transfer learning, Diagnostics, vol. 14, no. 3, Feb. 2024.
S. D. Mohanty, D. Lekan, T. P. McCoy, M. Jenkins, and P. Manda, Machine learning for predicting readmission risk among the frail: Explainable AI for healthcare, Patterns, vol. 3, no. 1, Jan. 2022.
J. Ma et al., Towards trustworthy AI in dentistry, Journal of Dental Research, vol. 101, no. 11, pp. 1263–1268, Oct. 2022.
C. Duckworth et al., Using explainable machine learning to characterize data drift and detect emergent health risks for emergency department admissions during COVID-19, Scientific Reports, vol. 11, no. 1, Dec. 2021.
Merry, P. Riddle, and J. Warren, A mental models approach for defining explainable artificial intelligence, BMC Medical Informatics and Decision Making, vol. 21, no. 1, Dec. 2021.
Aslam, Explainable artificial intelligence approach for the early prediction of ventilator support and mortality in COVID-19 patients, Computation, vol. 10, no. 3, Mar. 2022.
L. M. Thimoteo, M. M. Vellasco, J. Amaral, K. Figueiredo, Cátia Lie Yokoyama, and E. Marques, Explainable artificial intelligence for COVID-19 diagnosis through blood test variables, Journal of Control, Automation and Electrical Systems, vol. 33, no. 2, pp. 625–644, Apr. 2022.
Moreno-Sánchez, Improvement of a prediction model for heart failure survival through explainable artificial intelligence, Frontiers in Cardiovascular Medicine, vol. 10, 2023.
Salih Sarp, Murat Kuzlu, E. Wilson, U. Cali, and O. Guler, The enlightening role of explainable artificial intelligence in chronic wound classification, Electronics (Switzerland), vol. 10, no. 12, Jun. 2021.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dr. Elias T. Vance, Prof. Camille A. Lefevre (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright of their manuscripts, and all Open Access articles are disseminated under the terms of the Creative Commons Attribution License 4.0 (CC-BY), which licenses unrestricted use, distribution, and reproduction in any medium, provided that the original work is appropriately cited. The use of general descriptive names, trade names, trademarks, and so forth in this publication, even if not specifically identified, does not imply that these names are not protected by the relevant laws and regulations.