Unlocking the black box: towards robust clinical use of explainable artificial intelligence (XAI) in acute cardiovascular care

Artificial intelligence (AI) has shown promising performance in the identification of cardiac diseases, like the detection of cardiac ischemia from surface electrocardiograms (ECGs). Especially in acute cardiac care, where rapid decision-making and therefore trust in methods are central, the lack of transparency hinders clinical adoption. To bridge this gap, explainable artificial intelligence (XAI) methods are proposed to visualize model behavior and to enable validation with expert clinical reasoning. The aim of the present study is to implement XAI methods for AI-based disease identification in ECGs in the context of acute cardiovascular care.

To implement XAI methods for visualization of model behavior, we used our previously developed convolutional neural network (CNN) capable of discriminating myocardial infarction in emergency department (ED) patients presenting with the chief complaint of chest pain. The model architecture is a ResNet-inspired 1D CNN, incorporating multi-scale dilated convolutions, squeeze-and-excitation (SE) blocks, and dropout layers for robustness and generalization (Figure 1A). For training and testing data, the MIMIC-IV-ED dataset was used together with the MIMIC-IV-ECG dataset. MIMIC-IV is derived from electronic health records of all patients admitted to the ED or ICU of Beth Israel Deaconess Medical Center between 2008 and 2019. Model training was conducted on 29,001 ECGs, including 1,746 ischemia cases. To investigate model interpretability, three XAI attribution techniques were implemented: (1) Gradient-weighted Class Activation Mapping (Grad-CAM), a visualization emphasizing class-discriminative regions; (2) Saliency Maps, based on input gradients to highlight sensitive regions; and (3) SmoothGrad×SIGN, a technique that combines noise smoothing and signal-domain enhancements for clearer visual attributions.

Our model achieved an AUROC of 0.83 for cardiac ischemia detection from 12-lead ECGs. All three XAI methods successfully visualized the model's decision-making process. To enhance interpretability, XAI outputs were similarly overlaid onto ECGs (Figure 1B: Grad-CAM, 1C: Saliency Map, 1D: SmoothGrad×SIGN). Interestingly, while all methods highlighted clinically relevant regions, the specific areas identified varied among methods.

We implemented three XAI methods for the visualization of AI-based interpretations of surface ECGs in an ED context. Overlaying the XAI outputs onto ECG curves facilitates clinical use. Since the three XAI methods highlight different regions in the same ECG when evaluated by the same model, further research is required. The next step will be a systematic evaluation of XAI methods incorporating the expertise of clinical domain specialists, aiming to enhance the practical value and trustworthiness of XAI in acute cardiovascular care.