Distilling knowledge from AI models used to detect heart conditions in 12-lead ECG traces

Deep learning matches or exceeds expert arrhythmia detection, but its black-box nature limits clinical use. Explainable AI (XAI) can accelerate cardiovascular research by converting accurate AI predictions into actionable human knowledge, especially in underexplored domains. However, ECG-focused XAI faces issues: analyses limited to single predictions, superficial heatmaps lacking causal depth, and vulnerability to cherry-picking. Concept Relevance Propagation (CRP) offers dataset-wide, concept-level explanations, but has not been adopted for ECG data in cardiovascular applications and lacks thorough medical data validation.

In this work, we tested the hypothesis that CRP can translate AI insights into human-understandable knowledge in a quantitatively and qualitatively verifiable way. To evaluate this, we analyzed a high-performing model predicting atrial fibrillation (AF) and assessed how its patterns align with clinical knowledge.

To enable our analysis, we adapted a state of the art ECG model in PyTorch via knowledge distillation, training on PTB-XL (median age 62, IQR 22, 52% male) and validating on CPSC-2018 (median age 62, IQR 26, 54% male) using AUROC. A custom canonizer/composite enabled CRP through batch-norm-rich residual blocks. We validated implementation choices using Symmetric Relevance Gain (SRG) bit-flipping across layers. Concepts were ranked by mean relevance for AF prediction across both datasets and visualized with RelMax. To test faithfulness, we pruned the model to the top <0.5% most relevant nodes in the final layer and re-evaluated performance. Qualitative interpretation of selected concepts was done with a clinical expert.

Our distilled model reached an AUROC of 0.96 for AF, with pruning to <0.5% of the most relevant last-layer nodes with similar AUROC (+0.01), suggesting good out of distribution performance using just 25 nodes. Concept analysis showed that higher-level concepts focused on consistent ECG regions, while lower-layer ones aligned with specific wave components. Expert adjudication found that, although classification logic was interpretable for single ECGs, the CRP-enabled global analysis revealed that intermediate nodes often combine multiple medical concepts, limiting interpretability (Figure).

This is the first quantitative application of CRP to ECG and arrhythmia analysis. Our open-source pipeline integrates state-of-the-art XAI methods. While CRP enables more quantitative and causal reasoning about model decisions, our findings also expose current limitations. A key challenge ahead is improving concept purity, potentially by incorporating it directly into model training.

Relevance Heatmap of Concept 9 for AF