Optimization of pre-test probability models for obstructive coronary artery disease using explainable artificial intelligence

Computed tomography coronary angiography (CCTA) has emerged as a powerful non-invasive imaging modality for the evaluation of coronary artery disease (CAD), with a global trend towards a "CT-first" strategy in stable patients with suspected CAD. While overall beneficial, this approach requires a shift in resources, straining specific parts of healthcare systems. Pre-test probability (PTP) models stratify risks of obstructive CAD and are intended to result in better utilization of CCTA. However, these models show variability across subpopulations and have modest discriminatory power with low positive predictive values (PPV).

This study aims to leverage explainable artificial intelligence (XAI) for optimization of PTP models.

Patients that underwent a CCTA for suspected CAD with a CAD-Reporting and Data System (CAD-RADS) classification and Coronary Artery Calcium Score (CACS) were included. Obstructive CAD was defined as stenosis >50% (CAD-RADS 3 or higher). Missing data was imputed. The Risk Factor Weighted Clinical Likelihood (RF-CL) model based on clinical risk factors (age, sex, type of symptoms, smoking, hypertension, dyslipidemia, diabetes and familial predisposition) was calculated. Two gradient descent-based decision tree models, "AI-ECG" and "AI-CAC" were developed. AI-ECG is built using clinical risk factors and electrocardiogram (ECG) classification. AI-CAC is built using the aforementioned features with addition of CACS. Sensitivity analysis was performed using a 15% threshold, aligning with the threshold for CCTA acquisition. Comparative analysis was performed using Python 3.10.2, with area under the curve (AUC) analysis using the DeLong test and feature importance calculation using shapely values.

Preliminary results: We identified 6178 patients with a CCTA between 2022 and 2024 in the Cardiology Centers Netherlands (CCN) and included 2083 patients (mean age 57.4 ± 10.2; 48.1% male). Obstructive CAD was present in 413 (19.8%) patients. The RF-CL model was outperformed by the AI-ECG model (AUC: 0.69 vs 0.80, p<0.001) and the AI-CACS model (AUC: 0.69 vs 0.90, p<0.001). Although sensitivity increased in both the AI-ECG and AI-CACS models from 0.46 to 0.87 and 0.83 respectively, PPV increased only in the AI-CACS model (0.37 to 0.61). Most important features were CACS, age, sex, ECG classification and familial predisposition.

Our study demonstrates that the incorporation of XAI with clinical risk factors, ECG classification and CACS improves the pre-test probability estimation of obstructive CAD. ECG classification appears to be a robust feature. Adopting these models can stratify patients at risk for obstructive CAD, enabling targeted use of CCTA, reducing unnecessary testing, and thus alleviate the burden on healthcare systems. However, further external validation is necessary to assess the model’s performance in prospective real-world settings.

Sensitivity analysis