Cardiology hospital admission risk prediction: training, internal validation and technical implementation in the electronic health record

Rising healthcare demand is increasingly outpacing available outpatient capacity in cardiology, where follow-up is often scheduled at fixed intervals despite substantial variation in individual patient risk. This uniform follow-up approach contributes to high outpatient workload and inefficient use of clinical resources. Accurate risk estimation using routinely collected electronic health record (EHR) data may support more individualized follow-up planning by identifying patients at very low risk of mortality or unplanned hospitalization, in whom follow-up intervals could be safely extended.

We developed and validated a machine-learning model as part of the Cardiology Hospital Admission Risk Prediction (CHARP) program. The retrospective baseline cohort comprised 307 792 outpatient visits from 52 989 unique patients at Amsterdam UMC. The primary endpoint was a composite of unplanned cardiac hospitalization or all-cause death within 2 years; the 1-year composite endpoint served as a secondary outcome. Model development and validation were performed in a filtered, leakage-safe prediction cohort using gradient-boosted decision trees (XGBoost) with strict patient-level GroupKFold cross-validation. All predictions and performance metrics were retrospectively evaluated at the outpatient visit (trigger) level. Retrospective model performance was assessed using AUROC, AUPRC, Brier score, calibration curves, and SHAP-based explainability. The final model was technically deployed within the electronic health record to allow automated, visit-level risk estimation in a prospective silent-running environment. In the filtered prediction cohort (199 961 visits), the 2-year composite endpoint prevalence was 16.8%. Across five cross-validation folds, the CHARP model achieved a mean AUROC of 0.77 ± 0.00 and AUPRC of 0.42 ± 0.01, with a Brier score of 0.12, indicating strong overall discrimination and good calibration. Key predictors included NT-proBNP, renal function indices, prior hospitalizations, and cardiac function measures. The deployed CHARP pipeline successfully generated daily risk predictions for all scheduled cardiology outpatients in the EHR environment throughout the silent-running period.

This study shows that machine-learning applied to routine EHR data can deliver clinically meaningful, visit-level risk stratification for cardiology outpatients. The successful EHR integration of CHARP enables prospective evaluation of data-driven follow-up strategies aimed at reducing outpatient clinic burden through safe de-intensification of follow-up for low-risk patients.