Practical implementation lessons from the trenches of cardiovascular decision support deployment

Many cardiovascular clinical decision support systems (CDSS) fail during clinical implementation despite strong performance in research [1-3]. This highlights a gap between development and real-world use, where development often prioritises technical performance over implementation readiness [4,5]. This challenge is exemplified by ARGUS, a machine learning-based CDSS for estimating pre-test probability of chronic coronary artery disease. This study describes our approach to overcome barriers encountered during ARGUS deployment, offering practical lessons learned to bridge the gap between development and clinical use.

To identify real-world deployment challenges and key design considerations encountered during the implementation of ARGUS, and to provide actionable recommendations for improving implementation of CDSS in healthcare institutions.

We conducted a mixed-methods design analysis, based on existing frameworks, workflow observations and data analysis. We identified three practical design considerations: data availability, workflow integration and clinical guideline adherence. We systematically addressed these by developing a data availability pipeline to map accessible data at relevant decision timepoints, creating a detailed clinical workflow overview in collaboration with cardiologists, and analyzing relevant cardiology protocols to align ARGUS with established diagnostic guidelines.

Among 9,029 cardiology consultations, only 26.6% had all required features accessible at the time of decision making (Figure 1). The end of the first consultation was identified as critical decision timepoint when CDSS output must be available to guide decision-making. This clarified that any data registered after this point introduces effective missingness for the model (Figure 2). Integration of clinical protocols further refined ARGUS’s intended use, restricting it to symptomatic patients, since asymptomatic patients require different diagnostic steps. These findings demonstrate how data timing, workflow constraints, and clinical protocols directly impact CDSS deployment in an otherwise technically successful algorithm with high AUC. Accounting for these factors early in model development improves implementation robustness.

Early consideration of real-world constraints is crucial for the successful implementation of clinical AI tools. Moving from promising research to practical application requires early alignment with data availability, clinical workflows and protocols. This is not optional, it is the difference between successful deployment and starting over.Figure 1:

Data completeness distribution