External validation of a cloud-based artificial intelligence platform to detect atrial fibrillation from single lead electrocardiograms

Atrial fibrillation (AF) is commonly reported in the general population and is associated with significant mortality and morbidity. While easy to use smartphone-based portable devices exist to record 1-lead ECG, the ability of commercially available software to automatically detect AF using those devices remains limited with poor positive predictive value (PPV).

We aimed to conduct an external validation of an existing cloud-based deep learning platform for the automatic detection of AF in a large cohort of patients with 1-lead ECG records.

8,528 patients with 1-lead 30 seconds ECGs from an handheld device originating from a Cardiology Challenge were included in this study. Ground truth for the presence or absence of AF was obtained from the challenge labeling process including both a benchmark algorithm and manual labeling by experts. The could-based Artificial Intelligence (AI) platform, whose deep learning algorithms were not trained with ECGs recorded by this type of handheld devices, was used to automatically detect cardiac arrhythmias including AF. Additional reading was conducted by a cardiology experts committee to review false positive (FP) and false negative (FN) cases. Performance metrics including sensitivity, specificity, accuracy, F1-score, PPV, and negative predictive value (NPV) for AF classification were computed considering AI’s AF detection with standalone cloud-based AI software as a medical device (SaMD) versus challenge expert labeling or additional expert committee labeling.

The AI platform achieved an accuracy of 96.1% and 96.4%, a sensitivity of 83.3% and 84.2%, a specificity of 97.3% and 97.6%, and a F1-score of 79.0% and 80.9%, when considering the initial challenge labels and additional expert review as the ground truth, respectively (Table 1, Figure 1). PPV was reported as 75.2% and 78.0% and NPV as 98.4% and 98.4%, largely exceeding previously reported metrics using commercial software to detect AF from same 1-lead ECG records. In addition to AF, the AI platform automatically detected other arrhythmias present on those ECG records such as different types of premature ventricular complexes (PVCs) or premature atrial complexes (PACs) along with 1-degree atrioventricular block.

The results of this external validation indicate that the existing AI platform could achieve cardiologist-level accuracy in detecting AF from 1-lead ECG records. Single-channel portable ECG devices coupled with cloud-based and device agnostic deep learning platforms are therefore promising tools for AF screening. Such an AI platform could facilitate and standardize remote AF. It has the potential to improve accuracy in non-cardiology expert healthcare professional interpretation and trigger further tests for effective patient management. Further health economic and outcomes research is necessary to evaluate the impact on healthcare providers and payers of the presented AI solution.Table 1:

AI platform AF classification