Predicting residual risk using machine learning in AF patients on OAC

In the GLORIA-AF registry phase II/III, 4 machine learning models outperformed 2 traditional clinical risk scores in predicting residual thrombotic risk in AF patients receiving oral anticoagulation (OAC) therapy.

This summary is based on the publication of Liu Y, Chen Y, Olier I, et al. - Residual risk prediction in anticoagulated patients with atrial fibrillation using machine learning: A report from the GLORIA-AF registry phase II/III. Eur J Clin Invest. 2024 Dec 11:e14371 [Online ahead of print]. doi: 10.1111/eci.14371

Introduction and methods

Background

Although oral anticoagulation (OAC) decreases the risk of thromboembolism in patients with AF, they still have a risk of thrombotic events [1-4]. Assessment of this residual risk may be beneficial for secondary prevention in this population.

Aim of the study

The authors explored the utility of machine learning (ML) techniques in predicting residual thrombotic risk in AF patients receiving OAC therapy.

Methods

Data of 15,829 patients with newly diagnosed (<3 months before baseline visit) nonvalvular AF were collected from the global, multicenter, prospective GLORIA-AF (Global Registry on Long-Term Oral Anti-Thrombotic Treatment in Patients with Atrial Fibrillation) registry phase II and III [5]. Median follow-up duration was 1176 days (IQR: 228–1545). The dataset was randomly split into training and test sets in a 9:1 ratio. To predict the residual risk of the composite outcome of thrombotic events (defined as ischemic stroke, non–central nervous system arterial embolism, TIA, or MI), the authors constructed 4 algorithm-based prediction models: Logistic Regression (LR), Random Forest (RF), Light Gradient Boosting Machine (LGBM), and Extreme Gradient Boosting Machine (XGBM). In these ML models, additional risk factors and continuous variables that are not considered in conventional clinical risk scoring systems were incorporated. To optimize the hyperparameters of the models, they used the grid search method with 10-fold cross-validation. The models’ performances were mainly evaluated by the area under the receiver-operating characteristic curve (AUC), G-means, and F1 scores. In addition, performances were compared with those of the CHA₂DS₂-VA and 2MACE scores, which are both based on clinical risk factors.

Main results

• During follow-up, 641 patients (4.0%) experienced thrombotic events (243 (1.5%) ischemic stroke, 265 (1.7%) MI, 134 (0.8%) TIA, and 31 (0.2%) systemic embolism).

• In the test set, the LR model had the best performance, with a higher AUC trend of 0.712 (95%CI: 0.653–0.772; P=0.500).

• The RF model showed the highest G-means (0.295) and F1 score (0.249). Its AUC was 0.693 (95%CI: 0.631–0.772; P=0.852).

• The performances of most ML models were superior when compared with the CHA₂DS₂-VA score (AUC: 0.698; 95%CI: 0.638–0.754; reference) and 2MACE score (AUC: 0.696; 95%CI: 0.642–0.758; P=0.923). The G-means and F1 scores of these traditional clinical risk scores were also lower.

• Feature importance ranking indicated the top 5 factors associated with residual risk in the LR model were age, history of thromboembolism or MI, OAC discontinuation, eGFR, and sex.

Conclusion

In this study based on the GLORIA-AF registry phase II/III, 4 ML models for prediction of residual thrombotic risk in AF patients receiving OAC therapy–Logistic Regression, Random Forest, Light Gradient Boosting Machine, and Extreme Gradient Boosting Machine–outperformed 2 traditional clinical risk scores (CHA₂DS₂-VA and 2MACE). The top 5 influential factors were age, history of thromboembolism or MI, OAC discontinuation, eGFR, and sex. The authors do remark that “ML models, while powerful in data mining and feature discovery, are not as easily calculated as traditional scores, which limits their broader application.”

Find this article online at Eur J Clin Invest.

References

1. Ruff CT, Giugliano RP, Braunwald E, et al. Comparison of the efficacy and safety of new oral anticoagulants with warfarin in patients with atrial fibrillation: a meta-analysis of randomized trials. Lancet. 2014;383(9921):955-962.
2. Freedman B, Martinez C, Katholing A, Rietbrock S. Residual risk of stroke and death in anticoagulant-treated patients with atrial fibrillation. JAMA Cardiol. 2016;1(3):366-368.
3. Link MS, Giugliano RP, Ruff CT, et al. Stroke and mortality risk in patients with various patterns of atrial fibrillation: results from the ENGAGE AF-TIMI 48 Trial (effective anticoagulation with factor Xa Next Generation in Atrial Fibrillation-Thrombolysis in Myocardial Infarction 48). Circ Arrhythm Electrophysiol. 2017;10(1):e004267.
4. Albertsen IE, Rasmussen LH, Overvad TF, Graungaard T, Larsen TB, Lip GY. Risk of stroke or systemic embolism in atrial fibrillation patients treated with warfarin: a systematic review and meta-analysis. Stroke. 2013;44(5):1329-1336.
5. Huisman MV, Lip GY, Diener HC, et al. Design and rationale of global registry on long-term Oral antithrombotic treatment in patients with atrial fibrillation: a global registry program on long-term oral antithrombotic treatment in patients with atrial fibrillation. Am Heart J. 2014;167(3):329-334.