Development of individualised treatment effect prediction models
Personalized cardiovascular disease prevention by applying individualized prediction of treatment effects
van der Leeuw J, Ridker PM, van der Graaf Y et al.
Eur Heart J. 2014 Feb 9. [Epub ahead of print]
BackgroundThe single estimate of effect, such as relative risk or hazard ratios, that are commonly provided in trial publications, is an average group-level estimate. Individuals vary greatly in risk and response to treatment. Thus far, no tools are available that enable clinicians to estimate the absolute effect of treatment for an individual patient. Thus, the same treatment is commonly administered to a wide range of patients who are presumed to resemble the ‘mean’ patient behind the point estimate of treatment effect.
Ideally, treatment is given to those patients who are likely to have the greatest benefit and the least harm, but information on who these patients are is rarely available. Data from large randomised controlled trials and trial meta-analyses can be ‘re –used’ to develop multivariable prediction models that provide an estimate of absolute treatment effect for individual patients based on their specific characteristics.
This paper aims to illustrate how to translate group-level evidence from large CV risk management trials to the treatment of individual patients in everyday clinical practice by applying treatment effect prediction models. By selecting the right patients for treatment, the number of patients that are treated unnecessarily may be reduced, as well as treatment-related harm and healthcare costs.
Current clinical practiceTreatment decisions are based on risk factor thresholds , pre-treatment CVD risk and the average treatment effect known from landmark trials. This view is rather simplistic since patients vary greatly, and somewhat arbitrary cut-off values suggest that risk suddenly increases when a risk factor reaches a certain level. Thus, the commonly dichotomous classification of risk factors as ‘normal’ or ‘abnormal’ are insufficient to estimate the potential benefit a given patient might derive from treatment. An absolute risk reduction (ARR) might better inform the clinician and patient about the estimated treatment effect.
How can individual treatment effect be calculated?The effect of treatment for an individual patient, thus de ARR for this patient, can be calculated as the difference between the estimated risk of events without treatment and the estimated risk of events with treatment. The former can be calculated with existing risk prediction tools, either for patients free of vascular disease or with a history of CV disease. The risk for an individual patient with treatment can be obtained by multiplying pre-treatment risk by the average relative risk ratio that was seen in the trial. Calculating the ARR for an individual patient in this way is only possible provided that a risk prediction tool is available for this patient, for the outcome of interest, and provided treatment is consistent across patient subgroups. If evidence exists of a treatment interaction or no suitable model is available, a new prediction model can be developed on the data of the clinical trial, that takes into account a treatment term.
Examples of variation in individual treatment effects are given in the article.
An average ARR may be difficult to interpret and to apply to a single patient, thus most clinicians find the number-needed-to-treat (NNT) a more intuitive method to express the benefit that may be anticipated from treatment. NNT is however also a point estimate based on ‘average’ individuals in a clinical trial. Thus, an individual NNT (iNNT) based on multiple patient characteristics would be more informative, and can be calculated with multivariable prediction models using data available from trials. Such an iNNT represents the number of individuals with the same characteristics that need to be treated to prevent one event. It is still a group estimate, but with more precision about a specific set of relevant clinical characteristics.
Interpretation of treatment effects with individual number-needed-to-treat
Weighing treatment benefits and harms for individual patientsPhysicians and patients need to consider whether beneficial effects of a treatment outweigh potential negative effects. The beneficial effect of treatment of a CVD risk needs to be seen in terms of reduction of the risk of vascular events, rather than changing the level of a biomarker.
It can be difficult to accurately model the excess risk of adverse events on an individual basis, thus to determine a treatment threshold at which the positive and negative effects of treatment are considered to be equal. Large trials often cannot provide relevant risk estimates of adverse events, because these events are rare and because patients at high risk may have been excluded. Risk scores from large cohort studies do not estimate the effect of treatment, and are partly a reflection of patients who need the drug. Thus, treatment threshold may sometimes need to be determined based on an average estimate of harm. Threshold ARR may also be expresses as a number-willing-to- treat (NWT), which represents the number of patients one is prepared to treat to prevent one event.
Effects of individualized treatment prediction on population levelAttempts have been made to evaluate whether standard use of treatment effect prediction models in clinical practice is a better approach than guideline-recommended strategies. Prediction-based treatment of patients with the highest estimated effect can result in treating fewer patients while still preventing most events. Plotting the net clinical benefit of various strategies yields a decision curve; relative weighing of positive and negative effects of treatment will help determine the optimal strategy on a population level, for a given condition and treatment.
Current and future perspective of individualized treatment prediction for clinical practiceTreatment effect calculators to estimate the individual effect, as described in this paper, have already been published and made available for statins and aspirin in the primary prevention of CVD, as well as for the individual incremental benefit of high vs. usual-dose statin therapy in secondary prevention (see www.vasculairegeneeskundeutrecht.nl/calculators). More individualized treatment effect prediction models are currently under development. Ultimately, an integrated calculator estimating individualized treatment effects could be linked to electronic patient records and automatically present up-to-date individual effect estimates.
Potential effects on adherencePatients may become more engaged in treatment decisions when they are more aware of the individual risks and benefits of treatment. As a consequence, treatment adherence may improve.
LimitationsThe most important criticism of prediction-based treatment is probably that doctors do not use prediction models because they are complicated and time-consuming. Widespread use of electronic patient records can facilitate implementation of the models in clinical practice.
It should be kept in mind that the available trials on which the prediction models are based, generally have a relatively short follow-up time, although meaningful CVD predictions usually cover a 10-year period. Thus, data need to be extrapolated. This is, however, also true for prediction of the ‘average’ treatment effect. Predictive models always need to be validated. As with recommendations on ‘average’ treatment effects, generalisability is a concern, since trial participants are a selected , often healthier and more compliant, population. Since assessing the effect of a single treatment only partly reflects clinical practice where patient are often treated with multiple drugs, considering the estimated ARRs of several different treatments may help prioritise risk factor management.
Clinical trials provide a lot of information that is not exploited when only presenting an average effect of treatment. Incorporating multiple patient characteristics into a therapeutic prediction model can yield individual estimates of treatment effect. This approach can help improve individual patient management, can identify patients who will benefit most from treatment, and reduce unnecessary treatment and healthcare costs.
Find this article on Pubmed