Stem cell therapy for heart disease: too good to be true?
Discrepancies in autologous bone marrow stem cell trials and enhancement of ejection fraction (DAMASCENE): weighted regression and meta-analysis
Nowbar AN, Mielewczik M, Karavassilis M, DAMASCENE writing group
BMJ. 2014 Apr 28;348:g2688
BackgroundAutologous bone marrow stem cells (BMSCs) are considered to offer an exciting opportunity for improvement of left ventricular function, reverse modelling and scar size reduction in patients with ischaemic heart disease [1,2]. Trials have, however, yielded conflicting results. The source of the discrepancies between different studies has not been identified.
Recently, unexplained discrepancies in some pioneering trials of autologous BMSCs have raised questions about the validity of these trials . The discrepancies have never been systematically studied as a possible explanation for the variability of the effect size of autologous BMSCs on ejection fraction.
This report examined the published randomised controlled trials of BMSC therapy, in search of discrepancies of design, methods or results and related the number of discrepancies to the reported effect size. A discrepancy was defined as two (or more) reported facts that cannot both be true because they are logically or mathematically incompatible. They were categorised into discrepancies in design, in methods and baseline characteristics or in results.
133 reports were identified, documenting 49 randomised trials on the effect of BMSC on ejection fraction for cardiac disease.
- 604 instances of discrepancy were identified within a trial report or between reports of that trial, and 44 discrepancies in the reports of the study on safety.
- Discrepancies were found in many different aspects of the report, including the primary endpoint. Sometimes discrepancies seemed to affect whether a difference between trial arms would be statistically significant.
- Effect size (increment in ejection fraction) ranged from -3.9 to 14%. Numbers of discrepancies in individual trials across all reports ranged from 0 to 89.
A statistically significant correlation was found between the number of discrepancies and the reported ejection fraction effect size: Spearman’s rho=0.4, P-0.005).
- The five studies with no discrepancies had a weighted mean effect size of -0.4%. 24 trials with 1-10 discrepancies had a mean effect size of 2.1%, 12 studies with 10-20 discrepancies gave 3.0%, 3 studies with 21-30 gave 5.7% on average, and five high discrepancy trials (>30 each) showed a mean effect size of 7.7%.
- A meta-analysis was performed on data of 31 trials, of which the exact standard error of the effect size estimate could be extracted. The weighted mean effect size was 0.0 (95%CI: −4.67 to 4.65) for trials with no discrepancies, 1.9 (95%CI: 0.30 to 3.57) for trials with 1-10 discrepancies, 4.6 (95%CI: 1.64 to 7.61) for trials with 11-20 discrepancies, 4.4 (95%CI: −0.97 to 9.75) for trials with 21-30 discrepancies, and 10.4 (95%CI: 8.44 to 12.36) for high discrepancy trials.
Download Nowbar BMJ 2014 PACE.pptx
DiscussionThis study shows that scientists who achieved progressively better consistency of reporting find progressively smaller effects on ejection fraction of treatment with BMSCs. In trials with a discrepancy count of zero, the effect on ejection fraction seemed to be zero.
Other meta-analyses have previously addressed the effect of BMSC therapy on ejection fraction. They all concluded that the average effect was a significant increase, but they did not discuss the discrepancies of the work. In the current work, when viewing all studies together as a single entity, on average a positive effect on ejection fraction is seen. However, the authors noted that the positivity was not consistent across the spectrum of discrepancy count. Averaging effect size across all studies does not seem valid, as this does not reflect the factual accuracy of the individual studies.
The authors speculate on possible causes of the discrepancies. They had asked for resolution of over 150 discrepancies through journals, all to no avail. Furthermore, they discuss the implications for correctness of values reported in trials and for interpreting trial design, and for the safety of BMSC therapy.
When advising patients on which BMSC trial to enter, maximising the benefit to them as well as maximising their contribution to reliable evidence for future patients is important. Currently, these two aspects seem to be in conflict.
Find this article on Pubmed, to read more specifics on the types of discrepancies identified.
Cochrane systematic review Approximately simultaneously, a Cochrane review  examined the effect of stem cell therapy on ischaemic heart disease and congestive heart failure. Upon evaluation of 23 RCTs, involving 1255 participants, they observed considerable (I2 > 75%) heterogeneity among the studies. It was found that stem cell therapy lowers mortality and rehospitalisation for heart failure on the long term (>12 months), but not in the short term, although for each of these findings evidence was considered to be of low quality. Based on 6 studies, left ventricle ejection fraction was found to be improved by 2.62% on the long-term, for which evidence was of moderate quality.
The authors conclude by saying that ‘they found some evidence for a potential beneficial clinical effect in terms of mortality and performance status in the long term (after at least one year) in people who suffer from chronic ischaemic heart disease and heart failure, although the quality of evidence was low’.
1 Jeevanantham V, Butler M, Saad A, et al., Adult bone marrow cell therapy improves survival and induces long-term improvement in cardiac parameters: a systematic review and meta-analysis. Circulation 2012;126:551-68.
2 Finegold JA, Asaria P, Francis DP. Mortality from ischaemic heart disease by country, region, and age: statistics from World Health Organization and United Nations. Int J Cardiol 2012;168:934-45.
3 Francis DP, Mielewczik M, Zargaran D, Cole GD. Autologous stem cell therapy in heart disease: silent discrepancies and scientific and editorial responses. Int J Cardiol 2013;168:3381-403.
4. Fisher SA, Brusnkill SJ, Doree C et al., Stem cell therapy for chronic ischaemic heart disease and congestive heart failure. Cochrane Database Syst Rev. 2014; Apr 29:4: CD007888.