Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Review Article
T
here is a natural tendency to simplify the findings of a clini- From the Department of Medical Statis-
cal trial into a binary conclusion: Was there a positive outcome or not? tics, London School of Hygiene and Trop-
ical Medicine, London (S.J.P.); and Colum-
In order to address this question with some objectivity, attention is typi- bia University Medical Center, New York
cally focused on whether the prespecified measure of success for the primary Presbyterian Hospital, and the Cardio-
outcome has been met that is, whether a P value of less than 0.05 has been vascular Research Foundation all in
New York (G.W.S.). Address reprint re-
achieved for the difference in treatments. In reality, a more nuanced interpretation quests to Dr. Pocock at the Department
requires a thorough examination of the totality of the evidence, including second- of Medical Statistics, London School of
ary end points, safety issues, and the size and quality of the trial. In this article, Hygiene and Tropical Medicine, Keppel St.,
London WC1E 7HT, United Kingdom; or
which focuses on the evaluation of positive studies as in our previous article,1 at stuart.pocock@lshtm.ac.uk.
which focused on the appraisal of negative studies our intent is to facilitate
This article was updated on September 8,
a more sophisticated and balanced interpretation of trial evidence. Again, we make 2016, at NEJM.org.
our points using examples from trials involving cardiovascular disease (our area
N Engl J Med 2016;375:971-9.
of expertise), but the messages can be easily applied to other subject areas. DOI: 10.1056/NEJMra1601511
Copyright 2016 Massachusetts Medical Society.
dergoing primary percutaneous coronary inter- was inconclusive. Had the trial continued to its
vention (PCI) were randomly assigned to receive intended completion, a significantly lower rate of
preventive angioplasty (complete revasculariza- death or myocardial infarction may have emerged,
tion during the index procedure) or initial angio- which would have greatly enhanced the value of
plasty of the infarct artery only.25 The hazard the trial.
ratio for the composite primary outcome (refrac- Another recent example is the SPRINT trial31
tory angina, myocardial infarction, or cardiac of intensive versus standard blood-pressure con-
death) with preventive angioplasty versus angio- trol, which had a composite primary outcome of
plasty of the infarct artery only was 0.35 (95% myocardial infarction, acute coronary syndrome,
CI, 0.21 to 0.58; P<0.001), with similarly lower stroke, heart failure, or cardiovascular death. The
risks for each of the three components. This trial was stopped early at a median of 3.26 years
controversial finding was based on relatively few rather than at the intended 5-year follow-up, and
primary events (21 in the intervention group vs. at the time the trial was stopped, the hazard
53 in the group receiving standard care) and ratio for the primary outcome with intensive
selective enrollment (recruitment took 5 years control was 0.75 (95% CI, 0.64 to 0.89; P<0.001).
and was stopped early), giving the impression The exceptional speed to publication was sur-
that the 65% reduction in hazard was too good to prising32; only 4 weeks lapsed between the time
be true. Two subsequent, similarly sized trials26,27 that the trial was stopped and the time at which
showed mixed results. Thus, more evidence is the manuscript was submitted for publication.
needed to justify such a radical change in STEMI The quality and completeness of any interim
management, and the results of the ongoing, database are inevitably imperfect there will
large-scale COMPLETE trial28 are awaited with be outstanding primary (and other) events yet to
interest. be ascertained and adjudicated. In addition, the
moment at which a trial is stopped is the time
Was the Trial Stopped Early? at which an exaggerated estimate of efficacy is
Sometimes a trial is stopped early because in- more likely to be present. Orderly trial closure
terim results show strong evidence of treatment after early stoppage takes several months and is
superiority, which is often a newsworthy event. necessary to achieve robust interpretation of all
Unfortunately, this practice tends to exaggerate the evidence. Earlier reporting of preliminary,
treatment efficacy.29 As a trial progresses, the incomplete results is usually unwise.
estimated treatment effect varies randomly in
relation to the true effect. If the interim estimate Do Concerns about Safety Counterbalance
is based on a randomly high indication of effi- Positive Efficacy?
cacy, it is more likely to cross a statistical stop- When a new treatment has superior efficacy, it is
ping boundary and to convince a data and safety important to identify concerns about safety that
monitoring board that overwhelming evidence might offset the benefits. A balanced account of
of benefit exists. Stopping early also truncates both efficacy and safety must be provided.33 Ab-
evidence for important secondary (and safety) solute benefits and risks should be presented in
outcomes. terms of differences in percentages. Consider-
In the FAME 2 trial,30 for example, PCI was ation of the number needed to treat for benefit
compared with medical therapy alone in patients versus the number needed to harm may provide
with stable coronary artery disease and hemody- a guide to net clinical benefit.
namically significant lesions (as assessed by In the DAPT trial,34 for example, an addi-
means of fractional flow reserve). The trial was tional 18 months of dual antiplatelet therapy
stopped early because the hazard ratio for the versus aspirin alone beginning 1 year after the
primary outcome (all-cause death, myocardial implantation of a drug-eluting stent resulted in
infarction, or urgent revascularization) favoring rates of major adverse cardiac and cerebrovascu-
PCI was 0.39 (95% CI, 0.26 to 0.57; P<0.001). lar events and of stent thromboses (the two
This benefit with PCI was driven by fewer urgent primary efficacy outcomes) that were lower by
revascularizations, an arguably soft outcome 1.6% and 1.0%, respectively. However, this ben-
in an unblinded trial. The rate of death or myo- efit came at the cost of higher rates of major
cardial infarction, although lower with PCI (haz- bleeding events. According to Global Use of
ard ratio 0.79; 95% CI, 0.49 to 1.29; P=0.35), Strategies to Open Occluded Arteries (GUSTO)
Intensive Standard
Treatment Treatment Percent Treatment
Subgroup (N=4678) (N=4683) Difference (95% CI)
no. of events (percent)
Primary outcome 243 (5.2) 319 (6.8)
Death from any cause 155 (3.3) 210 (4.5)
Heart failure 62 (1.3) 100 (2.1)
Serious hypotensive episode 158 (3.4) 93 (2.0)
Serious syncope episode 163 (3.5) 113 (2.4)
Serious acute renal event 204 (4.4) 120 (2.6)
3 2 1 0 1 2 3
criteria, the rate of moderate or severe bleed- Is the Balance of Efficacy and Safety Patient-
ing events was 0.9% higher with continued dual Specific?
antiplatelet therapy, and according to Bleeding The net clinical benefit of a new treatment may
Academic Research Consortium (BARC) criteria, be patient-specific that is, worthwhile for
the rate of bleeding that required medical atten- those at an increased risk for the primary effi-
tion was 2.7% higher. All-cause mortality was cacy outcome but deleterious for those at an in-
0.5% higher with prolonged dual antiplatelet creased risk for adverse events. Calculating the
therapy (P=0.05), and this change was attributed individual patient trade-offs between efficacy
primarily to greater noncardiovascular mortality and safety is not straightforward, and statistical
(P=0.002), although some experts have argued modeling techniques may be useful.35
that the higher rates of death may have been due For the DAPT trial,34 multivariable models
to chance. Debate ensues: Is the net effect of were developed for predicting the risk of myo-
prolonged dual antiplatelet therapy in this popu- cardial infarction or stent thrombosis and the
lation beneficial or harmful? risk of major bleeding.36 In addition to account-
Similarly, in the SPRINT trial,31 intensive low- ing for the duration of antiplatelet treatment
ering of blood pressure resulted in a rate of the (12 months vs. 30 months), these models in-
primary composite cardiovascular outcome that cluded 9 patient and procedural characteristics
was 1.6 percentage points lower and a rate of and were designed to allow determination of the
death that was 1.2 percentage points lower than relative risks of ischemia versus bleeding in indi-
the rates with standard blood-pressure control vidual patients. Limitations included the omis-
during a median follow-up period of 3.26 years. sion of some variables that are known to predict
However, these benefits must be weighed against the risk of ischemia and bleeding, the failure to
rates of hypotension, syncope, and acute kidney directly consider the predictors of mortality, and
injury, which were higher (by 1.4 percentage the absence of external validation from a con-
points, 1.1 percentage points, and 1.8 percent- temporary data set (as yet). Nonetheless, this
age points, respectively) with intensive blood- analysis represents a useful development toward
pressure control (Fig.3). Note that all these individualizing patient care.
benefits and risks, although statistically signifi-
cant, represent small absolute differences. Thus, Are There Flaws in Trial Design or Conduct?
guideline committees, treating physicians, and A highly significant result for the primary out-
patients face a challenge when trying to deter- come goes a long way toward substantiating the
mine which strategy to adopt. view that findings cannot be attributed to chance.
Nonetheless, biases in the design and conduct of control exclusively in patients with type 2 diabetes
the trial must be ruled out before a genuine and reported no effect on cardiovascular events
benefit can be acknowledged. as compared with standard therapy. Whether the
For instance, the first randomized trial of divergent results of the SPRINT and ACCORD
renal denervation in treatment-resistant hyper- trials are due to differences in patient character-
tension, SYMPLICITY HTN-2,37 showed that at istics, medications, trial methods, or other factors
6months, systolic blood pressure was markedly remains undetermined.
lower in the treatment group than in the control The geographic representation in a trial may
group (mean difference, 31 mm Hg; P<0.0001). also affect the generalizability of its results.
However, the absence of blinding introduced Many major trials are multinational, which is an
major issues38 (e.g., placebo and Hawthorne ef- advantage in conferring global meaning. But
fects, ascertainment bias, and regression to the health care practices may vary across regions
mean). In the subsequent, sham-controlled trial, (e.g., the use of primary PCI vs. fibrinolysis in
SYMPLICITY HTN-3,39 renal denervation appeared patients with STEMI). If patient recruitment is
to be ineffective, thereby emphasizing the poten- dominated by one region, worldwide applicabil-
tial unreliability of unblinded trials. ity may be limited. In addition, genetic, anatomi-
Limitations in the completeness and quality cal, environmental, and dietary differences among
of the data can also corrupt the validity of a peoples sometimes make outcomes difficult to
trial. Not all patients fully comply with intended generalize across countries.
treatment regimens, and some withdraw from Similarly, the results from single-center trials
follow-up. Judgment is required in determining must be viewed with caution. Center-specific ef-
whether the extent of nonadherence or with- fects, such as particular systems of care and the
drawals casts doubt on the legitimacy of a trial. background therapies used, may preclude the gen-
For instance, the ATLAS ACS 2TIMI 51 trial40 of eralizability of the findings, and single-center
rivaroxaban versus placebo in patients with acute trials often lack quality-control measures. Results
coronary syndromes showed highly significant from single-center studies, even those with a
between-group differences in favor of the lower reasonable sample size, should rarely serve as
dose of rivaroxaban with respect to both the the basis for changing guidelines unless the re-
primary outcome (cardiovascular death, myocar- sults have been validated in subsequent multi-
dial infarction, or stroke) and cardiovascular center trials. For example, the single-center
death alone. But 27.6% of the patients discontin- TAPAS trial44 of thrombus aspiration during
ued treatment prematurely, and data on vital primary PCI, which involved 1071 patients with
status were missing for 7.2% of the patients STEMI, showed dramatically lower mortality at
factors that introduced uncertainty. These prob- 1 year after PCI and thrombus aspiration than
lems appeared to be greater in this trial than in after conventional PCI (hazard ratio, 0.60; 95%
other large trials that address acute coronary CI, 0.36 to 0.98; P=0.04). In hindsight, this out-
syndromes and contributed to the FDA decision come was unrealistic given the modest benefit in
to withhold its approval of rivaroxaban for this reperfusion success, which was the primary out-
indication.41 come. However, this study led to widespread
adoption of thrombus aspiration for many years.
Do the Findings Apply to My Patients? Two multicenter trials involving more than
The findings of any trial apply to the specific 17,000 patients have now convincingly shown that
patients enrolled and the therapies administered routine thrombus aspiration offers no advan-
(both background and experimental). The ques- tages with regard to mortality or cardiovascular
tion of whether the results can be generalized to events.45,46
other patients must be considered. For instance, Finally, by the time the long-term findings
the SPRINT trial31 excluded patients younger for the primary outcome of a randomized trial
than 50 years of age and those with diabetes or become available, advances in care may have
a history of stroke. The trial results (Fig.3) thus lessened their relevance to contemporary practice.
apply to only approximately 20% of all patients For example, in the SYNTAX and FREEDOM
with hypertension who are seen in practice.42 trials,47,48 patients with left main or multivessel
Investigators in the ACCORD trial43 previously disease were assigned to PCI with first-genera-
considered the use of intensive blood-pressure tion drug-eluting stents or to CABG. However,
References
1. Pocock SJ, Stone GW. The primary sus enalapril in heart failure. N Engl J 15, 2015 (http://www.tctmd.com/show.aspx
outcome fails what next? N Engl J Med Med 2014;371:993-1004. ?id=133383).
2016;375:861-70. 5. Lees KR, Zivin JA, Ashwood T, et al. 9. Guidance for industry:diabetes mel-
2. The Cardiac Arrhythmia Suppression NXY-059 for acute ischemic stroke. N Engl litus evaluating cardiovascular risk in
Trial (CAST) Investigators. Preliminary J Med 2006;354:588-600. new antidiabetic therapies to treat type 2
report: effect of encainide and flecainide 6. Shuaib A, Lees KR, Lyden P, et al. diabetes. Silver Spring, MD:Food and
on mortality in a randomized trial of NXY-059 for the treatment of acute ische Drug Administration, December 2008
arrhythmia suppression after myocardial mic stroke. N Engl J Med 2007;357:562-71. (http://www.fda.gov/downloads/drugs/
infarction. N Engl J Med 1989;321:406- 7. Cannon CP, Blazing MA, Giugliano guidancecomplianceregulatoryinformation/
12. RP, et al. Ezetimibe added to statin therapy guidances/ucm071627.pdf).
3. Pocock SJ, Ware JH. Translating sta- after acute coronary syndromes. N Engl J 10. The Action to Control Cardiovascular
tistical findings into plain English. Lancet Med 2015;372:2387-97. Risk in Diabetes Study Group. Effects of
2009;373:1926-8. 8. ORiordan M. Ezetimibe fails to gar- intensive glucose lowering in type 2 dia-
4. McMurray JJV, Packer M, Desai AS, et ner support for CV-event reduction claim betes. N Engl J Med 2008;358:2545-59.
al. Angiotensinneprilysin inhibition ver- from FDA advisory committee. December 11. Follath F, Cleland JG, Just H, et al. Ef-
ficacy and safety of intravenous levosi- acetylcysteine. N Engl J Med 2000;343: 38. Howard JP, Shun-Shin MJ, Hartley A,
mendan compared with dobutamine in 180-4. Bhatt DL, Krum H, Francis DP. Quantify-
severe low-output heart failure (the LIDO 24. Sun Z, Fu Q, Cao L, Jin W, Cheng L, ing the 3 biases that lead to unintention-
study): a randomised double-blind trial. Li Z. Intravenous N-acetylcysteine for pre- al overestimation of the blood pressure-
Lancet 2002;360:196-202. vention of contrast-induced nephropathy: lowering effect of renal denervation.
12. Mebazaa A, Nieminen MS, Packer M, a meta-analysis of randomized, controlled Circ Cardiovasc Qual Outcomes 2016;9:
et al. Levosimendan vs dobutamine for trials. PLoS One 2013;8(1):e55124. 14-22.
patients with acute decompensated heart 25. Wald DS, Morris JK, Wald NJ, et al. 39. Bhatt DL, Kandzari DE, ONeill WW,
failure: the SURVIVE randomized trial. Randomized trial of preventive angio- et al. A controlled trial of renal denerva-
JAMA 2007;297:1883-91. plasty in myocardial infarction. N Engl J tion for resistant hypertension. N Engl J
13. Fox KAA, Poole-Wilson PA, Henderson Med 2013;369:1115-23. Med 2014;370:1393-401.
RA, et al. Interventional versus conserva- 26. Engstrm T, Kelbk H, Helqvist S, 40. Mega JL, Braunwald E, Wiviott SD,
tive treatment for patients with unstable et al. Complete revascularization versus et al. Rivaroxaban in patients with a re-
angina or non-ST-elevation myocardial treatment of the culprit lesion only in pa- cent acute coronary syndrome. N Engl J
infarction: the British Heart Foundation tients with ST-segment elevation myocar- Med 2012;366:9-19.
RITA 3 randomised trial. Lancet 2002; dial infarction and multivessel disease 41. Krantz MJ, Kaul S. The ATLAS ACS
360:743-51. (DANAMI-3PRIMULTI): an open-label, 2-TIMI 51 trial and the burden of missing
14. Fox KAA, Poole-Wilson P, Clayton TC, randomised controlled trial. Lancet 2015; data: (Anti-Xa Therapy to Lower Cardio-
et al. 5-Year outcome of an interventional 386:665-71. vascular Events in Addition to Standard
strategy in non-ST-elevation acute coro- 27. Gershlick AH, Khan JN, Kelly DJ, et al. Therapy in Subjects With Acute Coronary
nary syndrome: the British Heart Founda- Randomized trial of complete versus Syndrome ACS 2-Thrombolysis In Myo-
tion RITA 3 randomised trial. Lancet 2005; lesion-only revascularization in patients cardial Infarction 51). J Am Coll Cardiol
366:914-20. undergoing primary percutaneous coro- 2013;62:777-81.
15. Bavry AA, Kumbhani DJ, Rassi AN, nary intervention for STEMI and multives- 42. Fuster V. Unraveling the complexities
Bhatt DL, Askari AT. Benefit of early inva- sel disease: the CvLPRIT trial. J Am Coll of statistical presentation: why it is im-
sive therapy in acute coronary syndromes: Cardiol 2015;65:963-72. portant. J Am Coll Cardiol 2015;66:2909-
a meta-analysis of contemporary random- 28. ClinicalTrials.gov. Complete vs culprit- 10.
ized clinical trials. J Am Coll Cardiol only revascularization to treat multi- 43. The ACCORD Study Group. Effects of
2006;48:1319-25. vessel disease after primary PCI for STEMI intensive blood-pressure control in type 2
16. Fox KAA, Clayton TCC, Damman P, (COMPLETE) (https://clinicaltrials.gov/ diabetes mellitus. N Engl J Med 2010;362:
et al. Long-term outcome of a routine ver- show/NCT01740479). 1575-85.
sus selective invasive strategy in patients 29. Pocock S, White I. Trials stopped 44. Vlaar PJ, Svilaas T, van der Horst IC,
with non-ST-segment elevation acute cor- early: too good to be true? Lancet 1999; et al. Cardiac death and reinfarction after
onary syndrome a meta-analysis of indi- 353:943-4. 1 year in the thrombus aspiration during
vidual patient data. J Am Coll Cardiol 30. De Bruyne B, Fearon WF, Pijls NHJ, percutaneous coronary intervention in
2010;55:2435-45. et al. Fractional flow reserveguided PCI acute myocardial infarction study (TAPAS):
17. Mentzer RM Jr, Bartels C, Bolli R, et al. for stable coronary artery disease. N Engl a 1-year follow-up study. Lancet 2008;371:
Sodium-hydrogen exchange inhibition by J Med 2014;371:1208-17. 1915-20.
cariporide to reduce the risk of ischemic 31. The SPRINT Research Group. A ran- 45. Frbert O, Lagerqvist B, Olivecrona
cardiac events in patients undergoing domized trial of intensive versus standard GK, et al. Thrombus aspiration during
coronary artery bypass grafting: results blood-pressure control. N Engl J Med 2015; ST-segment elevation myocardial infarc-
of the EXPEDITION study. Ann Thorac 373:2103-16. tion. N Engl J Med 2013;369:1587-97.
Surg 2008;85:1261-70. 32. Drazen JM, Morrissey S, Campion EW, 46. Jolly SS, Cairns JA, Yusuf S, et al. Ran-
18. Zinman B, Wanner C, Lachin JM, et al. Jarcho JA. A SPRINT to the finish. N Engl domized trial of primary PCI with or with-
Empagliflozin, cardiovascular outcomes, J Med 2015;373:2174-5. out routine manual thrombectomy. N Engl
and mortality in type 2 diabetes. N Engl J 33. Ioannidis JP, Evans SJ, Gtzsche PC, J Med 2015;372:1389-98.
Med 2015;373:2117-28. et al. Better reporting of harms in random- 47. Mohr FW, Morice MC, Kappetein AP,
19. Cholesterol Treatment Trialists (CTT) ized trials: an extension of the CONSORT et al. Coronary artery bypass graft surgery
Collaborators. The effects of lowering LDL statement. Ann Intern Med 2004; 141: versus percutaneous coronary intervention
cholesterol with statin therapy in people 781-8. in patients with three-vessel disease and
at low risk of vascular disease: meta-analy- 34. Mauri L, Kereiakes DJ, Yeh RW, et al. left main coronary disease: 5-year follow-
sis of individual data from 27 randomised Twelve or 30 months of dual antiplatelet up of the randomised, clinical SYNTAX
trials. Lancet 2012;380:581-90. therapy after drug-eluting stents. N Engl J trial. Lancet 2013;381:629-38.
20. Wang R, Lagakos SW, Ware JH, Hunter Med 2014;371:2155-66. 48. Farkouh ME, Domanski M, Sleeper
DJ, Drazen JM. Statistics in medicine 35. Pocock SJ, Stone GW, Mehran R, Clay- LA, et al. Strategies for multivessel revas-
reporting of subgroup analyses in clinical ton TC. Individualizing treatment choices cularization in patients with diabetes.
trials. N Engl J Med 2007;357:2189-94. using quantitative methods. Am Heart J N Engl J Med 2012;367:2375-84.
21. Wallentin L, Becker RC, Budaj A, et al. 2014;168:607-10. 49. Palmerini T, Benedetto U, Biondi-
Ticagrelor versus clopidogrel in patients 36. Yeh RW, Secemsky EA, Kereiakes DJ, Zoccai G, et al. Long-term safety of
with acute coronary syndromes. N Engl J et al. Development and validation of a pre- drug-eluting and bare-metal stents: evi-
Med 2009;361:1045-57. diction rule for benefit and harm of dual dence from a comprehensive network
22. Carroll KJ, Fleming TR. Statistical antiplatelet therapy beyond 1 year after meta-analysis. J Am Coll Cardiol 2015;65:
evaluation and analysis of regional inter- percutaneous coronary intervention. JAMA 2496-507.
actions: the PLATO trial case study. Stat 2016;315:1735-49. 50. Windecker S, Stortecky S, Stefanini
Biopharm Res 2013;5:91-101. 37. Symplicity HTN-2 Investigators. Renal GG, et al. Revascularisation versus medi-
23. Tepel M, van der Giet M, Schwarzfeld sympathetic denervation in patients with cal treatment in patients with stable coro-
C, Laufer U, Liermann D, Zidek W. Pre- treatment-resistant hypertension (The Sym- nary artery disease: network meta-analy-
vention of radiographic-contrast-agent plicity HTN-2 Trial): a randomised con- sis. BMJ 2014;348:g3859.
induced reductions in renal function by trolled trial. Lancet 2010;376:1903-9. Copyright 2016 Massachusetts Medical Society.