From Clinical Trials to Public Health

Policy: The Path from Imaging

to Screening
Leslee J. Shaw,

PhD,

and Rita F. Redberg,

he abundance of new health-care technology is

among the greatest achievements in todays stateT
of-the-art field of cardiovascular medicine. Marked
reductions in mortality have directly resulted from
improved treatment and use of an array of imaging
modalities in cardiovascular medicine.1 Although several available modalities are used for the assessment
of ischemia in treating obstructive coronary disease,
recent developments in imaging have focused on risk
markers for the development of subclinical disease.
Electron-beam computed tomography (EBCT) is such
a modality and has been advocated as a new diagnostic test and screening tool over the past decade.2 Other
imaging modalities that have been used for atherosclerosis detection include magnetic resonance imaging,
intravascular ultrasound, and carotid ultrasound.

CHALLENGES IN DEVELOPING NEW

TECHNOLOGY IN THE ERA OF COST
CONTAINMENT
Costs of care have increased dramatically over the
past 4 decades, often increasing at a rate in excess of
inflation. Controlling rising costs of care has become
a major emphasis on the part of (private and public)
health-care payers. Cardiac diagnostic testing rates
have increased, on average, 5% to 15% annually.3,4
Recent estimates from the American College of Cardiology have revealed that 28% of Medicare reimbursement to cardiologists is for cardiac-imaging procedures, accounting for approximately 1% of all US
health-care expenditures.4 As a result of dramatic increases in spending for cardiac imaging, the introduction of new technologies and the growth of conventional methods have been dampened by a variety of
cost-containment efforts. Thus, many managed-care
organizations have sought to control all health-care
costs by limiting use of cardiovascular services, with
imaging being a major component in new, restrictive,
use guidelines. However, the development of highquality evidence applied in heterogeneous population
samples can be an effective method for shaping large
From the Department of Health Policy and Management, Rollins
School of Public Health, Emory University, Atlanta, Georgia, USA; and
the Division of Cardiology, Department of Medicine, University of
California at San Francisco, San Francisco, California, USA.
Address for reprints: Leslee J. Shaw, PhD, Atlanta Cardiovascular
Research Institute, 5665 Peachtree Dunwoody Road, Suite 225, Atlanta, GA 30342. E-mail: lshaw@acai.com

62E

2001 by Excerpta Medica, Inc.

All rights reserved.

MD, MSc

health-care system and payer practice guidelines as

well as societal health policies for the use of cardiac
imaging. In general, the development of a test with
established quality and economic value provides a
basis for effecting progressive changes to the incidence and prevalence of obstructive coronary disease
and its associated events in this country. As such, new
cardiac-imaging methods must now provide a justification for use that includes an abundance of supportive evidence for specific clinical indications or population subsets.

EVIDENCE-BASED CARDIAC IMAGING

In years past, the rate-limiting factor to the assimilation of new imaging technology was improved image quality or abnormality detection. Current evidence
suggests, however, that only a small number of standard imaging practices had proven effectiveness data.5
Evidence-based medicine has recently been advocated
as a method for defining the impact of technology
using an outcome-oriented strategy to maximize patient health status.6 By definition, evidence-based
medicine advocates integrating patient outcomes from
published, peer-reviewed literature with clinical reasoning as the basis for establishing clinical pathways
of testing.7,8 Furthermore, recent strategies for garnering support of medical societies for clinical guidelines
or expert consensus documents require (1) a threshold
of evidence to justify the cost of a test and (2) consideration of the best available evidence on the effectiveness, risk, and cost of a procedure before implementation into routine clinical practice.7,8 As evidence-based medicine evolves, new thresholds for
effectiveness data will be used to set higher standards
and establish quality guidelines for reimbursement,
and as a means to allocate ever-increasingly limited
health-care resources.
Based on standards of evidence developed for therapeutic clinical trials, there is a hierarchy of evidence
that unfolds from (1) small observational series to
generate associations; (2) meta-analyses for the generation of a more reliable outcome estimate; (3) larger
observational series and controlled clinical trials to
study mechanisms; and (4) larger, randomized, controlled trials to obtain reliable answers to clinical
questions.7,9 The unfolding of evidence often takes
years of concentrated research efforts, but it allows (1)
the accumulation of clinical and economic outcome
data and the development of standards for reimbursement and (2) the development of practice guidelines,
disease management strategies, and quality standards
0002-9149/01/$ see front matter
PII S0002-9149(01)01787-8

FIGURE 1. A theoretical approach to the unfolding of clinical and economic evidence required to provide supportive justification for
reimbursement in todays health-care environment. Initial tiers of evidence include small observational series, followed by randomized or controlled clinical trials (RCTs) to garner more rigorous and unbiased data. From these data, the development of high-risk
cost-effectiveness models based on targeting treatment (Rx) to high-risk subsets are generated to set standards for reimbursement. A
second tier of evidence is driven by observational data to design clinical guidelines, develop risk-marker tools, and to develop costbased disease (Dz) management strategies aimed at resource efficiency and improving the processes of care. Finally, the goal of this
evidence is to set quality standards to be used for benchmarking and profiling of physicians, as well as laboratory evaluations. Ca
calcium.

for an imaging modality (Figure 1). In this supplement, a review of current randomized clinical trials
with coronary calcium screening as well as a current
review of observational data on the estimation of
prognosis with EBCT are presented.2
Although several investigators have advocated the
evaluation of diagnostic test performance as a means
to compare cardiac-imaging accuracy, the core of evidence-based medicine is the reliance on the development of patient-centered outcomes research (ie, to
determine whether we have enough information on
prognosis, which then becomes the measure of noninvasive test performance). As the identification of
risk is the basis for therapeutic intervention, the evaluation of risk by office-based or noninvasive test
parameters is critical to developing risk reduction
strategies that integrate any cardiac-imaging modality.

RISK ASSESSMENT IN
ASYMPTOMATIC POPULATIONS
A screening modality that accurately classifies risk
could be extremely valuable in the primary-prevention
setting. Using the example of EBCT coronary calcium

screening, the presence and extent of calcification

closely relates to overall atherosclerotic plaque burden.2 A recent synthesis of available observational
evidence was completed by OMalley et al,10 which
revealed wide variability in the risk ratios for coronary
calcium as an estimator of cardiac death or myocardial
infarction. The risk ratios ranged from 1.01 to 22.12,
with a wide range of confidence intervals (CI) from
1.0 to 286.31. This pattern of wide variability (1) is
consistent with evolving new technology, (2) has been
shown in an array of other technologies, and (3) is
largely driven by insufficient sample sizes.11,12 Despite the limited sample size, the summary effect size
was a 4.2-fold increased risk (95% CI, 1.57 to
11.25).10 It is likely that sample sizes of 5,000 to
10,000 individuals may be necessary to achieve sufficient statistical power to detect small (1% to 2%)
differences in rates of cardiac death or myocardial
infarction in these lower-risk populations. Some investigators are actively pursuing the development of
large observational data sets. These data will help us
gain more stable estimations of risk from sufficiently
powered data sets for the estimation of the value of
coronary calcium measures derived from EBCT.
A SYMPOSIUM: FIRST INTERNATIONAL SAI MEETING

63E

FIGURE 2. Event-free survival by combined cardiac risk factor plus coronary risk percentage score. CI confidence interval.

NEW METHODS FOR RISK

ASSESSMENT: MULTIVARIABLE
MODELS, RISK-ADJUSTED
OUTCOMES, AND INCREMENTAL
VALUE

Figure 2 depicts 3-year survival (95% CI) using a risk

factor and coronary-calcium scoring system. This
score was calculated using the following equation:

A critical component to establishing the value of

EBCT is the development of risk-adjusted outcome
models that include the development of multivariable
predictive models integrating conventional risk-factor
assessment with EBCT coronary calcium scores. New
standards for the presentation of risk assessment data
have evolved over the past decade and include the
development of multivariable models and risk-adjusted outcome data.1318 Risk adjustment or controlling for clinical confounders (most commonly, established cardiac risk factors) includes the development
of multivariable models. Several risk-adjusted logistic
regression models have been used, but they vary in
their results.2 The log of the coronary calcium score
was not a significant estimator of cardiac death or
myocardial infarction (odds ratio: 1.24 [95% CI, 0.49
to 3.11]) when adjusting for sex, age, diabetes, left
ventricular hypertrophy (by electrocardiography),
smoking, hypertension, family history of disease, and
cholesterol levels. Similarly, Secci et al19 reported that
the log-transformed calcium score did not enter into
their logistic regression model as a significant independent predictor of hard coronary events. From a
prior published series of 676 individuals, the multivariable model for prediction of subsequent cardiac
death or myocardial infarction at 3 years (model chisquare analysis 35.5, p 0.00001) revealed that
risk-adjusted odds ratios for coronary calcium percentiles were 5.1-fold (95% CI, 1.0 to 26.1) for the 25th
percentile to 11.5-fold (95% CI, 3.1 to 42.9) for the
90th percentile.20 These varied results may be attributable to sample sizes or disease prevalence samples,
but these differences in odds ratios require consideration of the nature of events over time (ie, Cox proportional hazards model). From this series of 676
individuals, a predictive risk score was developed.20
64E THE AMERICAN JOURNAL OF CARDIOLOGY

Score smoking (25 hypercholesterolemia)

(2 diabetes mellitus) (5 family history of
coronary disease) (10 hypertension) (35
coronary calcium [age- & gender-adjusted] percentile
where 0 10th percentile, 1 25th percentile, 2
50th percentile, 3 75th percentile, and 4 90th
percentile).
As can be illustrated using this combined risk-factor
and calcium score, 50% of subjects with the lowest
scores have a 3-year cardiac event-free survival of
98% as compared with a 92% survival for those
20% of patients with the highest scores.
This ability to clearly differentiate risk subsets
using an integrated approach to EBCT and historical
measures is such a mark of the unfolding evidence to
support the utility of coronary calcium screening.
From these results, clinical algorithms of care may be
designed to optimize the intensity of treatment based
on the expected risk by a cardiac risk factor and
calcium score.21
Finally, when evaluating outcome, it is critical to
consider the added value of test information after
considering all of the information available to the
clinician before EBCT referral. In the previously mentioned data set of 676 asymptomatic individuals with
known cardiac risk factors, in patients with 1% risk
of death or myocardial infarction, an additional 63%
of new prognostic information above and beyond established cardiac risk factors was added by knowledge
of the coronary calcium score.21 Certainly, this information will have to be validated in sufficiently large
patient samples; but, by comparison, approximately
13% of new information has been reported for exercise treadmill testing, using the Duke treadmill score,
in estimating 5-year cardiac survival.22

VOL. 88 (2A)

JULY 19, 2001

CONCLUSION
Concordant with standards of evidence-based medicine, new technology is being held to consistently
higher standards than previously established modalities. Several strategies can be explored for the development of acceptable levels of evidence, including the
organized development of high-quality clinical and
economic outcomes data based on well-established
epidemiologic methodologies. Using prior research in
the area of conventional risk-factor development as
well as data on noninvasive test accuracy, it is likely
that a focused strategy for research in EBCT may
produce valuable insight into the incremental value of
testing across a wide array of population subsets.
1. Fuster V. Epidemic of cardiovascular disease and stroke: the three main

challenges. Circulation 1999;99:11321137.

2. ORourke RA, Brundage BH, Froelicher VF, Greenland P, Grundy SM,

Hachamovitch R, Pohost GM, Shaw LJ, Weintraub WS, Winters WL Jr. American College of Cardiology/American Heart Association Expert Consensus Document on electron-beam computed tomography for the diagnosis and prognosis of
coronary artery disease. J Am Coll Cardiol 2000;36:326 340.
3. The American Heart Association: 2001 heart and stroke statistical update.
Available at: http://www.americanheart.org/statistics/medical.html. Accessed
February 2001.
4. The American College of Cardiology: 2000 Medicare physician fee schedule.
Available at: http://www.acc.org/advocacy/advoc_issues/medicare2000.htm. Accessed February 2001.
5. Assessing the efficacy and safety of medical technologies. Washington, DC:
Office of Technology Assessment; September 1978. National Technical Information Service (Springfield, VA), order no. PB-286929.
6. European Parliamentary Technology Assessment. Available at: http://www.
eptanetwork.org/epta/index.php3. Accessed February 2001.
7. Lonn EM, Yusuf S. Evidence based cardiology: emerging approaches in
preventing cardiovascular disease. BMJ 1999;318:13371341.

8. Eddy DM. Clinical Decision Making: From Theory to Practice. Sudbury, MA:

Jones & Bartlett Publishers, 1996.

9. Yusuf S, Anand S. Cost of prevention: the case of lipid lowering. Circulation

1996;93:1774 1776.
10. OMalley PG, Taylor AJ, Jackson JL, Doherty TM, Detrano RC. Prognostic
value of coronary electron-beam computed tomography for coronary heart disease events in asymptomatic populations. Am J Cardiol 2000;85:945948.
11. Shaw LJ, Peterson ED, Kesler K, Hasselblad V, Califf RM. A meta-analysis
of predischarge risk stratification after acute myocardial infarction with stress
electrocardiographic, myocardial perfusion, and ventricular function imaging.
Am J Cardiol 1996;78:13271337.
12. Shaw LJ, Hachamovitch R, Papatheofanis FJ. Outcomes and Technology
Assessment in Nuclear Medicine. Reston, VA: Society of Nuclear Medicine
Press, 1999.
13. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in
developing models, evaluating assumptions and adequacy, and measuring and
reducing errors. Stat Med 1996;15:361387.
14. Shaw LJ, Eisenstein EL, Hachamovitch R, Heller GV, Miller DD. A primer
of biostatistic and economic methods for diagnostic and prognostic modeling in
nuclear cardiology. II. J Nucl Cardiol 1997;4:52 60.
15. Shaw LJ, Hachamovitch R, Eisenstein EL, Kesler KL, Heller GV, Miller DD.
A primer of biostatistic and economic methods for diagnostic and prognostic
modeling in nuclear cardiology. I. J Nucl Cardiol 1996;3:538 545.
16. Hachamovitch R, Shufelt C. Statistical analysis of medical data. I. Univariable analysis. J Nucl Cardiol 2000;7:146 152.
17. Shufelt C, Hachamovitch R. Statistical analysis of medical data. II. J Nucl
Cardiol 2000;7:263266.
18. Hachamovitch R, Shufelt C. Statistical analysis of medical data. III. multivariable analysis. J Nucl Cardiol 2000;7:484 495.
19. Secci A, Wong N, Tang W, Wang S, Doherty T, Detrano R. Electron beam
computed tomographic coronary calcium as a predictor of coronary events:
comparison of two protocols. Circulation 1997;96:11221129.
20. Raggi P, Callister TQ, Cooil B, He ZX, Lippolis NJ, Russo DJ, Zelinger A,
Mahmarian JJ. Identification of patients at increased risk of first unheralded acute
myocardial infarction by electron-beam computed tomography. Circulation 2000;
101:850 855.
21. Rumberger JA, Brundage BH, Rader DJ, Kondos G. Electron beam computed
tomographic coronary calcium scanning: a review and guidelines for use in
asymptomatic persons. Mayo Clinic Proc 1999;74:243252.
22. Shaw LJ, Peterson ED, Shaw LK, Kesler KL, DeLong ER, Harrell FE Jr,
Muhlbaier LH, Mark DB. Use of a prognostic treadmill score in identifying
diagnostic coronary disease subgroups. Circulation 1998;98:16221630.

A SYMPOSIUM: FIRST INTERNATIONAL SAI MEETING

65E