Appraising

Prognostic Study

Introduction Prognosis

Important phase of a disease

progression of a disease.
Patients, doctors, insurances concern
Prognosis: the prediction of the future
course of events following the onset of
disease.
can include death, complications,
remission/recurrence, morbidity,
disability and social or occupational
function.

Introduction Prognosis
Possible
outcomes of a disease and
the frequency with which they can
be expected to occur.
Natural history: the evolution of
disease without medical intervention.
Clinical course: the evolution of
disease in response to medical
intervention.

Natural History
Studies

Degree to which natural history can be

studied depends on the medical system
(Scandinavia) and the type of disease
(rare, high risk).
The natural history of some diseases
can be studied because:
remain unrecognized (i.e., asymptomatic) e.g.,
anemia, hypertension.
considered normal discomforts e.g., arthritis,
mild depression.

Natural History
Studies
Natural history studies permit the
development of rational strategies
for:
early detection of disease
e.g., Invasive Cervical CA.

treatment of disease
e.g. Diabetes

Prognosis
Patients at risk
of target event

Prognostic
factor

Suffer target
outcome
Time
Do not suffer
target outcome

?
?

A. ARE THE RESULTS OF THIS

PROGNOSIS STUDY VALID?
1.

2.
3.
4.

Was a defined, representative sample of patients

assembled at a common (usually early) point in the
course of their disease?
Was the follow-up of the study patients sufficiently
long and complete?
Were objective outcome criteria applied in a blind
fashion?
If subgroups with different prognoses are identified,
was there adjustment for important prognostic factors
and validation in an independent test set patients?

A.1. Was a defined, representative sample

of patients assembled at a common
(usually early) point in the course of their
How well defined the individuals in
disease?

the study criteria - representative of

the underlying population.
inclusion, exclusion
sampling method

similar, well-defined point in the

course of their disease cohort

A.2. Was follow-up sufficiently

long and complete?

Ideal follow-up period

Until EVERY patient recovers or has one of the other
outcomes of interest,
Until the elapsed time of observation is of clinical
interest to clinicians or patients.
Short follow up time too few study patients with
outcome of interest little information of use to
patient
Loss to follow up influence the estimate of the risk of
the outcome validity?.
Patients are too ill (or too well); Die; Move, etc
Most journals require at least 80% follow-up for a
prognosis study to be considered valid.
Best and worst case scenario (sensitivity analysis)

A.3. Were objective outcome

criteria applied in a blind
fashion?
Investigators making judgments
about clinical outcomes are kept
blind to subjects clinical
characteristics and prognostic
factors.
Minimize measurement bias!

Measurement
bias
Measurement bias can be minimized
by:

ensuring observers are blinded to the

exposure status of the patients.
using careful criteria (definitions) for all
outcome events.
apply equally rigorous efforts to
ascertain all events in both exposure
groups.

A.4. If subgroups with different prognoses

are identified, was there adjustment for
important prognostic factors and
validation
in an
independent
test setwith
Prognostic
factors:
factors associated
a particular outcome among disease
patients?
subjects. Can predict good or bad outcome
Need not necessarily cause the outcome, just
be associated with them strongly enough to
predict their development
examples includes age, co-morbidities, tumor
size, severity of disease etc.
often different from disease risk factors e.g., BMI
and pre-menopausal breast CA.

A.4. If subgroups with different prognoses

are identified, was there adjustment for
important prognostic factors and validation
in an independent test set patients?

Risk factors

distinct from prognostic factors,

include lifestyle behaviors and environmental
exposures that are associated with the
development of a target disorder.
Ex: smoking = important risk factor for
developing lung cancer, but tumor stage is the
most important prognostic factor in individuals
who have lung cancer.

Bias in Follow-up
Studies

A. Selection or Confounding Bias

1. Assembly or susceptibility bias: when
exposed and non-exposed groups differ
other than by the prognostic factors
under study, and the extraneous factor
affects the outcome of the study.
Examples:

differences in starting point of disease (survival

cohort)
differences in stage or extent of disease, comorbidities, prior treatment, age, gender, or race.

Bias in Follow-up
Studies
A. Selection or Confounding Bias
2.

Migration bias:

patients drop out of the study (lost-to-follow-up).

usually subjects drop out because of a valid
reason e.g., died, recovery, side effects or
disinterest.
these factors are often related to prognosis.
asses extent of bias by using a best/worst case
analysis.
patients can also cross-over from one exposure
group to another
if cross-over occurs at random = non-differential
misclassification of exposure

Bias in Follow-up
Studies
A. Selection or Confounding Bias
3. Generalizability bias
related to the selective referral of
patients to tertiary (academic)
medical centers.
highly selected patient pool have
different clinical spectrum of disease.
influences generalizability

Survival Cohorts
Survival cohort (or available patient cohort)
studies can be very biased because:
convenience sample of current patients are likely
to be at various stages in the course of their
disease.
individuals not accounted for have different
experiences from those included e.g., died soon
after trt.

Not a true inception cohort e.g.,

retrospective case series.

Survival Cohort
Bias

True Cohort
Assemble
Cohort
N=150

Observed
True
ImprovementImprovement

Measure Outcomes
Improved = 75
Not improved = 75

50%

50%

80%

50%

Survival Cohort
Assemble
patients

Begin
Follow-up
N = 50

Measure Outcomes
Improved = 40
Not improved = 10

Not
Observed
N = 100

Dropouts:
Improved = 35
Not improved = 65

II. Bias in Follow-Up

Studies
B. Measurement bias
Measurement (or assessment) bias
occurs when one group has a higher (or
lower) probability of having their outcome
measured or detected.
likely for softer outcomes
side effects, mild disabilities, subclinical disease or
the specific cause of death.

B. Are the results of this

study important?
1. How likely are the outcomes over time?
2. How precise is this prognostic estimate?

B.1. How likely are the outcomes

over time?
% of outcome of interest at a particular
point in time (1 or 5 year survival rates)
Median time to the outcome (e.g. the
length of follow-up by which 50% of
patients have died)
Event curves (e.g. survival curves) that
illustrate, at each point in time, the
proportion of the original study sample
who have not yet had a specified
outcome.

Survival Rate
1 year survival
A. Good
B. 20%
C. 20%
D. 20%
Median survival
A. ?
B. 3 months
C. 9 months
D. 7.5 months

B.2 How precise is this

prognostic estimate?
Precision 95% confidence interval
The narrower the confidence interval, the
more precise is the estimate.

If survival over time is the outcome of

interest shorter follow-up periods
results in more precision follow up
period important to be clinically
important

C. Can we apply this valid,

important evidence about
prognosis to our patients?

1. Is our patient so different from

those in the study that its results
cannot apply?
2. Will this evidence make a clinically
important impact on our
conclusions about what to offer or
tell our patient?

Is our patient so different from

those in the study that its
results
cannot
apply?
How well
do the study results generalize
to the patients in your practice?

Compare patients' important clinical

characteristics,
Read the definitions thoroughly
The closer the match between the patient
before you and those in the study, the more
confident you can be in applying the study
results to that patient.

For most differences, the answer to this

question is no, we can use the
study results to inform our prognostic
conclusions.

C.2 Will this evidence make a

clinically important impact on our
conclusions about what to offer or
Useful
tell
our for
patient?
Initiating or not therapy,
monitoring therapy that has been
initiated,
deciding which diagnostic tests to order.
providing patients and families with the
information they want about what the
future is likely to hold for them and their
illness.

C.2 Will this evidence make a

clinically important impact on our
conclusions about what to offer or
Communicating
tell
our patient? to patients their

likely fate
Guiding treatment decisions
Comparing outcomes to make
inferences about quality of care

Conclusion
Prognosis study beneficial
Communicating to patients their likely
fate
Guiding treatment decisions
Comparing outcomes to make
inferences about quality of care