Prognosis - Overview

Competence and Credible, Excel in Organization, Excel in Operation, Beyond Expectations, Management by Objective
CRITICAL APPRAISAL OF
PROGNOSTIC STUDY
Kuntjoro Harimurti
Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

RSUPN Dr. Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia

Introduction - Prognosis
 Prognosis: the prediction of the

future course of events following
the onset of disease.
 can include death, complications,
remission/recurrence, morbidity,
disability and social or occupational
function.
 Patient’s, doctor’s, insurance’s
concern
Introduction - Prognosis
 Possible outcomes of a disease and the frequency

with which they can be expected to occur.
 Natural history: the evolution of disease without medical
intervention.
 Clinical course: the evolution of disease in response to
medical intervention.

Prognostic vs. Risk Factors

 Prognostic factors: factors associated with a particular
outcome among disease subjects. Can predict good or
bad outcome
 Need not necessarily cause the outcome, just be
associated with them strongly enough to predict their
development
 examples includes age, co-morbidities, tumor size, severity of
disease etc.
 often different from disease risk factors e.g., BMI and pre-
menopausal breast CA.

Prognostic vs. Risk Factors
 Risk factors
 distinct from prognostic factors,
 include lifestyle behaviors and environmental exposures

that are associated with the development of a target
disorder.
 Ex: smoking = important risk factor for developing lung
cancer, but tumor stage is the most important prognostic
factor in individuals who have lung cancer.

Prognosis Suffer target

outcome
Patients at risk Prognostic

Risk factors of target event Time
factor
Do not suffer
target outcome

Types of reports on prognosis
 Best evidence: systematic review of prognosis

studies
 Most available: Cohort studies
 Investigators follow individuals with disease overtime
and monitor for occurrence of the outcome
 Other source:
 Control group of RCTs
 Case-control studies

Critical Appraisal of Prognostic Study

Critical appraisal of study
 Validity Assessing risk of bias
Magnitude of result and

 Importance
its precision
External validity /
 Applicability
generalizability

Bias in Epidemiology Study: GATE Approach

Bias in Epidemiology Study: GATE Approach

The PECOT RAMBOMAN
P
Population/ Recruitment Recruitment Bias
Participants
Allocation Allocation Bias
Exposure and Maintenance Maintenance Bias

Comparison E C
Blind
Outcomes Measurement Bias
Objective
yes
O Measurements
Time no
T ANalyses Confounding Bias
Bias in A Study of Prognosis

Selection/ Allocation Maintained
Recruitment Bias Bias
Bias
Appropriate- Appropriate
definition of Long and
ness of
prognostic complete
participants’
factors follow-up
selection
Measurement/ Confounding
Observation Bias
Bias
Multivariate
Blinding and and/or
objective subgroup
measurement analyses

A. ARE THE RESULTS OF THIS PROGNOSIS

STUDY VALID?
 Was a defined, representative sample of patients
assembled at a common (usually early) point in the
course of their disease?
 Was the follow-up of the study patients sufficiently
long and complete?
 Were objective outcome criteria applied in a blind
fashion?
 If subgroups with different prognoses are identified,
was there adjustment for important prognostic factors
and validation in an independent “test set” patients?
A.1. Was a defined, representative sample of

patients assembled at a common (usually early)
point in the course of their disease?
 How well defined the individuals in the study –

criteria - representative of the underlying
population.
 inclusion, exclusion
 sampling method
 similar, well-defined point in the course of their

disease

a) No inception cohort b) Inception cohort

5 cases identifed at different points in time 5 cases identifed at the same point in time
0 1 2 3 4 5 0 1 2 3 4 5
Years Years
Start cohort Pre-clinical phase Start cohort Pre-clinical phase
Clinical phase Clinical phase
Av. duration of survival= 14.5/5 = 2.9 yrs Av. duration of survival= 11/5 = 2.2 yrs

A.2. Was follow-up sufficiently long and complete?
 Ideal follow-up period: until every patient recovers

or has one of the other outcomes of interest,
 Short follow up time  too few study patients with
outcome of interest  little information of use to
patient
 Loss to follow up  influence the estimate of the
risk of the outcome  validity?.
 “5 and 20” rule
 Best and worst case scenario (sensitivity analysis)
A.3. Were objective outcome criteria applied in a

blind fashion?
 Investigators making judgments about clinical

outcomes are kept “blind” to subjects’ clinical
characteristics and prognostic factors.
 Minimize measurement bias!

Measurement bias
 Measurement bias can be minimized by:

 ensuring observers are blinded to the exposure
status of the patients.
 using careful criteria (definitions) for all outcome
events.
 apply equally rigorous efforts to ascertain all
events in both exposure groups.

A.4. If subgroups with different prognoses are identified,

was there adjustment for important prognostic factors
and validation in an independent “test set” patients?
 Adjustment  ensure that predictions are not being
distorted by unequal occurrence of another pognostic
factors
 Exp: Different prognosis for developing CAD in
hypertensive women and men
 Other prognostic factors?  smoking, BMI, HRT
 Prognosis and prognostics factor identified in one
subgroups of patients with disease might be different
in another subgroups
 Need to validate in another group
B. Are the results of this study important?
 How likely are the outcomes over time?
 How precise is this prognostic estimate?

B.1. How likely are the outcomes over time?
 % of outcome of interest at a particular point in

time (1 or 5 year survival rates)
 Median time to the outcome (e.g. the length of
follow-up by which 50% of patients have died)
 Event curves (e.g. survival curves) that illustrate, at
each point in time, the proportion of the original
study sample who have not yet had a specified
outcome.
Survival Rate
 1 year survival
A. Good (or too short)
B. 20%
C. 20%
D. 20%
 Median survival
A. ?
B. 3 months
C. 9 months
D. 7.5 months

B.2 How precise is this prognostic estimate?
 Precision  95% confidence interval

 The narrower the confidence interval, the more precise is
the estimate.
 If survival over time is the outcome of interest 
early follow-up periods results in more precision
(more patients than in later periods)

C. Can we apply this valid, important

evidence about prognosis to our patients?
 Is our patient so different from those in the study
that its results cannot apply?
 Will this evidence make a clinically important
impact on our conclusions about what to offer or
tell our patient?

C.1. Is our patient so different from those in the

study that its results cannot apply?
 How well do the study results generalize to the
patients in your practice?
 Compare patients' important clinical characteristics
 Read the definitions thoroughly
 The closer the match between the patient before you and
those in the study, the more confident you can be in applying
the study results to that patient.
 For most differences, the answer to this question is
“no”,  we can use the study results to inform our
prognostic conclusions.

C.2 Will this evidence make a clinically important

tell our patient?
 Useful for
 Initiating or not therapy,
 monitoring therapy that has been initiated,
 deciding which diagnostic tests to order.
 providing patients and families with the information they

want about what the future is likely to hold for them and
their illness.

C.2 Will this evidence make a clinically important

tell our patient?
 Communicating to patients their likely fate
 Guiding treatment decisions
 Comparing outcomes to make inferences about
quality of care

Conclusion
 Prognosis study beneficial

 Communicating to patients their likely fate
 Guiding treatment decisions
 Comparing outcomes to make inferences about quality

of care


Exercise Critical Appraisals of A Prognostic

Study

Answerable question (PICO)

 P – Patients hospitalized due to CAP
 I–
 C–
 O – Long-term (5-year) mortality
 P – Patients hospitalized due to CAP

 I – Several prognostic factors (age, cardiovascular disease,
immunocompromization, low serum albumin level on admission, active
smoking, microbial etiology)
 C – No risk factors
 O – Long-term (5-year) mortality

Are the results of this prognosis study valid?
 Was a defined, representative sample of patients

assembled at a common (usually early) point in the
course of their disease?
 Yes. Patients were followed from the date of

hospital discharge until the closing date of Decem-
ber 31st 2014. (Methods, p. 3/16)

 2. Was patient follow-up sufficiently long and

complete?
 Yes. Median follow- up of 1804 days (range 1–2520

days) and only 1 patient was lost to follow-up
(censored) at day 1. (Results, p. 6/16)

 3. Were objective outcome criteria applied in a

“blind” fashion?
 Outcome of CAP was 5-year mortality. Mortality

was determined utilizing data from the Cause of
Death Registry kept by the Norwegian Institute of
Public Health. (Methods, p. 5/16)

 4. If subgroups with different prognoses are

identified, was there adjustment for important
prognostic factors?
 Yes. When evaluating independent predictors for 5-

year survival, author used Cox proportional hazard
models  multivariate regression analysis of
survival data. (Methods, p. 5/16).
 5. Was there validation in an independent group

(“test-set”) of patients?
 Unclear. But there is information regarding

comparison mortality due to CAP in the study
cohort with the expected total mortality of
Norwegian population  standardized mortality
ratio (SMR) > 1.00. (Methods, p. 6/16)
Are the (valid) results of this prognosis study

important?
 How likely are the outcomes over time?
 Mortality occurred in 79 (30.5%) of the 259 hospital

survivors of CAP during a median follow- up of
1804 days (range 1–2520 days). (Results, p. 6/16)

Are the valid results of this prognosis study

important?


30-day survival rates 98.8% (95% CI 97.4–

100.0%)
1-year survival rates 91.1% (95% CI 87.6–
94.6%),
3-year survival rates 82.2% (95% CI 77.5–
86.9%) Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
5-year survival rates
RSUPN72.9% (95% –CI
Dr. Cipto Mangunkusumo 67.4–
Fakultas Kedokteran Universitas Indonesia

Can you apply this valid, important evidence

about prognosis in caring for your patient?
 Were the study patients similar to your own?
 Or: Are the study patients so different from ours

that we should not use the results at all in making
predictions for our patients?


Can you apply this valid, important evidence

about prognosis in caring for your patient?
 2. Will this evidence make a clinically important
impact on your conclusions about what to offer or
tell your patient?


Prognosis - Overview

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Prognosis - Overview

Caricato da

Copyright:

Formati disponibili

Competence and Credible, Excel in Organization, Excel in Operation, Beyond Expectations, Management by Objective

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

 Prognosis: the prediction of the

 Possible outcomes of a disease and the frequency

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Prognostic vs. Risk Factors

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Prognostic vs. Risk Factors

 include lifestyle behaviors and environmental exposures

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Prognosis Suffer target

Patients at risk Prognostic

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Types of reports on prognosis

 Best evidence: systematic review of prognosis

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Critical Appraisal of Prognostic Study

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Critical appraisal of study

 Validity Assessing risk of bias

Magnitude of result and

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Bias in Epidemiology Study: GATE Approach

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Bias in Epidemiology Study: GATE Approach

Exposure and Maintenance Maintenance Bias

Bias in A Study of Prognosis

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

A. ARE THE RESULTS OF THIS PROGNOSIS

A.1. Was a defined, representative sample of

 How well defined the individuals in the study –

 similar, well-defined point in the course of their

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

a) No inception cohort b) Inception cohort

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

A.2. Was follow-up sufficiently long and complete?

 Ideal follow-up period: until every patient recovers

A.3. Were objective outcome criteria applied in a

 Investigators making judgments about clinical

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

 Measurement bias can be minimized by:

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

A.4. If subgroups with different prognoses are identified,

B. Are the results of this study important?

 How likely are the outcomes over time?

 How precise is this prognostic estimate?

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

B.1. How likely are the outcomes over time?

 % of outcome of interest at a particular point in

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

B.2 How precise is this prognostic estimate?

 Precision  95% confidence interval

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

C. Can we apply this valid, important

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

C.1. Is our patient so different from those in the

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

C.2 Will this evidence make a clinically important

 monitoring therapy that has been initiated,

 deciding which diagnostic tests to order.

 providing patients and families with the information they

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit