Clinical Epidemiology

Interpreting Medical Tests and

Other Evidence

Interpreting Medical Tests and Other Evidence

Dichotomous model
Developmental characteristics
Test parameters
Cut-points and Receiver Operating Characteristic
(ROC)
Clinical Interpretation
Predictive values: keys to clinical practice
Bayes Theorem and likelihood ratios
Pre- and post-test probabilities and odds of disease
Test interpretation in context
True vs. test prevalence
Combination tests: serial and parallel testing
Disease Screening
Why everything is a test!
2

Dichotomous model

Simplification of Scale

Test usually results in continuous or complex

measurement

Often summarized by simpler scale -reductionist, e.g.

ordinal

grading, e.g. cancer staging

dichotomization

-- yes or no, go or stop

Dichotomous model
Disease
Yes (D+) No (D-) Total
Positive (T+)
a
b
a+b
Test

Negative (T-)
Total

c+d

a+c

b+d

Test Errors from Dichotomization

Types of errors
False Positives = positive tests that are wrong = b
False Negatives = negative tests that are wrong = c

Developmental characteristics: test parameters

Positive (T+)
Test

Negative (T-)
Total

Disease
Yes (D+) No (D-) Total
a
b
a+b
c

c+d

a+c

b+d

Error rates as conditional probabilities

Pr(T+|D-) = False Positive Rate (FP rate) =
b/(b+d)
Pr(T-|D+) = False Negative Rate (FN rate) =
c/(a+c)
5

Developmental characteristics: test parameters

Disease
Yes (D+) No (D-) Total
Positive (T+)
a
b
a+b
Test

Negative (T-)
Total

c+d

a+c

b+d

Complements of error rates as desirable test properties

Sensitivity = Pr(T+|D+) = 1 - FN rate = a/(a+c)
Sensitivity is PID (Positive In Disease) [pelvic
inflammatory disease]
Specificity = Pr(T-|D-) = 1 - FP rate = d/(b+d)
Specificity is NIH (Negative In Health) [national
institutes of health]
6

Typical setting for finding

Sensitivity and Specificity
Best if everyone who gets the new test
also gets gold standard
Doesnt happen
Even reverse doesnt happen
Not even a sample of each (case-control
type)
Case series of patients who had both tests

Setting for finding Sensitivity

and Specificity
Sensitivity should not be tested in sickest
of sick
Should include spectrum of disease
Specificity should not be tested in
healthiest of healthy
Should include similar conditions.

Developmental characteristics: Cut-points and

Receiver Operating Characteristic (ROC)

Healthy
9

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)

Healthy

Sick
10

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)

Fals pos= 20% True pos=82%

11

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)

Fals pos= 9% True pos=70%

12

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)

F pos= 100% T pos=100%

13

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)

F pos= 50% T pos=90%

14

15

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)
Receiver Operating Characteristic (ROC)

16

Developmental characteristics: Cut-points and Receiver

Operating Characteristic (ROC)
Receiver Operating Characteristic (ROC)

17

Receiver Operating Characteristic (ROC)

ROC Curve allows comparison of different

tests for the same condition without
(before) specifying a cut-off point.
The test with the largest AUC (Area under
the curve) is the best.

18

19

Developmental characteristics: test parameters

Problems in Assessing Test Parameters

Lack of objective "gold standard" for testing, because

unavailable,
too

except e.g. at autopsy

expensive, invasive, risky or unpleasant

Paucity of information on tests in healthy

too

expense, invasive, unpleasant, risky, and

possibly unethical for use in healthy

Since

test negatives are usually not pursued with

more extensive work-ups, lack of information on
false negatives
20

Clinical Interpretation: Predictive Values

Most test positives below are sick. But this is because
there are as many sick as healthy people overall. What
if fewer people were sick, relative to the healthy?

21

Clinical Interpretation: Predictive Values

Now most test positives below are healthy. This is because
the number of false positives from the larger healthy group
outweighs the true positives from the sick group. Thus, the
chance that a test positive is sick depends on the
prevalence of the disease in the group tested!

22

Clinical Interpretation: Predictive Values

But
the prevalence of the disease in the group tested
depends on whom you choose to test
the chance that a test positive is sick, as well as the
chance that a test negative is healthy, are what a
physician needs to know.
These are not sensitivity and specificity!
The numbers a physician needs to know are the
predictive values of the test.

23

Clinical Interpretation: Predictive Values

Sensitivity (Se)
Pr{T+|D+}
true positives
total with the disease

Positive Predictive Value (PV+, PPV)

Pr{D+|T+}
true positives
total positive on the test

24

Positive Predictive Value

Predictive value positive

The predictive value of a positive test.
If I have a positive test, does that mean I have
the disease?
Then, what does it mean?
If I have a positive test what is the chance
(probability) that I have the disease?
Probability of having the disease after you
have a positive test (posttest probability)
(Watch for OF. It usually precedes the
denominator
Numerator is always PART of the denominator)
25

Clinical Interpretation: Predictive Values

D+
T+

T+
and
D+

26

Clinical Interpretation: Predictive Value

Specificity (Sp)
Pr{T-|D-}
true negatives
total without the disease

Negative Predictive Value (PV-, NPV)

Pr{D-|T-}
true negatives
total negative on the test

27

Negative Predictive Value

Predictive value negative
If I have a negative test, does that mean I
dont have the disease?
What does it mean?
If I have a negative test what is the chance
I dont have the disease?
The predictive value of a negative test.

28

Mathematicians dont Like PV

PV- probability of no disease given a negative

test result
They prefer (1-PV-) probability of disease given
a negative test result
Also referred to as post-test probability (of a
negative test)
Ex: PV- = 0.95 post-test probability for a
negative test result = 0.05
Ex: PV+ = 0.90 post-test probability for a
positive test result = 0.90
29

Mathematicians dont Like

Specificity either

They prefer false positive rate, which is 1

specificity.

30

Where do you find PPV?

Table?

NO
Make new table
Switch to odds

31

Use This Table ? NO

Test
Result
+
Total

Disease
+
95
5
100

Total

8
92
100

103
97
200

You would conclude that PPV is 95/103 = 92%

32

Make a New Table

Test
Result
+
Total

Disease
+
95
5
100

72
828
900

Total
167
833
1000

33

Make a New Table

Disease
Test
Result
+
Total

Total

95
5
100

72
828
900

167
833
1000

Probability of having the disease before testing was

10%. (pretest probability prevalence)
Posttest probability (PPV) = 95/167 = 57%
So we went up from 10% probability to 57% after
having a positive test
34

Switch to Odds

1000 patients. 100 have disease. 900

healthy. Who will test positive?

Diseased 100 X .95 = 95

Healthy
900 X .08 = 72
We will end with 95+72= 167 positive tests
of which 95 will have the disease
PPV = 95/167

35

From pretest to posttest odds

Diseased 100
X.95 = 95
Healthy 900 X.08 = 72
100 = Pretest odds
900
.95 = Sensitivity = prob. Of positive test in dis
.08
1-Specificity prob. Of positive test in hlth
95 =Posttest odds. Probability is 95/(95+72)
72

36

Remember to switch back to probability

37

What is this second fraction?

Likelihood Ratio Positive
Multiplied by any patients pretest odds
gives you their posttest odds.
Comparing LR+ of different tests is
comparing their ability to rule in a
diagnosis.
As specificity increases LR+ increases
and PPV increases (Sp P In)

38

Clinical Interpretation: likelihood ratios

Likelihood ratio =
Pr{test result|disease present}
Pr{test result|disease absent}

LR+ = Pr{T+|D+}/Pr{T+|D-} = Sensitivity/(1-Specificity)

LR- = Pr{T-|D+}/Pr{T-|D-} = (1-Sensitivity)/Specificity

39

Clinical Interpretation: Positive Likelihood Ratio and PV+

O = PRE-TEST ODDS OF DISEASE
POST-ODDS (+) = O x LR+ =

SENSITIVITY
Ox

1 - SPECIFICITY

POST- ODDS(+)
PV+ = PPV =
1 + POST- ODDS(+)
40

Likelihood Ratio Negative

Diseased 100_ X.05 =_5__

Healthy 900 X.92 = 828
100 = Pretest odds
900
.05 = 1-sensitivity = prob. Of neg test in dis
.92
Specificity
prob. Of neg test in hlth
(LR-)
Posttest odds= 5/828. Probability=5/833=0.6%
As sensitivity increases LR- decreases and NPV
increases (Sn N Out)
41

Clinical Interpretation: Negative Likelihood Ratio and PVPOST-ODDS (-) = O x LR- =

1 - SENSITIVITY
Ox

SPECIFICIT
Y

42

Remember to switch to probability and

also to use 1 minus

43

Post test probability given a

negative test
= Post odds (-)/ 1- post odds (-)
POST- ODDS(-)
PV- = NPV = 1 1 + POST- ODDS(-)
44

Value of a diagnostic test depends

on the prior probability of disease

Prevalence
(Probability) = 5%
Sensitivity = 90%
Specificity = 85%
PV+ = 24%
PV- = 99%
Test not as useful
when disease unlikely

Prevalence
(Probability) = 90%
Sensitivity = 90%
Specificity = 85%
PV+ = 98%
PV- = 49%
Test not as useful
when disease likely
45

Clinical interpretation of posttest probability

Probability of disease:
Don't
treat for
disease

Do further
diagnostic
testing

Treat for
disease

1
Testing
threshold

Disease
ruled out

Treatment
threshold

Disease
ruled in
46

Advantages of LRs

The higher or lower the LR, the higher or lower

the post-test disease probability
Which test will result in the highest post-test
probability in a given patient?
The test with the largest LR+
Which test will result in the lowest post-test
probability in a given patient?
The test with the smallest LR47

Advantages of LRs

Clear separation of test characteristics

from disease probability.

48

Likelihood Ratios - Advantage

Provide a measure of a tests ability to rule
in or rule out disease independent of
disease probability
Test A LR+ > Test B LR+

Test

A PV+ > Test B PV+ always!

Test A LR- < Test B LR Test

A PV- > Test B PV- always!

49

Using Likelihood Ratios to Determine PostTest Disease Probability

Pre-test
probability
of disease

Pre-test
odds of
disease
Likelihood
ratio

Post-test
odds of
disease

Post-test
probability
of disease

50

51

Predictive Values
Alternate formulations:Bayes Theorem
PV+ =
Se Pre-test Prevalence
Se Pre-test Prevalence + (1 - Sp) (1 - Pre-test Prevalence)
High specificity to rule-in disease
PV- =
Sp (1 - Pre-test Prevalence)
Sp (1 - Pre-test Prevalence) + (1 - Se) Pre-test Prevalence
High sensitivity to rule-out disease

52

Clinical Interpretation: Predictive Values

PV+ And PV-1 Of Electrocardiographic Status2
For Angiographically Verified3 Coronary Artery
Disease, By Age And Sex Of Patient
Sex
Age
PV+ (%)
PV- (%)
F
F
F

<40
40-50
50+

32
46
62

88
80
68

M
M
M

<40
40-50
50+

62
75
85

68
54
38

1. Based on statistical smoothing of results from 78 patients referred to NC

Memorial Hospital for chest pain. Each value has a standard error of 6-7%.
2. At least one millivolt horizontal st segment depression.
3. At least 50% stenosis in one or more main coronary vessels.
53

Clinical Interpretation: Predictive Values

54

If Predictive value is more

useful why not reported?
Should they report it?
Only if everyone is tested.
And even then.
You need sensitivity and specificity from
literature. Add YOUR OWN pretest
probability.

55

So how do you figure pretest

probability?

Start with disease prevalence.

Refine to local population.
Refine to population you serve.
Refine according to patients presentation.
Add in results of history and exam (clinical
suspicion).
Also consider your own threshold for testing.

56

Why everything is a test

Once a tentative dx is formed, each piece of new

information -- symptom, sign, or test result -should provide information to rule it in or out.
Before the new information is acquired, the
physicians rational synthesis of all available
information may be embodied in an estimate of
pre-test prevalence.
Rationally, the new information should update that
estimate to a post-test prevalence, in the manner
described above for a diagnostic test.
In practice it is rare to proceed from precise
numerical estimates. Nevertheless, implicit
understanding of this logic makes clinical practice
more rational and effective.
57

Pretest Probability: Clinical

Significance
Expected test result means more than
unexpected.
Same clinical findings have different
meaning in different settings
(e.g.scheduled versus unscheduled visit).
Heart sound, tender area.
Neurosurgeon.
Lupus nephritis.

58

What proportion of all patients

will test positive?
Diseased X sensitivity
+ Healthy X (1-specificity)
Prevalence X sensitivity +
(1-prevalence)(1-specificity)
We call this test prevalence
i.e. prevalence according to the test.

59

SENS = SPEC = 95%

What if test prevalence is 5%?
What if it is 95%?

60

61

Combination tests: serial and parallel testing

Combinations of specificity and sensitivity superior to the
use of any single test may sometimes be achieved by
strategic uses of multiple tests. There are two usual
ways of doing this.
Serial

testing: Use >1 test in sequence, stopping at the

first negative test. Diagnosis requires all tests to be
positive.

Parallel

testing: Use >1 test simultaneously, diagnosing

if any test is positive.

62

Combination tests: serial testing

Doing the tests sequentially, instead of together with

the same decision rule, is a cost saving measure.

This strategy

increases specificity above that of any of the individual tests,

but
degrades sensitivity below that of any of them singly.

However, the sensitivity of the serial combination may

still be higher than would be achievable if the cutpoint of any single test were raised to achieve the
same specificity as the serial combination.
63

Combination tests: serial testing

Demonstration: Serial Testing with Independent Tests

SeSC = sensitivity of serial combination

SpSC = specificity of serial combination

SeSC = Product of all sensitivities= Se1X Se2Xetc

Hence SeSC < all individual Se

1-SpSC = Product of all(1-Sp)

Hence SpSC > all individual Spi

Serial test to rule-in disease

64

Combination tests: parallel testing

Parallel Testing
Usual decision strategy diagnoses if any test positive.
This strategy

increases sensitivity above that of any of the individual tests,

but
degrades specificity below that of any individual test.

However, the specificity of the combination may be

higher than would be achievable if the cut-point of
any single test were lowered to achieve the same
sensitivity as the parallel combination.

65

Combination tests: parallel testing

Demonstration: Parallel Testing with Independent Tests

SePC = sensitivity of parallel combination

SpPC = specificity of parallel combination

1-SePC = Product of all(1 - Se)

Hence SePC > all individual Se

SpPC = Product of all Sp

Hence SpPC < all individual Spi

Parallel test to rule-out disease

66

Clinical settings for parallel

testing

Parallel testing is used to rule-out serious but

treatable conditions (example rule-out MI by
CPK, CPK-MB, Troponin, and EKG. Any positive
is considered positive)
When a patient has non-specific symptoms,
large list of possibilities (differential diagnosis).
None of the possibilities has a high pretest
probability. Negative test for each possibility is
enough to rule it out. Any positive test is
considered positive.
67

Because specificity is low, further testing is

now required (serial testing) to make a
diagnosis (Sp P In).

68

Clinical settings for serial testing

When treatment is hazardous (surgery,

chemotherapy) we use serial testing to
raise specificity.(Blood test followed by
more tests, followed by imaging, followed
by biopsy).

69

Calculate sensitivity and

specificity of parallel tests
(Serial tests in HIV CDC exercise)
2 tests in parallel
1st test sens = spec = 80%
2nd test sens = spec = 90%
1-Sensitivity of combination =
(1-0.8)X(1-0.9)=0.2X0.1=0.02
Sensitivity= 98%
Specificity is 0.8 X 0.9 = 0.72
70

Typical setting for finding

Sensitivity and Specificity
Best if everyone who gets the new test
also gets gold standard
Doesnt happen
Even reverse doesnt happen
Not even a sample of each (case-control
type)
Case series of patients who had both tests

71

EXAMPLE
Patients who had both a stress test and
cardiac catheterization.
So what if patients were referred for
catheterization based on the results of the
stress test?
Not a random or even representative
sample.
It is a biased sample.

72

73

If the test is used to decide

referral for gold standard?
Disease

No
Disease

Total

Test
Positive

95

72

167

Test
Negative

828

833

Total

100

900

1000

Sn95/100
=.95

Sp 828/900 = .
92
74

If the test is used to decide

referral for gold standard?
Disease

No
Disease

Total

Test
Positive

95
85

72
65

167
167150

Test
Negative

5
1

828
99

833
833 100

Total

100
86

900
164

1000

Sn85/86=.99 Sp 99/164=.4

75

If the test is used to decide

referral for gold standard?
Disease

No Disease Total

Test
Positive

85

65

150

Test
Negative

99

100

Total

86

164

250

Sn85/86=.99

Sp 99/164=.4

76