NCSS Statistical Software NCSS.

com

Chapter 535

Binary Diagnostic
Tests – Single Sample
Introduction
An important task in diagnostic medicine is to measure the accuracy of a diagnostic test. This can be done by
comparing the test result with the true condition status of a number of patients. The results of such a study can be
displayed in a 2-by-2 table in which the true condition is shown as the rows and the diagnostic test result is shown
as the columns.

Diagnostic Test Result

True Condition Positive Negative Total
Present (True) T1 T0 n1
Absent (False) F1 F0 n0
Total m1 m0 N

Data such as this can be analyzed using the standard techniques for two proportions. However, specialized
techniques have been developed for dealing specifically with the questions that arise from such a study. These
techniques are presented in the book by Zhou, Obuchowski, and McClish (2002), and this is the reference that we
have used in developing this procedure.

Test Accuracy
Several measures of a diagnostic test’s accuracy are available. Probably the most popular measures are the test’s
sensitivity and the specificity. Sensitivity is the proportion of those that have the condition for which the
diagnostic test is positive. Specificity is the proportion of those that do not have the condition for which the
diagnostic test is negative. Other accuracy measures that have been proposed are the likelihood ratio and the odds
ratio.

Technical Details
Suppose you arrange the results of a diagnostic test into a 2-by-2 table as follows:

Diagnostic Test Result

True Condition Positive Negative Total
Present (True) T1 T0 n1
Absent (False) F1 F0 n0
Total m1 m0 N

535-1
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Binary Diagnostic Tests – Single Sample

Sensitivity and Specificity

The sensitivity is estimated as

 = T1
Se
n1
and the specificity is estimated as

 = F0
Sp
n0
Confidence intervals may be formed for these two statistics. Rather than use the common confidence interval for a
proportion that uses the normal approximation to the binomial, we use the more accurate score method of Wilson
(1927). This method has been shown by Agresti and Coull (1998) to have much better coverage probabilities than
either the exact method of inverting the binomial or the simple Wald confidence interval.
The confidence limits for the sensitivity based on the score method are

( )
2
2 Se  + z1−α / 2
 1 − Se
 + z1−α / 2 ± z
Se 4n1
1− α / 2
2n1 n1
z12−α / 2
1+
n1
and for specificity are

( )
2
2 Sp  + z1−α / 2
 1 − Sp
 + z1−α / 2 ± z
Sp 4n0
1− α / 2
2n0 n0
z12−α / 2
1+
n0
Likelihood Ratio
The likelihood ratio (LR) statistic may be used as a measure of accuracy of a diagnostic test. This statistic is
calculated both for positive and negative test results as follows
P(Test = Positive|Condition = Present )
LR( + ) =
P(Test = Positive|Condition = Absent )
Se
=
1 − Sp
Se
=
FPR

535-2
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Binary Diagnostic Tests – Single Sample

and
P(Test = Negative|Condition = Present )
LR( − ) =
P(Test = Negative|Condition = Absent )
1 − Se
=
Sp
FPR
=
Sp
where FPR is the false positive rate and FNR is the false negative rate.
Confidence limits for LR(+)are calculated using the skewness adjusted score method of Gart and Nam (1998).
The lower limit is the solution of

( )
2
 S (φ , ~
p2 ) µ3 z − 1
2
 − − z2 = 0
 
 V 6 
and the upper limit is the solution of

( )
2
 S (φ , ~
p2 ) µ3 z − 1
2
 + − z2 = 0
 
 V 6 
where ~
p2 is the appropriate solution of
Np2 [
~ 2 − φ ( n1 + T1) + F1 + n1 ~
p2 + m1 = 0 ]
and
T1 − n1( ~
p1 )
S (φ , ~
p2 ) = ~
φq1
−1
  q~1 q~2  
V =  φ 2  + ~  
  ( n1) p1 n0( p2 )  
~

~
p1 = φ~
p2
q~ = 1 − ~
1 p 1

q~2 = 1 − ~
p2

 q~ (q~ − ~ p2 ) 
p1 ) q~2 (q~2 − ~
µ~3 = v 3/ 2  1 1
−
 ( n1( ~p1 ))
2
(n0( ~p2 )) 
2

−1
 q~1 q~2 
v= ~ + ~ 
 n1( p1 ) n0( p2 ) 

535-3
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Binary Diagnostic Tests – Single Sample

Using the substitution

φ = LR( + )
Se
=
1 − Sp
p
= 1
p2
The formulas for LR(-) are similar. They are based on the substitution
φ = LR( − )
1 − Se
=
Sp
p
= 1
p2

Odds Ratio
Another measure of accuracy is the odds ratio which is

 Se 
 
 1 − Se 
o=
 1 − Sp 
 
 Sp 
Formulas for computing confidence limits of the odds ratio are given in the chapter on Two Proportions and they
will not be repeated here.

Data Structure
This procedure does not use data from the database. Instead, you enter the values directly into the panel. The data
are entered in the familiar 2-by-2 table format.

Procedure Options
This section describes the options available in this procedure.

Data Tab
Enter the data values directly on this panel.

Data Values
T1
This is the number of patients that had the condition of interest and responded positively to the diagnostic test.
T0
This is the number of patients that had the condition of interest but responded negatively to the diagnostic test.

535-4
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Binary Diagnostic Tests – Single Sample

F1
This is the number of patients that did not have the condition of interest but responded positively to the diagnostic
test.
F0
This is the number of patients that did not have the condition of interest and responded negatively to the
diagnostic test.

Report Options
Alpha - Confidence Limits
The confidence coefficient to use for the confidence limits of the difference in proportions. 100 x (1 - alpha)%
confidence limits will be calculated. This must be a value between 0 and 0.5.
Decimal - Proportions
The number of digits to the right of the decimal place to display when showing proportions on the reports.

Example 1 – Binary Diagnostic Test of a Single Sample

This section presents an example of how to run an analysis on hypothetical data. In this example, samples of 50
individuals known to have a certain disease and 50 individuals without the disease where selected at random. All
100 individuals were given a diagnostic test. Of those with the disease, 42 tested positively and 8 tested
negatively for it on the diagnostic test. Of those without the disease, 14 tested positively and 36 tested negatively
for it on the diagnostic test.
You may follow along here by making the appropriate entries or load the completed template Example 1 by
clicking on Open Example Template from the File menu of the Binary Diagnostic Tests – Single Sample window.

1 Open the Binary Diagnostic Tests – Single Sample window.

• On the menus, select Analysis, then Proportions, then Binary Diagnostic Tests - Single Sample. The
Binary Diagnostic Tests – Single Sample procedure will be displayed.
• On the menus, select File, then New Template. This will fill the procedure with the default template.

2 Specify the data.

• On the Binary Diagnostic Tests – Single Sample window, select the Data tab.
• In the T1 box, enter 42.
• In the T0 box, enter 8.
• In the F1 box, enter 14.
• In the F0 box, enter 36.

3 Run the procedure.

• From the Run menu, select Run Procedure. Alternatively, just click the green Run button.

535-5
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Binary Diagnostic Tests – Single Sample

Table and Data Sections

Counts Table Proportions

True Diagnostic Test Result Diagnostic Test Result

Condition Positive Negative Total Positive Negative Total
Present 42 8 50 0.4200 0.0800 0.5000
Absent 14 36 50 0.1400 0.3600 0.5000
Total 56 44 100 0.5600 0.4400 1.0000

Row Proportions Column Proportions

True Diagnostic Test Result Diagnostic Test Result

Condition Positive Negative Total Positive Negative Total
Present 0.8400 0.1600 1.0000 0.7500 0.1818 0.5000
Absent 0.2800 0.7200 1.0000 0.2500 0.8182 0.5000
Total 0.5600 0.4400 1.0000 1.0000 1.0000 1.0000

These reports display the data table that was input along with various proportions that make interpreting the table
easier. Note that the sensitivity and specificity are displayed in the Row Proportions table.

Sensitivity and Specificity Section

Lower 95.0% Upper 95.0% Number Number
Measure Value Conf. Limit Conf. Limit Matches Different Total
Sensitivity 0.8400 0.7149 0.9166 42 8 50
Specificity 0.7200 0.5833 0.8253 36 14 50

This report displays the sensitivity and specificity with their corresponding confidence limits. Note that for a
perfect diagnostic test, both values would be one. Hence, the higher the values the better.

Likelihood Ratio Section

Lower 95.0% Upper 95.0%
Measure Value Conf. Limit Conf. Limit
LR(Test=Posivitive) 3.0000 1.9753 5.0086
LR(Test=Negative) 0.2222 0.1062 0.4026

This report displays LR(+) and LR(-) with their corresponding confidence limits. You would want LR(+) > 1 and
LR(-) < 1, so you should place close attention that the lower limit of LR(+) is greater than one and that the upper
limit of LR(-) is less than one.
Note the LR(+) means LR(Test=Positive). Similarly, LR(-) means LR(Test=Negative).

Odds Ratio Section

Lower 95.0% Upper 95.0%
Measure Value Conf. Limit Conf. Limit
Odds Ratio (+ 1/2) 12.5862 4.8421 32.7155
Odds Ratio (Fleiss) 12.5862 4.6339 40.8823

This report displays estimates of the odds ratio as well as confidence limits for the odds ratio. Because of the
better coverage probabilities of the Fleiss confidence interval, we suggest that you use the second line of the
report.

535-6
© NCSS, LLC. All Rights Reserved.