CONFIDENCE INTERVAL ESTIMATION

I. Definitions

1. An interval estimate of a parameter is any pair of functions L ( x ) , U ( x ) of the

sample which satisfy L ( x ) U ( x ) x . If the realization x=X is observed and the
inference L ( x ) U ( x ) is made, then the random interval L ( x ) , U ( x ) is the
interval estimator.

Why is it called a random interval?

Is it open/closed/two sided?

2. The coverage probability is the probability that the random interval L ( x ) , U ( x )

{ }
covers the true parameter (ie P L ( x ) ,U ( x ) )

3. The confidence coefficient of L ( x ) , U ( x ) is the infimum of the coverage

probabilities.

Note: interval estimates + confidence coefficient = confidence intervals

Note: The concepts can be generalized to sets (need not be an interval)

Example 9.1.6

Set up to illustrate the concepts does not tell you how to obtain the interval.

X U ( 0, ) . Want an interval estimator for . Propose two intervals:

[ aY , bY ] ,1 a < b [Y + c, Y + d ] , 0 c < d where Y = X ( n )

Compute the coverage probabilities:

1 Y 1
P { [ aY , bY ]} = P {aY bY } = P
b a
1 1
= P T
b a
Now, use basic probability and the distribution of Y to develop this expression:
 n n
1 1
= = confidence coefficient (not a function of the parameter)
a b

n n
c d
Do the same for the other proposed interval: = 1 1 (a function of

the parameter). So, must construct the infimum, which occurs as . So the
confidence coefficient of this estimator is 0!

II. Methods for Computing Interval Estimators

1. Inverting a Test Statistic

- Builds directly from hypothesis testing.

- Hypothesis tests with good properties typically result in confidence sets/intervals

with good properties

- Essentially involves setting up a hypothesis test which determines a critical

region (and thereby implies an acceptance region) for a specified level (or size) of test
and then solving for the range on the parameter associated with the corresponding
interval.
- There is thus a dual relationship between the corresponding acceptance region
and the confidence interval. The hypothesis test fixes parameter and determines values of
the statistic that support this parameter value. The confidence interval fixes the sample
values and asks what values of the parameter are consistent with these sample values.
- See Figure 9.2.1 for an example of a test of the mean for normally distributed
variables.

Theorem 9.2.2 For each o , let A ( o ) be the acceptance region of a level test of
H o : o . For each x , define a set C ( x ) in the parameter space by
C ( x ) = { o : x A ( o )} . Then the random set C ( X ) is a 1 confidence set.
Conversely, let C ( X ) be a 1 confidence set. Then for any o , define
A ( o ) = { x : o C ( x )} . Then, A ( o ) is an acceptance region of a level test of
H o : o .

Proof: Straightforward utilization of the probability of a sample being in an

acceptance/rejection region. The alternative hypothesis determines the appropriate form
of this acceptance region (as in hypothesis testing)
 However, the actual construction of the CI may not be easy. The procedure that
is often used involves the LRT test. However, other tests can also be inverted.

Example 1: Compute the confidence interval for the parameter in an exponential

distribution by inverting an LRT test:

H o : = o
H1 : o

a) Set up the LRT test statistic, just as you would for a hypothesis test. After
simplifying,

xi xi
n

( x) = exp n
n o
o

b) So, the acceptance region is:

x
i xi
n
*
A ( o ) = x : exp k (See Fig 9.2.2)

o o

c) So, the confidence set is:

x
i xi
n
*
C ( x ) = : exp k (See Fig 9.2.2)

One is a function of x i and the other is a function of .

d) The confidence interval is then simplified to the form:

{
C ( x ) = { : L ( x ) U ( x )} = : L ( xi ) U ( xi ) }
So, substituting and noting that you are looking for the values where:

xi xi = xi exp xi
n n

exp
L ( xi ) L ( xi ) U ( xi ) U ( xi )

a n exp ( a ) = b n exp ( b )
The actual solutions are often computed numerically. See text for solution to this
example. You need to use distributions of the statistic here it is simple
( X i ~ Gamma ( n, ) ), but often not.

1
C ( x ) = { : L ( x ) U ( x )} = : xi xi
1
a b

where

1 1
P xi xi = P b
x i
a = 1
a b

Example 2: Determine a 1-sided confidence interval of the form

C ( x ) = { p : L ( x ) < p 1} for p in a Bernoulli distribution. The associated hypothesis

test:

H o : p = po
(alternative hypothesis implies p is small under the null hypothesis)
H1 : p > po

So, the confidence interval of the form C ( x ) = { p : L ( x ) < p 1} . Now, we know that
the resulting UMP hypothesis test is MLR and involves T = X i ~ Bin ( n, p ) .
Rejection region is: R = { x : T > k ( po )} , where k is selected to satisfy the size (level) of
the test.

k ( po ) k ( po ) 1
n y n y
p (1 p ) 1 and p (1 p ) < 1
n y n y

y =0 y y =0 y