Week 8 Annotated

ACTL2002/ACTL5101 Probability and Statistics: Week 8
ACTL2002/ACTL5101 Probability and Statistics

c Katja Ignatieva

School of Risk and Actuarial Studies
Australian School of Business
University of New South Wales
k.ignatieva@unsw.edu.au
Week 8
Week 2
Week 3
Week 4
Probability:
Week 6
Review
Estimation: Week 5
Week
7
Week
9
Hypothesis testing:
Week
10
Week
11
Week 12
Linear regression:
Week 2 VL
Week 3 VL
Week 4 VL
Video lectures: Week 1 VL
Week 1
Week 5 VL
First six weeks

Introduction to probability;
Moments: (non)-central moments, mean, variance (standard
deviation), skewness & kurtosis;
Special univariate (parametric) distributions (discrete &
continue);
Joint distributions;
Convergence; with applications LLN & CLT;
Estimators (MME, MLE, and Bayesian);
Evaluation of estimators;
Interval estimation.
2201/2250

Hypothesis testing
Overview
Hypothesis testing
Hypothesis testing
Overview
Exercise: testing a mean, variance unknown
Properties of hypothesis testing
Type I and Type II Errors, power of a test
Example: type I and II errors and power
p-value
Relation between Confidence intervals and hypothesis tests
Other tests
k-Sample tests
Jarque-Bera test
Other testing methods
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Hypothesis testing
Overview
This & last week

Hypothesis testing:
- Null hypothesis:
Selection of the null hypothesis;
Simple v.s. composite.
- Statistical tests;
- Rejection region;
Neyman-Pearson Lemma;
Uniformly most powerful;
Likelihood ratio test.
- Value of test;
- Accept the null or reject the null hypothesis.
Properties of hypothesis testing.

- Type I and II errors and power;
- p-value.
2202/2250
Using asymptotic distribution of LR for testing.

Hypothesis testing
Overview
Summary last week: Hypothesis tests

Testing procedure: When testing a hypothesis use the following
steps:
i. Define a statistical hypothesis.
Note that this includes a confidence level ();
ii. Define the test statistic T (using past weeks knowledge);
iii. Determine the rejection region C ? ;
iv. Calculate the value of the statistical test, given observed data
(x1 , . . . , xn );
v. Accept or reject H0 .
Note: we assume that H0 is true when testing! (see Type I
and Type II errors)
2203/2250

Hypothesis testing
Overview
Summary last week: Best critical region

Neyman-Pearson Lemma: simple null/simple alternative
test:
L(x; 0 )
< k s.t.: Pr (X C ? |H0 ) =
(x; 0 , 1 ) =
L(x; 1 )
Uniform most powerful test: simple/composite
null/composite alternative test:
i) max {Pr((X1 , . . . , Xn ) C ? |)} = ;
0
ii) for all 0 and all critical regions C of size we have:

Pr((X1 , . . . , Xn ) C ? |) Pr((X1 , . . . , Xn ) C |)
Generalized likelihood ratio test: simple/composite

null/composite alternative test:
(x; 0 , ) =
2204/2250
L(x; b0 )
<k
b
L(x; )
s.t.: Pr (X C ? |H0 ) = .

Hypothesis testing
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Hypothesis testing

Exercise: Let X1 , . . . , Xn be a random sample from a normal
distribution. Test at a level of significance :
H0 : = 0
H1 : = 1 > 0 .
v.s.
Note: the variance 2 is unknown, but s 2 is known.

Moreover, we have x > 0 .
Question: Which (NP, UMP, LRT) procedure can you use?
The likelihood is given by:
n
Y
L x1 , . . . , xn ; , 2 =
fXi (xi )
i=1

=
2205/2250
n
1X
exp
2
k=1
xk
2 !
.

Hypothesis testing

Define the parameter sets:

0 = , 2 : = 0 , 2 > 0

= , 2 : 0 , 2 > 0 .
We knowP
(week 5) that MLE:
b = x and
b2 = n1 ni=1 (xi x)2 (note: biased estimator!).

The maximum under , is given by (using ML-estimates):
n
b=x
and
1X
(xk x)2 .
b =
n
2
k=1
The maximum under 0 is given by (show this for yourself):

n
e = 0
and
e2 =
1X
(xk 0 )2 ,
n
k=1
2206/2250

Hypothesis testing
Thus, the likelihood ratio is given by:

max {L0 (, )}
L
e,
e2
0
=
<k
(x1 , . . . , xn ) =
max {L (, )}
L (b
,
b2 )

n
n
2
P
(xk 0 )
1
exp 12
n
P
2e
(xk 0 )2
k=1 n
2207/2250
1
2b
n
Pn
k=1
n
1 P
exp 2
k=1
1
n
(xk x)
n
P
(xk x)2
k=1
n/2
2 k=1 (xk 0 )2
e n/2
<k
P
n/2
n
2
n/2
2 k=1 (xk x)
e
Pn
(xk x)2
< k1
(= k 2/n ).
Pnk=1
2
(x
)
0
k=1 k

Hypothesis testing
Simplifying, we have:
=
(xk x)2
Pn
+ (x 0 ))2
Pn
2
k=1 (xk x)
= Pn
Pn
Pn
2
2
x) (x ))
k=1 (xk x) +
k=1 (x 0 ) + 2
k=1 ( (x
| k {z } | {z 0}
k=1 ((xk x)
=nxnx=0
Pn
=
Pn
x)
x) + n (x 0 )2
1
.
< k1
2 Pn
2
1 + n (x 0 )
k=1 (xk x)
n (x 0 )2
k2
n
P
(xk x)2
k=1 (xk
k=1
2208/2250
k=1 (xk
2
constant
(=
1
1 ).
k
| 1{z }
decreasing

Hypothesis testing

Consider:

x 0 2
=
k3
Pn
2
s
k=1 (xk x) /(n 1)

x 0
k4

s
(x 0 )2
* using s 2 =
Pn
2
k=1 (xk x)
n1

n1
= k2
n
p
(= k3 )
Thus we have that the best critical region is of the form:

x 0
?
?

C = (x1 , . . . , xn ) :
k ,
s
where k ? is such that:

X 0
?

Pr
k
H0 = .
S
2209/2250

Hypothesis testing
Pn
N(0 , 2 /n), thus:

,s

1 X 0
X 0
(n 1) S 2
1
=
T .
(n 1) = |{z}
2
S
n / n
n
| {z } |
t(n1)
{z
}
We have: X =
i=1 Xi /n|H0
=Z
2 (n1)
(n1)

X 0

?

Pr
k H0 =

S

Pr |T | n k ? =

Pr |T | t1/2,n1 =
Thus:
combining * and **:

t1/2,n1 =
n k?
n
Hence: C ? = (X1 , . . . , Xn ) : t1/2,n1
2210/2250
x
0
s/ n
k ? =t1/2,n1 / n.
o
t1/2,n1 .

Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Type I and Type II Errors

A Type I error is the mistake or error committed when H0 is
true but is rejected and a Type II error is the mistake or error
committed when H0 is false, but is not rejected.
The probability of a Type I error is often denoted by (also
called the level of significance) and the probability of a Type II
error is often denoted by .
We have the Type I error:
= Pr (Reject H0 |H0 is true ) ,
and the Type II error:
= Pr (Do not reject H0 |H0 is false ) .
2211/2250

Type I, Type II errors

f(x|H0)
(Type I
error) is the
blue shaded
area.
f(x|H1)
(Type II
error) is the
green
shaded area.
2212/2250
(1 )
(power) is
the purple
shaded area.

Type I error
Type I error: reject H0 when it is true, with probability :

- if H0 is simple this is the significance level of the test;
- if H0 is composite then this probability depends on the
member of H0 that is true - usually select the maximum of the
probabilities.
Question: Test H0 : 0 v.s. H1 : > 0 , what is the

Type I error?
Solution: = Pr (Reject H0 | = 0 ).
2213/2250

Type II error & power

Type II error: accept H0 when it is false; is probability of a
Type II error.
In case H1 is composite this depends on the particular
member of H1 that holds.
Power of a test is the probability that H0 is rejected when it is
false (= 1 ).
For a test of H0 against H1 , the power function of a test is the
function that gives the probability that, given that H1 is true,
the observable samples will fall within the critical region C .
The power function will be denoted by:
() = Pr((X1 , . . . , Xn ) C | H1 ).
2214/2250
The value of the power function at a specific parameter point

is called the power of the test at that point.

Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary


Suppose X1 , X2 , . . . , Xn is a random sample from N(, 1) and
that we wish to test H0 : = 0 versus H1 : 6= 0. Consider
the rejection (or critical) region as:
C = {(x1 , x2 , . . . , xn ) : |x| > 2}
The power function is equal to the probability that the sample
will fall within the critical region:
2215/2250
() = Pr ((X1 , X2 , . . . , Xn ) C | H1 )

= Pr X > 2| H1

= Pr X > 2| H1 + Pr X < 2| H1

= Pr Z > n (2 ) + Pr Z < n (2 )

= n (2 ) + n (2 ) .
P
2
* using X = ni=1 Xi /n N(, n ) (using m.g.f. technique).


We have that the level of significance is:
= Pr ((X1 , X2 , . . . , Xn ) C |H0 is true )

= Pr X > 2 | = 0

= Pr X > 2 | = 0 + Pr X < 2 | = 0

= Pr n X > 2 n + Pr n X < 2 n

= Pr Z > 2 n + Pr Z < 2 n

= 2 2 n .
* using X |H0 =
2216/2250
Pn
i=1 Xi /n
N(0, n1 ).


The probability of a Type II error when = 1 is given by:
= Pr (Do not reject H0 |H0 is false )
= 1 Pr ((X1 , X2 , . . . , Xn ) C | = 1 )

= 1 Pr X > 2 | = 1

= Pr X 2 | = 1

= Pr 2 X 2 | = 1

= Pr 3 n X 1 n n = 1
= Pr 3 n Z n

= n 1 3 n = n + 3 n 1.
The complement, 1 , gives the power of the test when = 1.
* using X |H1 N(1, 1/n).
2217/2250

Exercise power
We have X N(, 1).
Test: H0 : = 0 v.s. H1 : > 0 .
2218/2250
We know: a test at significance level would reject H0 if:

x 0
z1
z0 =
/ n
Exercise: Find the power of this test at any value .

Solution:
X 0
z1
() =Pr
/ n

X
0
z1 +

=Pr
/ n
/ n

0
=1 z1 +
.
/ n

Power function
1
When n
increases
the power of
the test is
higher, i.e.,
more likely
to reject H0
if H1 is
correct.
2219/2250
n=20
n=100

Exercise: Bernoulli random variable

Consider a reinsurer which is asked to give a quote when
taking over the financial risk for an insurer of a major flood in
the next La Nie
na.
Assume that major floods are well defined (using SOI) and if
they occur they will only occur during a La Nie
na.
La Nie
nas only occur (on average) once every 6 years.
Hence, we do not have much data, say we have a time period
of 10 La Nie
na.
The reinsurer has worldly experience with La Nie
nas and from
that he knows that the probability of a major flood during a
La Nie
na is 50%.
The reinsurer test whether this also holds for Australia, or
whether the probability is higher.
2220/2250

Exercise: Bernoulli random variable

Bernoulli random variable Y Ber(p) n = 10 trials, test:
H0 : p = 0.5
v.s.
H1 : p > 0.5,
with = 0.05. SOI> 10 in 50, 55, 71, 73, 75, 88, 10.
Question: What is the test statistic? Solution: Number of
successes X Bin(n, p).
Question: What is the rejection region? Solution: large
values of X , i.e., C = {8, 9, 10}, we then have
Pr (reject H0 when true) = = 0.0547, C ? = {9, 10}:
Pr (X C ? |p = 0.5) = = 0.0107.
2221/2250
Question: What is the power of the test? Solution: it

depends on the true value of p, because
Pr (X = 9, 10|p > 0.5) depends true value of p.

p-value
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

p-value
p-value
When the rejection region is of the form T > t (or T < t)
then t is called the critical value of the test (independent of
data).
The p-value is the smallest value of for which the null
hypothesis, given the data, will be rejected.
Note: p-value and critical value are both obtained using the
test statistic. However, critical value is computed given ,
whereas p-value is calculated by determining the probability
that the sample occurs given the null hypothesis (i.e., the
inverse).
- In case that the p-value is smaller than the level of significance
reject null.
- In case that the p-value is larger than the level of significance
accept null.
2222/2250

p-value
Example: p-value
(Continued from slide 2215) Suppose X1 , X2 , . . . , Xn is a
random sample from N (, 1) and that we wish to test
H0 : = 0 versus H1 : 6= 0 with constant as level of
significance.
Suppose the observed sample mean X turned out to be 3, we
can calculate the p-value:

p-value = Pr X 3 | = 0

= Pr X 3 | = 0 + Pr X 3 | = 0

= Pr Z 3 n + Pr Z 3 n

= 2 3 n
which is smaller than the level of significance (check this).
This implies that you would reject the null hypothesis H0 .
2223/2250

Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Confidence Intervals and Hypothesis Tests

Recall the example LRT: testing means normal r.v..
A possible other test is given by the following:
Reject H0 if this confidence interval does not contain 0 i.e.,
reject H0 if:

x < 0 z/2
or
x > 0 + z/2 n .
n
This test clearly has size , because:

Pr X z1/2 n < 0 < X + z1/2 n H0 = 1,
or, equivalently,

Pr
2224/2250
X 0
< z1/2 |H0
/ n

+ Pr
X 0
> z1/2 |H0
/ n

= ,
which is the probability of the rejection region under the null.


A confidence interval can be obtained by inverting a
hypothesis test, and vice versa.
You would reject H0 if:

X 0

/n z1/2 ,
for a level of significance .
Therefore, you accept the null hypothesis if:

x 0
< z1/2 .
/ n
Next, consider the CI of 0 .
2225/2250


For the CI of 0 we have:
z1/2 <
x 0
< z1/2 ,
/ n
or

x z1/2

< 0 < x + z1/2

,
which gives also a 100 (1 )% confidence interval for 0 .

We see that 0 lies in this confidence interval if and only if
you accept the null hypothesis.
In other words, the confidence interval consists of all the
values of 0 for which the null hypothesis is accepted.
2226/2250

Other tests
k-Sample tests
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Other tests
k-Sample tests
k-sample tests
Suppose that k independent random samples N(j , j2 ) are
obtained with sample size nj , j = 1, . . . , k.
We denote xij the observed value of the i th random variable,
from j th random sample. Moreover, denote N = n1 + . . . + nk :
xj
nj
P
xij
nj
i=1
nj
k P
P
j=1i=1
sj2 =
xij
N
k
P
nj x j
j=1
s2 =
nj
P
(xij x j )2
nj 1
i=1
n
j
k P
P
(xij x j )2
N1
j=1i=1
Test for equality in distribution means, assuming unknown,

but common variance j2 = 2 .
2227/2250
Test: H0 : 1 = . . . = k ; l2 = 2 for l = 1, . . . , k v.s.

H1 : i 6= j ; l2 = 2 , for l = 1, . . . , k for at least one pair
i 6= j.

Other tests
k-Sample tests
k-sample tests
We use GLR. Need to find the ratio of the likelihood functions
and thus the ML- estimates in restricted and unrestricted
case. Note: in unrestricted case we still have l2 = 2 for
l = 1, . . . , k.
Likelihood function:
nj
k Y
Y

(xij j )2
1
exp
L(x; 1 , . . . , k , ) =
2 2
2
j=1 i=1
nj
k X
X
1
=(2 2 )N/2 exp 2
(xij j )2
2
2
j=1 i=1
nj
1 XX
N
log(2 2 ) 2
log L =
(xij j )2
2
2
j=1 i=1
2228/2250

Other tests
k-Sample tests
Parameter space
= {(1 , . . . , k , 2 )| < j < , 2 > 0}
log L
j
log L
2
Compute MLE by taking partial derivatives to each of the

k + 1 parameters and equating them to zero:
Pnj
= 21 2 i=1
2(xij j )(1)
= 0, j = 1, . . . , k
Pk Pnj
N
1
2
= 22 2(2 )2 j=1 i=1 (xij j ) (1) = 0
bj =
nj
X
xij /nj = x j
nj
k X
X
b =
(xij x j )2 /N
2
j=1 i=1
i=1
log L
log L
2
b0 =
2229/2250
Under H0 we have 0 = {(, 2 )| < < , 2 > 0}:

P Pnj
= 21 2 kj=1 i=1
2(xij j )(1)
= 0
Pk Pnj
N
1
2
= 22 2(2 )2 j=1 i=1 (xij ) (1) = 0
k
X
j=1
nj x j /N = x
b02
nj
k X
X
j=1 i=1
(xij x)2 /N

Other tests
k-Sample tests
GLR k-sample test

The GLR is:

P Pnj
(xij x)2
(2b
02 )N/2 exp 2b1 2 kj=1 i=1
0

(x) =
=
Pnj
1 Pk
L(x;
b1 , . . . ,
bk ,
b2 )
N/2
2
(xij x j )2
(2b
)
exp 2b2 j=1 i=1
2 N/2
02 )N/2 exp (N/2)
b0
(2b
=
=
<k
N/2
2
b2
(2b
)
exp (N/2)
Pk Pnj
2
b02
j=1
i=1 (xij x)
2 = Pk Pnj
k1
(= k 2/N )
2
b
x
(x
)
j
j=1
i=1 ij
Pk
Pk
2
2
j=1 nj (x j x)
j=1 nj (x j x)
=1 + Pk Pnj
k1
k ?.
Pk Pnj
2
2
j=1
j=1
i=1 (xij x j )
i=1 (xij x j )
b0 ,
b02 )
L(x;
2230/2250
* using MLE estimates

b2 and
b02 from slide 2229.
** see next slide.

Other tests
k-Sample tests
**: see tutorial exercise 5:

nj
nj
k X
k X
k
X
X
X
2
2
(xij x) =
(xij x j ) +
nj (x j x)2 (1)
j=1 i=1
j=1 i=1
j=1
Thus GLR test is equivalent to rejecting H0 if:

Pk
Pk
2
2
2 V3
j=1 nj (x j x)
j=1 nj (x j x)
=
=
k ?,
P
P
nj
2
k
2
2 V2
(N
k)s
(x
x
)
p
j
j=1
i=1 ij
- V1 =
- V2 =
Pk
Pj=1
k
Pkj=1
j=1
Pnj
Pi=1
nj
where
(xij x)2 / 2 2 (N 1),
i=1 (xij
x j )2 / 2 2 (N k),
nj (x j x)2 / 2 2 (k 1).
using (1): V1 = V2 + V3 , thus V3 2 ((N 1) (N k)) = 2 (k 1).
- V3 =
Define F =
thus:
2231/2250
k?
V3 /(k1)
V2 /(Nk)
F (k 1, N k)
= F1 (k 1, N k)
(k1)
(Nk) .

Other tests
k-Sample tests
Analysis of variance
k-sample test: testing the effect of k different experimental
treatments.
Example: Are the mean claim sizes in different countries
equivalent.
We will see this also in Linear Regression.
Rational of the name Analysis of variance: recall (1):
nj
k X
X
(xij x)2 =
j=1 i=1
nj
k X
X
(xij x j )2 +
j=1 i=1
{z
SST
k
X
nj (x j x)2
j=1
{z
SSW
{z
SSB
where
2232/2250
- SST: total variability of the pooled sample data;

- SSW: variation within the individual samples (denominator);
- SSB: variation between the samples (nominator).

Other tests
k-Sample tests
Example: analysis of variance (past CT3 exam)

Summary of claim size data:
group
claim size y
sum assured x
Company
P4
yi
Pi=1
4
2
i=1 yi
1
0.11
0.46
0.71
1.45
1
A
2.73
2.8303
2
0.52
1.43
1.84
2.47
2
B
6.26
11.8018
3
1.48
2.05
2.38
3.31
3
C
9.22
23.0134
4
1.52
2.36
2.95
4.08
4
D
10.91
33.2289
Question: Can you test whether the sum assured has a

significant effect on the claim size?
2233/2250
total
29.12
70.8744

Other tests
k-Sample tests
Example: analysis of variance (past CT3 exam)

Solution:
SST= 70.8744 29.122 /16 = 17.8760;
SSB= (2.732 +6.262 +9.222 +10.912 )/429.122 /16 = 9.6709;
SSW= 17.876 9.6709 = 8.2051.
The value of the test statistic is (N = 16 and k = 4):

F =
2234/2250
SSB/(k 1)
9.6709/3
=
= 4.715
SSW /(N k)
8.2051/12
F&T page 173, 174: F3,12 (4.474) = 2.5% and

F3,12 (5.953) = 1%.
Hence evidence against (p-value between 1% and 2.5%).

Other tests
Jarque-Bera test
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Other tests
Jarque-Bera test
We often assume normality.

Example: Linear regression residuals (week 10-12).
We can match mean and variance of the assumed distribution
and the sample distribution.
How well does the assumed distribution fit the sample
distribution?
Use QQ-plots? How to formally test (see a test on slide
2247)?
Other approach: look at skewness and excess kurtosis
should be jointly zero.
That is the Jarque-Bera test, see Jarque and Bera (1987)

approx.
b2
bX2 + 4X
2 (2), reject H0 if
Test statistic: JB = n6
2235/2250
JB > 21 (2).

Other tests
Jarque-Bera test
OPTIONAL: The prove consists of four parts:

1. Rewrite
bX (and
bX ) asP
functions g (m1 , m2 , mP
3 ), where
P
n
n
2
m1 = i=1 xi /n, m2 = i=1 xi /n, and m3 = ni=1 xi3 /n.
2. Apply the (multivariate) CLT to find the joint probability
function of m1 , m2 1, m3 (MVN):
Y n
Z
/ n
!
n
1 X
Yi
n
n
1 X
Yi
n
!
n
N(0, 2 )
i=1
N(0, )
i=1
3. Use the (multivariate) delta method (week 4) to find the

variance of
bX = g (m1 , m2 , m3 )

Y =g (X ) g (X ) + X X g (X )
Var (Y ) g > (X )g (X )
E[Y ] g (X ),
2236/2250
4. Use n6
bX2 Z 2 2 ,
n 2
bX
24
Z 2 2 hence JB 2 (2)

Other tests
Jarque-Bera test
Prove: Jarque-Bera test (OPTIONAL)

P
1. Find the function (let mj = ni=1 xij /n):

1P
1 Pn
3 3xx 2 + 3x 2 x x 3
3
x
x)
(x
i
i
i
i
i=1
= n
bX n P
3/2

n
1
1P
2 3/2
x)
xi2 2xxi + x 2
(x
i=1 i
n
n
=
m3 3m12 m2 + 2m13
= g (m1 , m2 , m3 )
3/2
m2 m12
2. Note,
under H0 we have E[[m1 m2 m3 ]> ] = [0 1 0]> . Hence
n [m1 m2 1 m3 ]> N(0, ), where:
xi
xi xi
xi (xi2 1)
xi xi3

=E xi2 1 xi xi2 1 xi3 = E (xi2 1)xi (xi2 1)(xi2 1) (xi2 1)xi3
xi3
xi3 xi3
xi3 xi
xi3 (xi2 1)
xi3 xi
xi4
1 0 3
xi2
3
4
2
5
=E xi xi xi 2xi + 1 xi xi3 = 0 2 0
xi4
xi5 xi3
xi6
3 0 15
2237/2250

Other tests
Jarque-Bera test
Prove: Jarque-Bera test (OPTIONAL)

3. Find E[b
X ] and Var (b
X ) (* using quotient rule & algebra)
g (m1 , m2 , m3 )/m1
g (m1 , m2 , m3 ) = g (m1 , m2 , m3 )/m2
g (m1 , m2 , m3 )/m3
3(m1 m3 m22 )/(m2 m12 )5/2
= (3m1 (m1 1) + 3/2(m1 m2 m3 ))/(m2 m12 )5/2

1/(m2 m12 )5/2
Thus g (X ) = g (0, 1, 0) = 0 = E[b

X ]
Thus g (X ) = g (0, 1, 0) = [3 0 1]> , hence
Var (b
X ) = g > (0, 1, 0)g (0, 1, 0) = 6.
p
approx.
approx.
This implies: n/6b
X Z n/6b
X2 2 (1).
Similar for
bX , where h(X ) = [0 6 0 1].
2238/2250
4. Note:
bX and
bX are independent (using delta method).

Other tests
Jarque-Bera test
Jarque-Bera test: small sample sizes

The Jarque-Bera test is an approximate test (if n ). How
to use in small samples?
Calculated p-value equivalents to true levels at given
sample sizes (JB = in the approximate JB test):
True level n =20 n =30 n =50 n =70 n =100
JB =0.1
0.307
0.252
0.201
0.183
0.1560
JB
=0.05
0.1461 0.109
0.079
0.067
0.062
JB =0.025 0.051 0.0303 0.020
0.016
0.0168
JB =0.01
0.0064 0.0033 0.0015 0.0012
0.002
(Calculated using bootstrap test method (see later this
lecture) other values: use interpolation).
Hence: Jarque-Bera approximately test rejects the null
hypothesis more often than it should do.
2239/2250

Introduction
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Introduction
We have seen until now pivotal tests.

What if we cannot find a pivot?
Hypothesis testing techniques:
1. Pivotal tests: here we find a pivotal test-statistic T such that
T (X ) has the same distribution for all values of 0 .
2. Conditional tests: here we convert the composite hypothesis
into a simple by conditioning on a sufficient statistic.
3. Bootstrap tests: here we replace the composite hypothesis 0
b {}
b 0 .
by the simple {},
2240/2250

Conditional tests
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Conditional tests
Application
OPTIONAL
Consider the case in which an health insurer has two types of
insured, young individuals and old individuals.
You observe the age, you know the proportion of your
portfolio in young/old and you observe the total number of
claims each month.
You can approximate the number of claims by a Binomial
distribution (why?).
Question: Can you test wether the probability of issuing a
claim for young individuals is smaller than for old individuals?
Notation: X Bin(n1 , p1 ) the number of claims for young
individuals, Y Bin(n2 , p2 ) the number of claims for old
individuals, and S = X + Y the total number of claims.
2241/2250

Conditional tests
OPTIONAL: We test: H0 : p1 = p2 = p v.s. H1 : p1 < p2 .

1. The joint density under H0 is:

n1
n2
fX ,Y (x, y ) =
p x+y (1 p)n1 +n2 xy

x
y
2. The density of S = X + Y under H0 is S Bin(n1 + n2 , p).
3. The density of Y |S = s under H0 is equal to
Y = S X |S = s:
fS,Y (s, y )
fX ,Y (s y , y )
=
fS (s)
f (s)
n2 s S
n1
n1 +n2 s
sy y p (1 p)
= n1 +n2 s
=
p (1 p)n1 +n2 s
s
fY |S (y ) =
2242/2250
for y = 0, . . . , s and s = 0, . . . , n1 + n2 . Hence

Y |S Hyper(s, y , n1 + n2 ), independent of p!
n1
n2
sy y

n1 +n2
s

Conditional tests
Hence, best critical region, reject H0 if y k(s), where s is

the largest value such that:
n1
n2
s
X
i si
.
n1 +n2
i=y
Relation with sufficient statistics:

If a sufficient statistic S exists for an unknown nuisance
parameter , then the distribution of X |S will not depend on
.
2243/2250
P
Exercise: (OPTIONAL) Let Xi N(, ) and S = ni=1 Xi
is sufficient if 2 is known. Test H0 : 0 v.s. H1 : < 0 ,
when n = 2.
Note: can do for general n, but becomes messy.

Conditional tests
1. The joint density under H0 is:

2 n/2
fX (x) = (2 )
2. The density of S =
Pn
i=1 Xi
Pn
2
i=1 (xi )
exp
2
under H0 is S N(n , n 2 ).
3. The density
Pof Xn |S = s under H0 is equal to
Xn = S n1
i=1 Xi |S = s:
fS,X1 ,...,Xn1 (s, x1 , . . . , xn1 )
fS (s)
P
fX1 ,...,Xn (x1 , . . . , xn1 , s n1
i=1 xi )
=
fS (s)
P

Pn1
n1
2
2
2
n/2
i=1 (xi ) +(s i=1 xi )
(2 )
exp
2

=
2
(2 n 2 )1/2 exp (sn)
n 2
fX1 ,...,Xn1 |S (x1 , . . . , xn1 ) =
2244/2250

Conditional tests
Consider n = 2 we have:
fX1 |S (x1 ) = n (2 2 )(n1)/2

!
P
Pn1
Pn1
2 + (s
2
( ni=1 xi n )2
(x
)
x
)
i
i
i=1
exp
i=1
n 2
2

(x1 s/2)2
2
1/2
=(2 /n)
exp
2 2
* use: (x1 )2 + (s x1 )2 (s 2)2 /2 = (x12 + 2 2x1 )
+ ((s x1 )2 + 2 2(s x1 )) s 2 /2 22 + 2(x1 + x2 )
=x12 + (s x1 )2 s 2 /2
=2x12 + s 2 /2 2sx1 = (2x1 s) (x1 s/2) = 2(x1 s/2)2
Notice, this is the p.d.f. of N(s/2, 2 /2).
2245/2250
Hence, best critical region, reject

H0 if
x1 > k(s) = s/2 + z1 / 2:

Bootstrap tests
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Bootstrap tests
Introduction
Sometimes, we have a test which does not allow for pivots or
conditions.
Sometimes, we only have approximate test what if n is
small? Example: Jarque-Bera when n < 200.
Use bootstrap tests.
Idea:
1.
2.
3.
4.
Define H0 and H1 ;
Create a statistic T , based on H0 and H1 ;
Determine T ? based on the sample.
Simulate m times from H0 (m is large), calculate Tj for
j = 1, . . . , m.
5. Calculate the probability that T ? < Tj , use that for critical
region.
2246/2250

Bootstrap tests
Example: Testing goodness of fit

1. Test H0 : Xi Pareto(1, 2) (note: mean does not exist!) v.s.
H1 : not Pareto(1,2) (see excel file, n = 5, m = 100).
2. Use sum of the squared differences between the empirical
distribution and the fitted Pareto distribution function
evaluated at the sampled points, i.e.,
2
T (x) =
n
X
((i 0.5)/n) (1 (2/(x(i) + 2))1 ) .

|
{z
} |
{z
}
i=1
emp. quantile
FX (x(i) )
3. Calculate T ? using the sample data.

4. Simulate m times n random variables from a Pareto(1,2)
distribution and calculate for each the value of T .
2247/2250
5. Reject H0 if T ? is larger than the m (1 ) largest value of

T in the simulation.

Bootstrap tests
Example: Comparing models

1. Test H0 : Xi Gamma(b
ML , bML ) v.s. H1 :
Xi LN(b
ML , (b
2 )ML ).
2. Use log-likelihood to compare the models (what do we know if
n is large?)
ML , (b
2 )ML )) log(L0 (b
ML , bML )).
T (x) = log(L1 (b
3. Calculate T ? using the sample data.
4. Simulate m times n random variables from a Gamma
distribution and calculate for each the value of T .
5. Reject H0 if T ? is larger than the m (1 ) largest value of
T in the simulation.
2248/2250

Bootstrap tests
Bootstrap tests
Advantages:
- Allows a wider variety of tests (i.e., does not require test
statistic);
- Is exact, i.e., no approximations made.
Disadvantages:
- Time consuming (programming/ computer time);
- Depends on simulations: how many/how exact (quantifiable
using non-parametric hypothesis test: see week 9);
- More complicated to find the power of the test.
2249/2250

Summary
Summary
Hypothesis testing
Hypothesis testing
Overview
p-value
Other tests
k-Sample tests
Jarque-Bera test
Introduction
Conditional tests
Bootstrap tests
Summary
Summary

Summary
Summary
Summary
Type I error: Pr (Reject H0 |H0 is true) = .
Type II error: Pr (Do not reject H0 |H1 is true) = .
Power: = 1 = Pr (Do reject H0 |H1 is true).
p-value: For any level of significance less than the p-value
you would accept the null hypothesis.
k-sample test: k independent normal distributed samples,
test whether means are all equal: Use F -statistic.
Jarque-Bera test: Test whether a distribution is normally
distributed (more accurate: is symmetric (skewness = 0) and
excess kurtosis equals zero).
Other testing methods: Conditional tests (using sufficient
statistics) & Bootstrap tests.
2250/2250

Week 8 Annotated

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Week 8 Annotated

Caricato da

Copyright:

Formati disponibili

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics

ACTL2002/ACTL5101 Probability and Statistics: Week 8

First six weeks

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

This & last week

Properties of hypothesis testing.

Using asymptotic distribution of LR for testing.

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Summary last week: Hypothesis tests

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Summary last week: Best critical region

ii) for all 0 and all critical regions C of size we have:

Generalized likelihood ratio test: simple/composite

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Exercise: testing a mean, variance unknown

Note: the variance 2 is unknown, but s 2 is known.

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Exercise: testing a mean, variance unknown

b2 = n1 ni=1 (xi x)2 (note: biased estimator!).

The maximum under 0 is given by (show this for yourself):

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Thus, the likelihood ratio is given by:

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Exercise: testing a mean, variance unknown

Thus we have that the best critical region is of the form:

ACTL2002/ACTL5101 Probability and Statistics: Week 8

N(0 , 2 /n), thus:

combining * and **:

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Type I and Type II Errors

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Type I, Type II errors

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Type I error: reject H0 when it is true, with probability :

Question: Test H0 : 0 v.s. H1 : > 0 , what is the

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Type II error & power

The value of the power function at a specific parameter point

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Example: type I and II errors and power

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Example: type I and II errors and power

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Example: type I and II errors and power

ACTL2002/ACTL5101 Probability and Statistics: Week 8

We know: a test at significance level would reject H0 if:

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Exercise: Bernoulli random variable

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Exercise: Bernoulli random variable

Question: What is the power of the test? Solution: it

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics: Week 8

Confidence Intervals and Hypothesis Tests