Section on Survey Research Methods JSM 2008

Variance of Sample Variance

Eungchun Cho

Moon Jung Choy

Abstract

The variance of variance of nite samples taken from a nite population with replacement is expressed in terms of the sample
size and the second and fourth order moments of population.

Key Words: Sample Variance, Sample with replacement, Randomization Variance, Moments, Finite Population

1. Introduction
We give a formula of the variance of with-replacement sample variance in terms of the sample size and the second
and fourth moments of the population about the mean. The derivation of the formula does not require working
with the more elaborate \polykay" approach of Tucky 3] 4] 5] 6]. Formula for the variance of the variance of
without-replacement samples from a nite population given in Cho et al. 1] is quoted for comparison at the end of
this paper.

2. Main Theorem
Let A be a nite set fa1 : : : aN g and s a sample of n elements fx1 : : : xn g taken from A with replacement. n is

not bounded by the population size N , though often in practice n << N . The sample s is viewed as a realization
of independent identically distributed random variables X1 : : : Xn on A. Following notation will be used.

X =
k

Pn ;X ; X 2
Pn X
2
i=1 i
S = i=1 n ;i 1
PN a k
PNn (a ; )k
i=1

PN a
= i=1 i
N

i=1 i

Theorem 1 Let S be the variance of of with-replacement samples of size n from a set A of real numbers a1 a2 : : : aN .
2

The variance of S 2

V ar S 2

1
= n
= 1

; n;3
n;1

; ;
n 4

(1)

+ O n;2

Proof. Let Zi = Xi ; for i = 1 2 : : : n so that E (Zi) = 0. Since V ar ;S 2 = E ;S 4 ;

expression of E (S ) in terms of n and the moments. We can write

4

(2)
2

, we derive an

Pn Z 2 Pn 2
S 2 = n i=1 n i(n; ( 1) i=1 Zi )
;

and by squaring

;Pn Z 2 2 ; 2 n ;Pn Z 2 (Pn Z )2 + (Pn Z )4

i=1 i
i=1 i
i=1 i
i=1 i
2 (n ; 1)2
n
;
;Pn Z 2 (Pn Z )2 + E (Pn Z )4
2 E Pn Z 2 2 ; 2 n E
n
i=1 i
i=1 i
i=1 i
i=1 i

n2
=

E (S 4 ) =

n2 (n ; 1)2

Kentucky State University, Frankfort, KY, 40601, USA, Current address: eccho@math.snu.ac.kr, Seoul National University, Seoul,
Korea, Eungchun Cho's work at Seoul National University was supported by The Korea Research Foundation and The Korean Federation
of Science and Technology Socities Grant funded by Korea Government (MOEHRD, Basic Research Promotion Fund).
y U.S. Bureau of Labor Statistics 2 Massachusetts Avenue, NE Washington, DC 20212 USA
1291

Section on Survey Research Methods JSM 2008

Since Z1 : : : Zn are independent, we have

E i
;Z(ZZZ2j ) = 0 2
E i2 j = 2

E Zi 3 Zj = 0
;
E Zi 4 = 4

E Zi 2 Zj Zk = 0
for distinct i j k:

Routine algebraic simpli cation with the expected values given above yields

n
X 2!2

= n

Zi

i=1

+ n (n ; 1)

+ n (n ; 1)

= n

0 n !
1
X 2 X !2 A
Zi
Zi
E@
i=1
i=1
n
X !4
i=1
n

= n

Zi

+ 3 n (n ; 1)

(3)

(4)

(5)

Substitution of (3), (4) and (5) into the expansion of E S 4 and simpli cation give

;
E S4

and

V ar S 2

(n ; 1)

+ n2 ; 2n + 3
n (n ; 1)

(6)

= E S4 ; 22
;
(n ; 1) 4 + n2 ; 2n + 3
=
n (n ; 1)

1
= n 4 ; n ; 3 22
n;1
1; ; 2 + 2
= n 4 2
n(n ; 1)

To obtain an expression of the formula of V ar(S 2 ) in terms of and the moments 02 ,

substitute
2
4

into (1) and get

V ar S 2

=
=

0
2
0
4

; 2
; 4 03 + 6

2 0
2

;3

and

0
4

about zero, we

4
; n 03 ; n nn;;31) 02 2 +
(
n ; 3)
n ; 3)
+ 4n(2n ; 1) 2 02 ; 2n(2n ; 1)
(
(

1
= n

0
3

0
4

(7)

3. Comparison with Without-replacement Samples

Here we compare (1) with the variance of variance of without-replacement samples given in 1]. Let V arwo S 2
denote the variance of variance of without-replacement samples of size n from A. The following is a simpli ed
(improved) version from 1].

V arwo S 2 = c1
where

+ c3

(N ; n (
; 1)
c1 = n (N ; 1) (N);Nn(; N ; nN ; 1)
n
; 3) N ; 2) (
N (N ; n) N 2 n ; 3 n ; 3 N 2 + 6 N ; 3
c3 = ;
n (n ; 1) (N ; 1)2 (N ; 2) (N ; 3)
1292

(8)

(9)

Section on Survey Research Methods JSM 2008

We note

; n nn;;31) 2 2
; (
= V ar S 2
;
;
;
;
as expected. The di erence of V arwo S 2 and V ar S 2 is of order 1=N , that is, jV arwo S 2 ; V ar; S 2 j is O(N ;1 ).
;
In most practical situations where n = cN for some c;> 0 and 0 < ; < 1, jV arwo S 2 ; V ar S 2 j is O(n; 1 ).
p
For example, if n = N , then the di erence of V arwo S 2 and V ar S 2 is O(n;2 ). As we did for V ar(S 2 ), we
;
represent V arwo S 2 in terms of the moments 02 and 04 about zero by substitution of (7) into (8).
lim V arwo S 2

N !1

V arwo S 2

= c1

0
4

1
= n

+ c2

0
3

+ c3

02
2

+ c4

2 0
2

+ c5

(10)

where c1 and c3 are as before (9) and

(N ; n (
; 1)
c2 = ;4 n (N ; 1) (N);Nn(; N ; nN ; 1)
n
3) N ; 2) (
N 2 (N ; n) (2 Nn ; 3 N ; 3 n + 3)
c4 = 4
n (n ; 1) (N ; 1)2 (N ; 2) (N ; 3)
2
c5 = ;2 N (N ; n) (2 Nn2 ; 3 N ; 3 n + 3)
n (n ; 1) (N ; 1) (N ; 2) (N ; 3)
Here again, each ci converges to the corresponding coe cient in (7).
4
n ; 3)
lim c2 = ; n
lim c = 4(2n ; 1)
N !1 4
n(
lim c = ; n2n ; 3
N !1 5
(n ; 1)

N !1

4. Acknowledgment
The views expressed in this paper are those of the authors and do not necessarily re ect the policies of the U.S.
Bureau of Labor Statistics. The authors thank John Eltinge for many helpful suggestions that improved the paper.

References
1] Eungchun Cho, Moon Jung Cho and John Eltinge, The variance of sample variance from a nite population, Proceedings of Joint
American Statistical Association and International Statistical Institute Conference, Toronto, Canada, August, 2004
2] W. G. Cochran, Sampling Techniques (3rd ed. ), John Wiley, 1977.
3] J. W. Tukey, Some sampling simpli ed, Journal of the American Statistical Association, 45 (1950), 501-519.
4] J. W. Tukey, Variances of variance components: I. Balanced designs. The Annals of Mathematical Statistics, 27 (1956), 722-736.
5] J. W. Tukey, Variances of variance components: II. Unbalanced single classi cations. The Annals of Mathematical Statistics, 28
(1957), 43-56.
6] J. W. Tukey, Variance components: III. The third moment in a balanced single classi cation. The Annals of Mathematical Statistics,
28 (1957), 378-384.

1293