1

Extension of measure
Sayan Mukherjee
Dynkins theorem
We will soon need to dene probability measures on innite and possible uncountable sets, like the
power set of the naturals. This is hard. It is easier to dene the measure on a much smaller collection of
the power set and state consistency conditions that allow us to extend the measure to the the power set.
This is what Dynkins theorem was designed to do.
Denition 0.0.1 (-system) Given a set a system is a collection of subsets P that are closed under nite
intersections.
1) P is non-empty; 2) A B P whenever A, B P.
Denition 0.0.2 (-system) Given a set a system is a collection of subsets L that contains and is closed
under complementation and disjoint countable unions.
1) L; 2) A L A
c
L; 3) A
n
L, n 1 with A
i
A
j
= i = j

n=1
A
n
L.
Denition 0.0.3 (-algebra) Let F be a collection of subsets of . F is called a eld (algebra) if F and F
is closed under complementation and countable unions,
1) F; 2) A F A
c
F; 3) A
1
, ..., A
n
F

n
j=1
A
j
F; 4) A
1
, ..., A
n
, ... F

j=1
A
j
F.
A -algebra is a system.
A -algebra is a system but a system need not be a algebra, a system is a weaker system. The
difference is between unions and disjoint unions.
Example 0.0.1 = {a, b, c, d} and L = {, , {a, b}, {b, c}, {c, d}, {a, d}, {b, d}, {a, c}} L is closed under
disjoint unions but not unions.
However:
Lemma 0.0.1 A class that is both a system and a system is a -algebra.
Proof.
We need to show that the class is closed under countable union.
Consider A
1
, A
2
, ..., A
n
L note that these A
i
are not disjoint.
We disjointify them, B
1
= A
1
and B
n
= A
n
\

n1
i=1
A
i
= A
n

n1
i=1
A
c
i
. B
n
L by system and
complement property of system.

i=1
A
i
=

i=1
B
i
but the {B
i
} are disjoint so

_
i=1
A
i
=

_
i=1
B
i
L.
We will start constructing probability measures on innite sets and we will need to push the idea of
extension of measure. In this the following theorem is central.
Theorem 0.0.1 (Dynkin) 1) If P is a system and L is a system, then P L implies (P) L.
b) If P is a system (P) = L(P).
Proof of a).
Let L
0
be the system generated by P , the intersection of all systems containing P. This is a
system and contains P and is contained in every system that contains P. So P L
0
L. If L
0
is
also a system, it is a system and so is a algebra. By minimality of (P) it follows (P) L
0
so
P (P) L
0
L. So we need to show that L
0
is a system.
Set A (P) and dene
G
A
= {B (P) : A B L(P)}.
2
1) We show if A L(P) then G
A
is a system.
i) G
A
since A = A L(P).
ii) Suppose B G
A
. B
c
A = A\(AB) but B G
A
means AB L(P) and since A L(P)
it holds A \ (A B) = B
c
A L(P) by complementarity. Since B
c
A L(P) it holds
that B
c
G
A
.
iii) {B
j
} are mutually disjoint and B
j
G
A
then
A

_
j=1
B
j
) =

_
j=1
(A B
j
) L(P).
2) We show if A P then L(P) G
A
.
Since A P L(P) from 1) G
A
is a system,
For B P we have AB P since A P and P is a system. So if B P then AB P
L(P) implies B G
A
so P G
A
and since G
A
is a systemL(P) G
A
.
3) We rephrase 2) to state if A Pand B L(P) then AB L(P) (if we can extend Ato L(P)
we are done).
4) We show that if A L(P) then L(P) G
A
.
If B P and A L(P) then A B L(P) then from 3) it holds A B L(P). So when
A L(P) it holds P G
A
so from 1) G
A
is a system and L(P) G
A
.
5) If A L(P) then for any B L(P), B G
A
then A B L(P) .
The following lemma is a result of Dynkins theorem and highlights uniqueness.
Lemma 0.0.2 If P
i
and P
2
are two probability measures on (P) where P is a system. If P
1
and P
2
agree
on P, then they agree on (P).
Proof
L = {A P : P
1
(A) = P
2
(A)},
is a system and P L so (P) L.
We will need to dene a probability measure, , for all sets in a algebra. It will be convenient and
almost necessary to dene a (pre)-measure,
0
, on a smaller collection F
0
that extends to on (F
0
) and
is unique,
1
(F) =
2
(F), F F
0
implies
1
(F) =
2
(F), F (F
0
).
Restatement of Dynkins theorem in the above context is that the smallest algebra generated from
a system is the system generated from that system.
Theorem 0.0.2 Let F
0
be a system then (F
0
) = (F
0
).
We now dene the term extension and develop this idea of extension of measure. The main working
example we will use in the development is.
= R, S = {(a, b] : a b }, P((a, b]) = F(b) F(a).
Remember, that the Borel sets B(R) = (S).
Goal: we have a probability measure on S we want to extend it to B(R).
Extension: If G
1
and G
2
are collections of subsets of with G
1
G
2
and
P
1
: G
1
[0, 1], P
2
: G
2
[0, 1].
P
2
is an extension of P
1
if
P
2|G1
= P
1
, P
2
(A
1
) = P
1
(A
i
) A
i
G
1
.
3
Few terms:
A measure is P additive if for disjoint sets A
1
, ..., A
n
G
n

i=1
A
i

n
_
i=1
A
i
G P(
n

i=1
A
i
) =
n

i=1
P(A
i
).
A measure is additive if for disjoint sets A
1
, ..., A
n
, ... G

i=1
A
i
G P(

i=1
A
i
) =

i=1
P(A
i
).
Denition 0.0.4 (Semialgebra) A class S of subsets of is a semialgebra if
1) , S
2) S is a system
3) If A S there exists disjoint sets C
1
, ..., C
n
with each C
i
S such that A
c
=

n
i=1
C
i
.
Example of Semialgebra: Intervals
= R, S = {(a, b] : a b }.
If I
1
, I
2
S then I
1
I
2
S and I
c
is a union of disjoint intervals.
The idea/plan:
1. Start with a semialgebra S
2. Show a unique extension to an algebra A(S) generated by S
3. Show a unique extension to (A(S)) = (S).
If S is a semialgebra of then
A(S) = {

iI
S
i
: I is nite , {S
i
} is disjoint , S
i
S},
a system of nite collections of mutually disjoint sets.
An example of extension of measure for a countable set.
= {
1
, ....,
n
, ...} i p
i
p
i
0,

i = 1

p
1
= 1.
F = P() the power set of . We dene for all A F the following extension of measure
P(A) =

iA
p
i
.
For this to be a proper measure the follow need to hold (they do)
1. P(A) 0 for all A F
2. P() =

i=1
p
i
= 1
3. If {A
j
} with j 1 are disjoint then
P(

j=1
_
A
j
) =

wijAj
p
i
=

iAj
p
i
=

j
P(A
j
).
Is the power set P() countable ? What was the semialgebra above ? What would we do if = R ?
As we consider the following theorems and proofs or proof sketches a good example to keep in mind
is the case where = R and S == {(a, b] : a < b }.
4
Theorem 0.0.3 (First extension) S is a semialgebra of and P : S [0, 1] is -additive on S and P() = 1.
There is a unique extension P

of P to A(S) dened by
P

(S
i
) =

i
P(S
i
),
P

() = 1 and P

is additive on A(S).
Proof (Page 46, Probability Path by Resnick)
We need to show that
P

(S
i
) =

i
P(S
i
),
uniquely denes P

. Suppose that A A(S) has representations

A =

iI
S
i
=

jJ
S

j
,
we need to verify

iI
P(S
i
) =

jJ
P(S

j
).
Since S
i
A the following hold

iI
P(S
i
) =

iI
P(S
i
A) =

iI
P(S
i

jJ
S

j
)
=

iI
P(

jJ
S
i
S

j
),
and S
i
=

jJ
S
i
S

j
S and P is additive on S so

iI
P(S
i
) =

iI

jJ
P(S
i
S

j
) =

iI

jJ
P(S
i
S

j
),
and by the same argument

jJ
P(S

j
) =

iI

jJ
P(S
i
S

j
) =

iI

jJ
P(S
i
S

j
).
We now check that P

is additive on A(S). Suppose for i 1,

A
i
=

jJi
S
ij
A(S), S
ij
S,
and {A
i
, i 1} are mutually disjoint and
A =

i=1
A
i
A(S).
Since A A(S), A can be represented as
A =

kK
S
k
, S
k
S, k K,
where K is a nite index set. So by the denition of P

(A) =

kK
P(S
k
).
We can write
S
k
= S
k
A =

i=1
S
k
A
i
=

i=1

jJi
S
k
S
ij
.
5
Since S
k
S
ij
S and

i=1

jJi
S
k
S
ij
= S
k
S and P is additive on S

kK
P(S
k
) =

kK

i=1

jJi
P(S
k
S
ij
) =

i=1

jJi

kK
P(S
k
S
ij
).
Note that

kK

S
k
S
ij
= A S
ij
= S
ij
S,
and by additivity of P on S we show additivity

i=1

jJi

kK
P(S
k
S
ij
) =

i=1

jJi
P(S
ij
)
=

i=1
P(

jJi
S
ij
) =

i=1
P

(A
i
).
The last step is to check the extension is unique. Assume P

1
and P

2
aretwo additive extensions, then
for any
A =

iI
S
i
A(S)
we have
P

1
(A) =

iI
P(S
i
) = P

2
(A).
We now show how to extend from an algebra to a sigma algebra.
Theorem 0.0.4 (Second extension, Caratheodory) A probability measure P dened on an algebra A has a
unique extension to a probability measure P

on (A).
Proof sketch (Page 4, Probability Theory by Varadhan)
There are 5 steps to the proof.
1. Dene the extension:
{A
j
} are the set of all countable collections of A that are disjoint and we dene the outer measure
P

as
P

(A) = inf
AjAj
_
_

j
P(A
j
)
_
_
.
P

will be the extension.

2. ShowP

satises properties of a measure

i) Countable subattitivity
P

(
j
A
j
)

j
P

(A
j
).
ii) For all A A, P

(A) = P(A). This is done in two steps by showing 1) P

(A) P(A) and 2)

P

(A) P(A).
3. Checking measurability of the extension (are all of the extra sets in going from the algebra to the
algebra adding correctly).
A set E is measurable if
P

(A) P

(A E) +P

(A E
c
),
for all A A. The class M is the collection of measurable sets. We need to show that M is a
algebra and P

is additive on M.
6
4. ShowA M.
This implies that (A) Mand P

extends fromA to (A).

5. Uniqueness.
Let P
1
and P
2
be additive measures on A then show if for all A A
P
1
(A) = P
2
(A), then P
1
(B) = P
2
(B), B (A).
We now look at two examples. The rst is the extension of the Lebesgue measure to Borel sets. The
second example illustrates sets that are not measurable.
Lebesgue measure:
= [0, 1], F = B((0, 1]), S = {(a, b] : 0 a b 1}.
S is a semialgebra and we dene the following measure on it
: S (a, b], () = 0, ((a, b]) = b a.
This measure extends to the Borel set B((0, 1]).
To show that extends to B((0, 1]) we need to show that it is additive on B((0, 1]) .
We rst show nite additivity.
Set (a, b] =

K
i=1
(a
i
, b
i
], where a = a
1
, b
K
= b, b
i
= a
i+1
. We can see
K

i=1
(a
i
, b
i
] =
K

i=1
(b
i
a
i
)
= b
1
a
1
+b
2
a
2
+
= b
K
a
1
= b a.
We now note a few asymptotic properties of intervals
(a, b) =

_
n=1
_
a, b
1
n
_
, a < b
[a, b] =

n=1
_
a
1
n
, b
_
, a < b
{a} =

n=1
_
a
1
n
, a
_
, a < b.
The Borel algebra contains all singletons a as well as the forms
(a, b), [a, b], [a, b), (a, b].
We now have to address additivity. First note that the interval [0, 1] is compact, this means that
there exists a nite cover of the interval for any > 0. Let A
n
be a sequence of sets from S such that
A
n
. We can dene a set B
n
S such that its closure [B
n
] A
n
and
(A
n
) (B
n
) 2
n
.
Remember A
n
= and since the sets [B
n
] are closed there exists a nite n
0
() such that
n0
i=1
[B
n
] = .
Remember that A
n0
A
n01
A
1
so
(A
n0
) =
_
A
n0
\
n0

k=1
B
k
_
+
_
n0

k=1
B
k
_
=
_
A
n0
\
n0

k=1
B
k
_

_
n0
_
k=1
A
k
\ B
k
_

n0

i=1
(A
k
\ B
k
)
n0

k=1
2
k
.
7
The reason this shows additivity is that if for all A
n
S, A
n
implies (A
n
) 0 then additivity
holds on (S).
Not a measurable set:
Let = N be the natural numbers and E be the even numbers and E
c
the odd numbers and set
F =

_
k=0
{2
2k
+ 1, ..., 2
2k+1
} = {2, . 5, ..., 8, 17, ..., 32, 65, ..., 128, ...}.
1. For n = 2
2k
the ratio P
n
(F) = #[F {1, ..., n}]/n = (n 1)/3n 1/3 and for n = 2
2k+1
,
P
n
(F)(2n 1)/3n 2/3 so P
n
cannot converge as n .
2. The even and odd portions of F and F
c
, A F E and B F
c
E
c
, have relative frequencies
that do not converge.
3. C = A B does have a relative frequency that converges: P
2n
(C) = 1/2 + 1/2n so lim
n
P
n
=
1/2.
4. Both E and C have well dened asymptotic frequences but A = E C does not and the collection
of sets M for which P
n
converges is not an algebra.