2.2.

STANDARD SPACES

27

where Fi is either Fi or Fic . In fact, let us call such sets intersection sets and observe that any
two intersection sets must be disjoint since for at least one i one intersection set must lie inside Fi
and the other within Fic . Thus all intersection sets must be disjoint. Next observe that any field
element can be written as a finite union of intersection setsjust take the union of all intersection
sets contained in the given field element. Let G be an atom of F. Since it is an element of F, it is
the union of disjoint intersection sets. There can be only one nonempty intersection set in the union,
however, or G would not be an atom. Hence every atom is an intersection set. Conversely, if G is an
intersection set, then it must also be an atom since otherwise it would contain more than one atom
and hence contain more than one intersection set, contradicting the disjointness of the intersection
sets.
In summary, given any finite field F of a space we can find a unique collection of atoms A of
the field such that the sets in A are disjoint, nonempty, and have the entire space as their union
(since is a field element and hence can be written as a union of atoms). Thus A is a partition of
. Furthermore, since every field element can be written as a union of atoms, F is generated by A
in the sense that it is the smallest field containing A. Hence we write F = F(A). Observe that if we
assign nonnegative numbers pi to the atoms Gi in A such that their sum is 1, then this immediately
gives a finitely additive, nonnegative, normalized set function on F by the formula
X
pi .
P (F ) =
i:Gi F

Furthermore, this set function is trivially countably additive since there are only a finite number of
elements in F.
The next step is to consider an increasing sequence of finite fields that in the limit will give a
countable generating field. A sequence of finite fields Fn , n = 1, 2, . . . is said to be increasing if the
elements of each field are also elements of all of the fields with higher indices, that is, if Fn Fn+1 ,
all n. This implies that if An are the corresponding collections of atoms, then the atoms in An+1
are formed by splitting up the atoms in An . Given an increasing sequence of fields Fn , define the
limit F as the union of all of the elements in all of the Fn , that is,
F=

Fi .

i=1

F is easily seen to be itself a field. For example, any F F must be in Fn for some n, hence
the complement F c must also be in Fn and hence also in F. Similarly, any finite collection of sets
Fn , n = 1, 2, . . . , m must all be contained in some Fk for sufficiently large k. The latter field must
hence contain the union and hence so must F. Thus we can think of the increasing sequence Fn as
increasing to a limit field F and we write
Fn F
if Fn Fn+1 , all n, and F is the union of all of the elements of the Fn . Note that F has by
construction a countable number of elements. When Fn F, we shall say that the sequence Fn
asymptotically generates F.
Lemma 2.2.1 Given any countable field F of subsets of a space , then there is a sequence of
finite fields {Fn ; n = 1, 2, . . .} that asymptotically generates F. In addition, the sequence can be
constructed so that the corresponding sets of atoms An of the field Fn can be indexed by a subset of
the set of all binary n-tuples, that is, An = {Gun , un B}, where B is a subset of {0, 1}n , with the
property that Gun Gun1 . Thus if un is a prefix of um for m > n, then Gum is contained in Gun .
(We shall refer to such a sequence of atoms of finite fields as a binary indexed sequence.)

28

CHAPTER 2. STANDARD ALPHABETS

Proof: Let F = {Fi , i = 0, 1, . . .}. Consider the sequence of finite fields defined by Fn = F(Fi , i =
0, 1, . . . , n 1), that is, the field generated by the first n elements in F. The sequence is increasing
since the generating classes are increasing. Any given element in F is eventually in one of the Fn and
hence the union of the Fn contains all of the elements in F and is itself a field, hence it must contain
F. Any element in the union of the Fn , however, must be in an Fn for some n and hence must be an
element of F. Thus the two fields are the same. A similar argument to that used above to construct
the atoms of an arbitrary finite field will demonstrate that the atoms in F(F0 , . . . , Fn1 ) are simply
Tn1
all nonempty sets of the form k=0 Fk , where Fk is either Fk or Fkc . For each such intersection set
let un denote the binary n-tuple having a one in each coordinate i for which Fi = Fi and zeros in
the remaining coordinates and define Gun as the corresponding intersection set. Then each Gun is
either an atom or empty and all atoms are obtained as u varies through the set {0, 1}n of all binary
n-tuples. By construction,
Gun if and only if ui = 1Fi (), i = 0, 1, . . . , n 1,

(2.1)

where for any set F 1F () is the indicator function of the set F , that is,

1 F
1F () =
0 6 F .
Eq.2.1 implies that for any n and any binary n-tuple un that Gun Gun1 , completing the proof.
2
Before proceeding to further study sequences of finite fields, we note that the construction of the
binary indexed sequence of atoms for a countable field presented above provides an easy means of
determining which atoms contain a given sample point. For later use we formalize this notion in a
definition.
Given an enumeration {Fn ; n = 0, 1, . . .} of a countable field F of subsets of a sample space
and the single-sided binary sequence space
M = {0, 1}Z+ =
i=0 {0, 1},
define the canonical binary sequence function f : M by
f () = {1Fi () ; i = 0, 1, . . .}.

(2.2)

Given an enumeration of a countable field and the corresponding binary indexed set of atoms An =
{Gun } as above, then for any point we have that
Gf ()n , n = 1, 2, . . . ,

(2.3)

where f ()n denotes the binary n-tuple comprising the first n symbols in f ().
Thus the sequence of decreasing atoms containing a given point can be found as a prefix of
the canonical binary sequence function. Observe, however, that f is only an into mapping, that is,
some binary sequences may not correspond to points in . In addition, the function may be many
to one, that is, different points in may yield the same binary sequences.
Unfortunately, the sequence of finite fields converging upward to a countable field developed
above does not have sufficient structure to guarantee that probability measures on the finite fields
will imply a probability measure on the limit field. The missing item that will prove to be the key
is specified in the following definitions:
A sequence of finite fields Fn , n = 0, 1, . . ., is said to be a basis for a field F if it has the following
properties:

2.2. STANDARD SPACES

29

1. It asymptotically generates F, that is,

Fn F.

(2.4)

2. If Gn is a sequence of atoms of Fn such that if Gn An and Gn+1 Gn , n = 0, 1, 2, . . ., then

Gn 6= .

(2.5)

n=1

A basis for a field is an asymptotically generating sequence of finite fields with the property that a
decreasing sequence of atoms cannot converge to the empty set. The property (2.4) of a sequence
of fields is called the finite intersection property. The name comes from the similar use in topology
and reflects the fact that if all finite intersections of the sequence of sets Gn are not empty (which
is true since the Gn are a decreasing sequence of nonempty sets), then the intersection of all of the
Gn cannot be empty.
A sequence Fn , n = 1, . . . is said to be a basis for a measurable space (, B) if the {Fn } form a
basis for a field that generates B.
If the sequence Fn F and F generates B, that is, if
B = (F) = (

Fn ),

n=1

then the Fn are said to asymptotically generate the -field B.

A field F is said to be standard if it possesses a basis. A measurable space (, B) is said to be
standard if B can be generated by a standard field, that is, if B possesses a basis.
The requirement that a -field be generated by the limit of a sequence of simple finite fields is a
reasonably intuitive one if we hope for the -field to inherit the extendibility properties of the finite
fields. The second condition that a decreasing sequence of atoms has a nonempty limit is less
intuitive, however, and will prove harder to demonstrate. Although nonintuitive at this point, we
shall see that the existence of a basis is a sufficient and necessary condition for extending arbitrary
finitely additive measures on countable fields to countably additive measures.
The proof that the standard property is sufficient to ensure that any finitely additive set function
is also countably additive requires additional machinery that will be developed later. The proof of
necessity, however, can be presented now and will perhaps provide some insight into the standard
property by showing what can go wrong in its absence.
Lemma 2.2.2 Let F be a field of subsets of a space . A necessary condition for F to have the
countable extension property is that it be standard, that is, that it possess a basis.
Proof: We assume that F does not possess a basis and we construct a finitely additive set function
that is not continuous at and hence not countably additive. To have the countable extension
property, F must be countable. From Lemma 2.2.1 we can construct a sequence of finite fields Fn
such that Fn F. Since F does not possess a basis, we know that for any such sequence Fn there
must exist a decreasing sequence of atoms Gn of Fn such that Gn . Define set functions Pn on
Fn as follows: If Gn F , then Pn (F ) = 1, if F Gn = , then Pn (F ) = 0. Since F Fn , F either
wholly contains the atom Gn or F and Gn are disjoint, hence the Pn are well defined. Next define
the set function P on the limit field F in the natural way: If F F, then F Fn for some smallest
value of n (e.g., if the Fn are constructed as before as the field generated by the first n elements of F,
then eventually every element of the countable field F must appear in one of the Fn ). Thus we can

CHAPTER 2. STANDARD ALPHABETS

30

set P (F ) = Pn (F ). By construction, if m n then also Pm (F ) = Pn (F ) and hence P (F ) = Pn (F )

for any n such that F Fn . P is obviously nonnegative, P () = 1 (since all of the atoms Gn in
the given sequence are in the sample space), and P is finitely additive. To see the latter fact, let Fi ,
i = 1, . . . , m be a finite collection of disjoint sets in F. By construction, all must lie in some field
Fn for sufficiently large n. If Gn lies in one of the sets Fi (it can lie in at most one since the sets
are disjoint), then (1.7) holds with both sides equal to one. If none of the sets contains Gn , then
(1.7) holds with both sides equal to zero. Thus P satisfies (1.4), (1.5), and (1.7). To prove P to be
countably additive, then, we must verify (1.8). By construction P (Gn ) = Pn (Gn ) = 1 for all n and
hence
lim P (Gn ) = 1 6= 0,
n

and therefore (1.8) is violated since by assumption the Gn decrease to the empty set which has
zero probability. Thus P is not continuous at and hence not countably additive.
If a field is not standard, then finitely additive probability measures that are not countably
additive can always be constructed by putting probability on a sequence of atoms that collapses down
to nothing. Thus there can always be probability on ever smaller sets, but the limit cannot support
the probability since it is empty. Thus the necessity of the standard property for the extension of
arbitrary additive probability measures justifies its candidacy for a general, useful alphabet.
Corollary 2.2.1 A necessary condition for a countably generated measurable space (, B) to have
the countable extension property is that it be a standard space.
Proof: To have the countable extension property, a measurable space must have a countable generating field. If the measurable space is not standard, then no such field can possess a basis and
hence no such field will possess the countable extension property. In particular, one can always find
as in the proof of the lemma a generating field and an additive set function on that field which is
not countably additive and hence does not extend.
2

Exercises
1. A class of subsets V of A is said to be separating if given any two points x, y A, there is a
V V that contains only one of the two points and not the other. Suppose that a separable field B has a countable generating class V = {Vi ; i = 1, 2, . . .} that is also separating. Describe
the intersection sets

\
Vn ,
n=1

where

2.3

Vn

is either Vn or

Vnc .

Some properties of standard spaces

The following results provide some useful properties of standard spaces. In particular, they show
how certain combinations of or mappings on standard spaces yield other standard spaces. These
results will prove useful for demonstrating that certain spaces are indeed standard. The first result
shows that if we form a product space from a countable number of standard spaces as in Section 1.4,
then the product space is also standard. Thus if the alphabet of a source or random process for one
sample time is standard, then the space of all sequences produced by the source is also standard.

2.3. SOME PROPERTIES OF STANDARD SPACES

31

Lemma 2.3.1 Let Fi , i I, be a family of standard fields for some countable index set I. Let F
be the product field generated by all rectangles of the form F = {xI : xi Fi , i M}, where Fi Fi
all i and M is any finite subset of I. That is,
F = F(rect(Fi , i I)),
then F is also standard.
Proof: Since I is countable, we may assume that I = {1, 2, . . .}. For each i I, Fi is standard and
hence possesses a basis, say {Fi (n), n = 1, 2, . . .}. Consider the sequence
Gn = F(rect(Fi (n), i = 1, 2, . . . , n)),

(2.6)

that is, Gn is the field of subsets formed by taking all rectangles formed from the nth order basis
fields Fi (n), i = 1, . . . , n in the first n coordinates. The lemma will be proved by showing that Gn
forms a basis for the field F and hence the field is standard. The fields Gn are clearly finite and
increasing since the coordinate fields are. The field generated by the union of all the fields Gn will
contain all of the rectangles in F since for each i the union in that coordinate contains the full
coordinate field Fi . Thus Gn F. Say we have a sequence Gn of atoms of Gn (Gn Gn for all n)
decreasing to the null set. Each such atom must have the form
Gn = {xI : xi Gn (i); i = 1, 2, . . . , n}.
where Gn (i) is an atom of the coordinate field Fi (n). For Gn , however, this requires that
Gn (i) at least for one i, violating the definition of a basis for the ith coordinate. Thus Fn must
be a basis.
2
Corollary 2.3.1 Let (Ai , Bi ), i I, be a family of standard spaces for some countable index set I.
Let (A, B) be the product measurable space, that is,
(A, B) = iI (Ai , Bi ) = (iI Ai , iI Bi ),
where

iI Ai = all xI = {(xi , i I) : xi Ai ; i I}

is the cartesian product of the alphabets Ai , and

iI Bi = (rect(Bi , i I)).
Then (A, B) is standard.
Proof: Since each (Ai , Bi ), i I, is standard, each possesses a basis, say {Fi (n); n = 1, 2, . . .}, which
asymptotically generates a coordinate field Fi which in turn generates Bi . (Note that these are not
the same as the Fi (n) of (2.6).) From Lemma 2.3.1 the product field of the nth order basis fields
given by (2.6) is a basis for F. Thus we will be done if we can show that F generates B. It is an
easy exercise, however, to show that if Fi generates Bi , all i I, then
B = (rect(Bi , i I)) =
(rect(Fi , i I)) = (F(rect(Fi , i I))) = (F).

(2.7)
2