MA40043: Real and Abstract Analysis1

E P Ryan Department of Mathematical Sciences University of Bath Bath BA2 7AY Email: masepr@bath.ac.uk URL: http://people.bath.ac.uk/masepr

Timetable Tuesday 12.15 in 4W1.7 Wednesday 12.15 in 8W2.23 Thursday 9.15 in 6E2.1

January 10, 2011

Unit Catalogue

www.bath.ac.uk/catalogues/2009-2010/ma/MA40043.htm

Preface
The aims of the course are: to ll in gaps left in the basic analysis courses (MA20007 and MA20011) - the main gap concerns uniform approximation of continuous functions by polynomials or of continuous periodic functions by trigonometric polynomials; to augment the treatment of metric spaces in MA30041; to introduce elements of functional analysis (which will be built on in MA40057) functional analysis is a standardized way of dealing with functions (identify the function space, establish whether it is a metric, normed, or inner product space, establish whether it is complete, and apply appropriate general theorems. There are numerous books in the library which, collectively, cover the material in the course. Below is a sample list. Kolmogorov, A N & Fomin, S V, Introductory Real Analysis, Dover. Kreyszig, E, Introductory Functional Analysis with Applications, Wiley. Protter, M H & Morrey, C B, A First Course in Real Analysis, Springer-Verlag. Simmons, G F, Introduction to Topology and Modern Analysis, McGraw-Hill. Stoll, M, Introduction to Real Analysis, Addison Wesley. Sutherland, W A, Introduction to Metric and Topological Spaces, OUP.

Chapter 1

Uniform continuity, uniform convergence and uniform approximation

1.1 Uniform continuity

Until further notice, D will denote a non-empty subset of R, not necessarily an interval. Denition 1.1. A function f : D R is continuous at a point d D if, for all > 0, there exists > 0 such that x D & |x d| < = |f (x) f (d)| < . Note that, in the above denition, depends on and d in general. The following is an equivalent sequential denition of continuity. Denition 1.2. A function f : D R is continuous at a point d D if, for any sequence (xn ) D such that xn d as n , we have f (xn ) f (d) as n . Denition 1.3. A function f : D R is continuous if it is continuous at every point of D. Denition 1.4. A function f : D R is uniformly continuous if, for all > 0, there exists > 0 such that x, y D & |x y| < = |f (x) f (y)| < . The crucial point to note here is that depends only on . It is clear that uniform continuity = continuity, but continuity = uniform continuity. Example. Dene f : R R, x f (x) := sin(x2 ). Then f is continuous but not uniformly continuous. [You are asked to prove this in Exercise Sheet 1, Question 1.] Theorem 1.1. If D is closed and bounded and f : D R is continuous, then f is uniformly continuous. Proof. Let D be closed and bounded and assume that f : D R is continuous. Seeking a contradiction, suppose that f is not uniformly continuous. Then there exist > 0 and sequences (xn ), (yn ) D such that |xn yn | < 1 n and |f (xn ) f (yn )| n N. 2

By boundedness of D and the Bolzano-Weierstrass Theorem1 , there exists a subsequence (xnk ) of the sequence (xn ) in D with xnk x as k . Clearly, we also have ynk x as k . By closedness of D, we may infer that x D. By continuity of f , f (xnk ) f (x) and f (ynk ) f (x) as k . Choose k suciently large so that |f (xnk ) f (x)| < /2 and |f (ynk ) f (x)| < /2. We now arrive at a contradiction |f (xnk ) f (ynk )| |f (xnk ) f (x)| + |f (ynk ) f (x)| < . Therefore, f is uniformly continuous. 2

1.2

Uniform convergence

Denition 1.5. A sequence of functions (fn : D R) converges to f : D R (i) pointwise if, for all x D and all > 0, there exists N N such that n>N = |fn (x) f (x)| < ,

(ii) uniformly if, for all > 0, there exists N N such that xD & n>N = |fn (x) f (x)| < .

Note that, in (ii), N depends only on . In (i), f is the pointwise limit of (fn ); in (ii), f is the uniform limit of (fn ). The above denition is equivalent to the following. Denition 1.6. A sequence of functions (fn : D R) converges to f : D R (i) pointwise if, for each xed x D, we have fn (x) f (x) as n , (ii) uniformly if sup |fn (x) f (x)| 0 as n .
xD

Clearly, uniform convergence = pointwise convergence, but pointwise convergence = uniform convergence. Example. Consider the sequence (fn ) of functions x fn (x) = xn dened on D = [0, 1]. The pointwise limit of this sequence is the (discontinuous) function f : D R given by f (x) := 0 1 if if x [0, 1), x = 1.

However, the convergence is not uniform because sup |fn (x) f (x)| = 1 for all n N.
xD

fn (x) 1 n=1 2 3 0 x 1 converges pointwise n

f (x) 1

x 1

1 Recall (from MA20007) the Bolzano-Weierstrass Theorem: every bounded sequence in R has a convergent subsequence.

1.2.1

Uniform Cauchy Principle

Recall (from MA20007) that a sequence (xn ) of real numbers is a Cauchy sequence if, for each > 0, there exists N N such that m, n > N = |xn xm | < .

The standard Cauchy Principle states that (xn ) is a Cauchy sequence if, and only if, it is convergent. Our next goal is to extend these ideas to sequences of functions.

Augustin Louis Cauchy (1789-1857)

Denition 1.7. A sequence of functions (fn : D R) is a uniform Cauchy sequence if, for each > 0, there exists N N such that xD & m, n > N = |fn (x) fm (x)| < .

Note that, in the above denition, N depends only on . The following is the Uniform Cauchy Principle. Theorem 1.2. A sequence of functions (fn : D R) is a uniform Cauchy sequence if, and only if, it is uniformly convergent. Proof. Necessity. Assume that (fn ) is a uniform Cauchy sequence of functions D R. Then, for each x D, (fn (x)) is a Cauchy sequence of real numbers and so, by the standard Cauchy Principle, is convergent: we denote its limit by f (x). Thus, with each x D, we may associate a unique real number f (x). This denes (pointwise) a function f : D R. Let > 0 be arbitrary. Since (fn ) is a uniform Cauchy sequence, there exists N N such that |fn (x) fm (x)| < /2 m, n > N, x D, whence |fn (x) f (x)| = lim |fn (x) fm (x)| /2 <
m

n > N, x D,

and so (fn ) is uniformly convergent. Suciency. Assume that (fn ) is uniformly convergent, with limit f . Let > 0 be arbitrary. Then, there exists N N such that |fn (x) f (x)| < /2 for all n > N and all x D. Therefore, |fn (x) fm (x)| |fn (x) f (x)| + |fm (x) f (x)| < and so (fn ) is a uniform Cauchy sequence. m, n > N, x D 2

Theorem 1.3. Let (fn ) be a uniformly convergent sequence of continuous functions D R with uniform limit f . Then f is continuous.

Proof. Let > 0 and d D be arbitrary. By uniform convergence of (fn ), we may choose n N suciently large so that |fn (x) f (x)| < /3 for all x D. By continuity of fn , there exists > 0 such that x D & |x d| < = |fn (x) fn (d)| < /3. Therefore, x D & |x d| < = + + = 3 3 3 2

|f (x) f (d)| |f (x) fn (x)| + |fn (x) fn (d)| + |fn (d) f (d)| < and so f is continuous at each d D.

Some notation. The space of continuous functions over the domain D is denoted by C(D). We remark that, equipped with the usual notions of addition and scalar multiplication, C(D) is a vector space. When D is an interval, we will use abbreviated notation of the sort C[a, b] := C([a, b]), C(a, b) := C((a, b)), C[a, b) := C([a, b)), etc.. The space C[a, b] (note the domain is compact, that is, closed and bounded) will be the focus of considerable attention. On this space, we can introduce the norm f := sup |f (x)| = max |f (x)|.
x[a,b] x[a,b]

(1.1)

Recall the axioms of a norm: f 0; f = 0 if, and only if, f = 0; f = || f R;

f +g f + g .

One can easily check that these axioms hold for (1.1). The concept of a (uniformly) convergent sequence (fn ) in C[a, b], with uniform limit f , can be characterized by the property fn f 0 as n , and Theorem 1.3 implies that the uniform limit f is in C[a, b]. From the latter observation, together with Theorem 1.2, we may infer that every (uniform) Cauchy sequence in C[a, b] is (uniformly) convergent to an element of C[a, b]: this is the notion of completeness of the space C[a, b]. Therefore, equipped with the norm (1.1), C[a, b] is a complete normed vector space: such a space is referred to as a Banach space. These abstract notions (normed space, Banach space, etc) will be revisited later in the course: they are mentioned here to give a feel of the abstract setting.

1.3
1.3.1

Uniform approximation
Approximation of continuous functions by polynomials

First consider the uniform approximation problem for continuous functions. Given a function f C[a, b] (with a < b), can it be nicely approximated by a nice function, say, a polynomial p P [a, b] (the vector space of polynomials with domain [a, b] and with real coecients)? There are two (equivalent) ways of capturing the concept of uniform approximation of f C[a, b] by polynomials: (i) for all > 0, there exists p P [a, b] such that f p < ; (ii) there exists a sequence (pn ) P [a, b] such that f pn 0 as n . 5

1.3.2

Approximation of continuous periodic functions by trigonometric polynomials

Next, consider the problem of approximation of continuous periodic functions by trigonometric polynomials. Denition 1.8. The space C2 is the vector space of continuous 2-periodic functions C2 := {f C(R)| f (t + 2) = f (t), t R} equipped with the norm f
C2

:= max |f (t)|.
t[,]

A trigonometric polynomial of degree n is a function of the form

n

t c0 +

(ak cos kt + bk sin kt).

k=1

Here n N0 := N {0}, the convention being that one drops the sum in the above formula for n = 0. The vector space of trigonometric polynomials on R is denoted by TP. Clearly, TP C2 . There are two (equivalent) ways of capturing the concept of uniform approximation of a function in f C2 by trigonometric polynomials: (i) for all > 0, there exists p TP such that f p
C2

< ;
C2

(ii) there exists a sequence (pn ) in TP such that f pn

0 as n .

1.3.3

Weierstrass Approximation Theorems

Karl Theodor Wilhelm Weierstrass (1815-1897)

Theorem 1.4. Every function f C[a, b] can be uniformly approximated by polynomials. Theorem 1.5. Every function f C2 can be uniformly approximated by trigonometric polynomials. An alternative way of stating Theorem 1.4 (respectively, Theorem 1.5) is P [a, b] is dense in C[a, b] (respectively, TP is dense in C2 ). The notion of denseness will be revisited in the next chapter. We will postpone the proofs of these theorems to later in the course: in particular, it will turn out that the two theorems are simple corollaries of a more general result known as the Stone-Weierstrass Theorem.

Chapter 2

Metric spaces
2.1 Basic concepts and denitions

Denition 2.1. A metric space (X, d) is a non-empty set X equipped with a function d : X X R satisfying the following axioms: d(x, y) 0 x, y X, with d(x, y) = 0 i x = y; d(x, y) = d(y, x) x, y X; d(x, y) d(x, z) + d(z, y) x, y, z X. Any such function d is called a metric on X. When the metric d is clear from context, we write X in place of (X, d). Notation. The function d : (x, y) |x y| denes a metric on the set of real numbers: henceforth, unless otherwise stated, the symbol R should be interpreted as the metric space (R, d). Denition 2.2. A real normed space is a real vector space X equipped with a function : X R satisfying the following axioms: x 0 x X, with x = 0 i x = 0; x = || x x X R; x+y x + y x, y X.

Notation. The function : x x2 + + x2 denes a norm on the space Rn of n-tuples n 1 (x1 , ..., xn ) = x of real numbers: henceforth, unless otherwise stated, the symbol Rn should be interpreted as normed space (Rn , ). Denition 2.3. Let X be a metric space with metric d. A sequence (xn ) in X is said to converge to x X if d(xn , x) 0 as n . Denition 2.4. Let X be a metric space with metric d. A sequence (xn ) in X is a Cauchy sequence if, for all > 0, there exists N N such that m, n > N = d(xm , xn ) < . Denition 2.5. A metric space X is complete if every Cauchy sequence in X converges to an element of X. Complete normed spaces are called Banach spaces. Simple examples of Banach spaces include R and C[a, b]. As we have already seen, the fact that the latter is complete follows from Theorems 1.2 and 1.3. In Section 2.1.2 below, we will generalize the latter. 7

A normed space is a special case of a metric space: use d(x, y) := x y as the metric.

Stefan Banach (1892-1945)

2.1.1

Uniform continuity and uniform convergence - revisited

Let X and Y be metric spaces, with respective metrics dX and dY . We wish to consider properties of functions or maps f : X Y . Denition 2.6. A function f : X Y is continuous at a point p X if, for all > 0, there exists > 0 such that x X, dX (x, p) < = dY (f (x), f (p)) < . equivalently, f : X Y is continuous at a point p X if, for any sequence (xn ) in X converging to p, the sequence (f (xn )) in Y is convergent to f (p) Y . A function f : X Y is continuous if it is continuous at every point of X. Denition 2.7. A function f : X Y is uniformly continuous if, for all > 0, there exists > 0 such that x1 , x2 X, dX (x1 , x2 ) < = dY (f (x1 ), f (x2 )) < . Denition 2.8. A sequence (fn ) of functions fn : X Y is uniformly convergent with limit f : X Y if, for all > 0, there exists N N such that x X & n > N = dY (fn (x), f (x)) < . The following is a generalization, to a metric space setting, of Theorem 1.3. Theorem 2.1. The limit of a uniformly convergent sequence of continuous functions X Y is continuous. You are asked to prove this in Question 3 of Exercise Sheet 2.

2.1.2

Let X be a metric space with metric d and consider the space C(X) of bounded continuous functions X R. Equipped with the obvious notions of addition (for f, g C(X), f + g C(X) is the function x f (x) + g(x)) and scalar multiplication (for f C(X) and R, f C(X) is the function x f (x)), C(X) is a real vector space. Dening the function : C(X) R by f := sup |f (x)|,
xX

The Banach space C(X) of bounded continuous functions X R

it is readily veried that is a norm on C(X) and, so equipped, C(X) is a normed space. To conclude that C(X) is a Banach space, it remains to show that, as a metric space with metric (f, g) f g , it is complete. This you are asked to do in Question 4 Exercise Sheet 2.

2.1.3

Denseness on the Baire category theorem

Let y be an element of the metric space X, and let r > 0. Then Br (y) := {x X| d(x, y) < r} is the open ball of radius r centred at y. A set Y X is said to be open if, for all y Y , there exists r > 0 such that Br (y) Y . A set Z X is said to be closed if its complement, Y = X \ Z, is open1 . Equivalently, a set Z X is said to be closed if it contains all its limit points (recall that x X is a limit point of Z if every open set containing x also contains some point z Z, z = x). The closure, Y , of a set Y X is the union of Y and all its limit points.

The interior of Y X is the union of all open sets contained in Y and is denoted by Y . Equiv

alently, Y is the set of all interior points of Y (a point y is an interior point of Y if there exists > 0 such that B (y) Y ). Denition 2.9. Let X be a metric space, and let Y X. Then Y is (everywhere) dense in X if, for all x X and all > 0, there exists y Y such that d(x, y) < .

In other words, Y X is dense in X if every open ball (no matter how small) in X contains at least one point of Y . The prototype example is the set Q of rational numbers which forms a dense subset of the reals R. Lemma 2.1. Let X be a metric space (with metric d), and let Y X. The following statements are equivalent: (a) Y is (everywhere) dense in X; (b) for all x X, there exists a sequence (yn ) in Y such that yn x as n ; (c) Y = X. Proof. (a) = (b): Assume that Y is (everywhere) dense in X. Let x X be arbitrary. By denseness of Y , for each n N there exists yn Y such that d(x, yn ) < 1/n. The sequence (yn ) in Y , so dened, is such that yn x as n . Therefore, (b) holds. (b) = (c): Assume that (b) holds. Let x X be arbitrary. To prove that (c) holds, it suces to show that x Y . By (b), there exists a sequence (yn ) in Y such that yn x as n . If the sequence has only nitely many distinct terms, then x = yn for some n and so x Y Y . If the sequence has innitely many distinct terms, then x is a limit point of the set {yn | n N} and so is a limit point of the set Y : therefore, x Y . (c) = (a): Assume that Y = X. Then, for all x X and > 0, there exists y Y such that d(x, y) < and so (a) holds. 2

Denition 2.10. Let X be a metric space. A set Y X is nowhere dense if Y = . Exercise: show that a set Y is nowhere dense in the metric space X if, and only if, X \ Y is dense. We now proceed to introduce an important theorem, the Baire Category Theorem, relating to complete metric spaces. This theorem underpins three of the cornerstones of functional analysis: the uniform boundedness principle, the open mapping theorem and the closed graph theorem. Theorem 2.2. Let X be a complete metric space. If (Gn ) is a sequence of dense open subsets of X, then Gn is dense in X.
n=1
1 Note that the notions of open and closed are well dened without the assumption of the metric space being complete. Note also that the empty set and the whole space X are assumed to be open and closed simultaneously; this is a standard convention.

Ren-Louis Baire (1874-1932) e

Proof of Theorem 2.2 Let x0 X and 0 > 0 be arbitrary. It suces to show that B0 (x0 )
n=1

Gn = .

(2.1)

As G1 is dense, there exists x1 G1 B0 (x0 ). Note that G1 B0 (x0 ) is open (the intersection of a nite number of open sets is open). Therefore, there exists 1 > 0 such that B21 (x1 ) G1 B0 (x0 ) 0 and 1 < 2 . As G2 is dense, there exists x2 G2 B1 (x1 ). The set G2 B1 (x1 ) is open and so there exists 1 2 > 0 such that B22 (x2 ) G2 B1 (x1 ) and 2 < 2 . Proceeding inductively, we obtain sequences (xn ) in X and (n ) in (0, ) such that B2n (xn ) Gn Bn1 (xn1 ) and n < n1 . 2 We now have B2n (xn ) Bn1 (xn1 ) B2n1 (xn1 ) Bn2 (xn2 ) . . . B21 (x1 ) B0 (x0 ), and so m, n N, m > n This implies, in particular, that m, n N, m > n = d(xm , xn ) < n < 0 . 2n (2.2) = B2m (xm ) Bn (xn ) B0 (x0 ).

Therefore, (xn ) is a Cauchy sequence and, by completeness of X, it follows that (xn ) converges to some x X. By the triangle inequality, we have d(x, xn ) d(x, xm ) + d(xm , xn ) m, n N. Now x n and let m (and so d(x, xm ) 0 as m ). By (2.2) and (2.3), we have d(x, xn ) n . Consequently, x B2n (xn ) Gn Bn1 (xn1 ) Gn B0 (x0 ). As n is arbitrary, we may conclude that x (Gn B0 (x0 )) = B0 (x0 )
n=1 n=1

(2.3)

Gn 2

which establishes (2.1).

Intuitively, a nowhere dense subset of a metric space may be thought of as a set that does not contain very much of the space. The following reformulation of the Baire Category Theorem asserts that a complete metric space cannot be covered by a sequence of such sets. 10

Theorem 2.3. The complement of a countable union of nowhere dense subsets of a complete metric space is dense. A complete metric space is not a countable union of nowhere dense subsets. Proof Let (Yn ) be a sequence of nowhere dense subsets of a complete metric space X and, for each n, dene Gn := X \ Y n . For each n, the set Gn is dense and open. Moreover, we have X \ Yn X \ Y n =
n=1 n=1 n=1 n=1

X \ Y n = Gn .
n=1 n=1

But, by Theorem 2.2, Gn is dense. Therefore, X\ Yn contains a dense set and so is itself dense which, in turn, implies that Yn = X.
n=1

The title of Baires theorem arises from his terminology for sets. Denition 2.11. A set Y X is of rst category or meagre in X if it is a countable union of nowhere dense sets, otherwise, Y is of second category. Thus, Baires theorem asserts the following. Corollary 2.1. A complete metric space is of second category in itself. Corollary 2.2. R is uncountable. Proof. With metric d(x, y) = |x y|, R is a complete metric space. For any x R, the set {x} is closed and has empty interior. Thus, if R were countable, it would be a countable union of nowhere dense sets which is impossible in view of Theorem 2.3. 2 The following result asserts not only that there exist functions f C[a, b] that are nowhere dierentiable on (a, b) but also that such functions are dense in C[a, b]. Corollary 2.3. Let L := {f C[a, b] | there exists t (a, b) such that f is dierentiable at t}. Then the set C[a, b]\L of continuous functions on [a, b] that are nowhere dierentiable on (a, b) is dense in C[a, b]. Proof. For each N N, dene LN := {f C[a, b] | t [a, b] s.t. |f (s) f (t)| N |s t|, s [a, b]}. Intuitive picture: for all s [a, b], the modulus of the slope of the line connecting (t, f (t)) and (s, f (s)) is less than or equal to N . (t, f (t)) modulus of slope less than N (b, f (b)) (a, f (a)) (s, f (s))

Our strategy is as follows: We show that LN is closed for each N N. We then show that C[a, b]\LN is dense, and so LN is nowhere dense. Therefore, N LN is meagre. By Theorem 2.3, C[a, b]\ N LN is dense. 11

 Finally, we show that L N LN whence C[a, b]\L C[a, b]\ N LN and so C[a, b]\L is dense. First, we will prove that, for all N , LN is closed. Let N N and let (fn ) be a sequence in LN be such that fn f C[a, b] uniformly on [a, b]. We will show that f LN and so LN is a closed set. There exists a sequence (tn ) in [a, b] such that |fn (s) fn (tn )| N |s tn |, s [a, b].

By the BolzanoWeierstrass Theorem, there exists a subsequence (tnk ) such that tnk t [a, b] as k . Therefore, |f (s) f (t)| |f (s) fnk (s)| + |fnk (s) fnk (tnk )| + |fnk (tnk ) fnk (t)| + |fnk (t) f (t)| |f (s) fnk (s)| + N |s tnk | + N |tnk t| + |fnk (t) f (t)| |f (s) f (t)| N |s t|. Thus, f LN and so LN is closed. Next, we prove that C[a, b] \ LN is dense (and so LN is nowhere dense) for all N N. Let N N, > 0 and f C[a, b] be arbitrary. We will show that there exists g C[a, b] \ LN such that f g < . Since P [a, b] is dense in C[a, b], there exists p P [a, b] such that f p < /2. To complete the argument, it suces to show that p g < /2 for some g C[a, b] \ LN . With reference to the gure below, let g be any continuous piecewise linear function [a, b] R such that (a) its graph lies between the (smooth) graphs of the functions p() + (/2) and p() (/2), and (b) the slopes of the linear segments of the graph are greater in modulus than N . Every such g has the requisite properties: g C[a, b] \ LN and p g < /2. g() p() + (/2) p() p() (/2) Fixing s [a, b] arbitrarily and letting k we have

s [a, b].

We briey digress to give an explicit construction of one such g. Clearly, p is continuously dierentiable with derivative p . Choose M N such that M > p + N. 2(b a) Dene := (b a)/M and ti := a + i, i = 0, 1, ..., M . Now dene the continuous, piecewise linear function g : [a, b] R by the property that, for each i = 0, 1, ..., M , g(t) = 1 p(ti ) + (1)i /4 (ti+1 t) + p(ti+1 ) + (1)i+1 /4 (t ti ) t [ti , ti+1 ].

Then, p g = /4 and, for each i = 0, 1, ..., M , the modulus of the slope Si of the linear segment on [ti , ti+1 ] satises |Si | = 1 |p(ti+1 ) p(ti )| M |p(ti+1 ) p(ti ) + (1)i+1 /2| p > N. 2 2(b a)

In particular, we have g C[a, b] \ LN with p g < /2 as required. 12

Next, we show that L LN . Assume f L, and so f (t) exists for some t (a, b). Then there
N N

Returning to the main argument, we may now infer that, for each N N, C[a, b]\LN is dense and so LN is nowhere dense. By Theorem 2.3, it follows that C[a, b]\ N N LN is dense.

exists > 0 such that

0 < |s t| < Also, we have |s t| Therefore, s [a, b] Now, choose &

& s [a, b]

f (s) f (t) < |f (t)| + 1 . st

s [a, b]

f (s) f (t) |f (s) f (t)| 2 f . st |f (t)| + 1 + 2 f 2 f , |s t|.

|f (s) f (t)| with N

N N in which case we have

|f (t)| + 1 +

|f (s) f (t)| N |s t|,

s [a, b].
N N

Thus, every f L is also in LN for some N N. Therefore, L LN and so C[a, b]\L C[a, b]\ N N LN . Therefore, C[a, b]\L contains a dense set and so is itself dense. 2

2.2

Separable spaces

Loosely speaking, a metric space with the useful property that it has an at most countable subset from within which every element of the space can be approached is said to be separable (the prototypical example being the approximation of real numbers by rationals). More precisely: Denition 2.12. A metric space is separable if it contains an at most countable dense set. (Recall: a set S countable if there exists a bijection S N.) Example 2.1. R is separable because it contains the dense countable subset Q. Example 2.2. Polynomials with rational coecients are dense in C[a, b] and are countable (see Question 5, Exercise Sheet 3). This implies that C[a, b] is separable. Denition 2.13. Let X be a metric space, and let Y X. A collection {Z , I} of sets (respectively, open sets) in X such that Y I Z is said to be a cover (respectively, an open cover) for Y . Theorem 2.4. Let X = (X, d) be a separable metric space. Every open cover of X has a countable subcover. Proof. Since X is separable, it has an at most countable dense subset S. Let B denote the collection of open balls with centres in S and rational radii, that is, B := {U | U = Bq (s), s S, q Q, q > 0}. This collection is countable. Claim. For every open ball Br (x) in X, there is an open ball Ux in B such that x Ux Br (x). 13

To establish this claim, we argue as follows. Let Br (x) be an open ball in X. Since S is dense in X, there exists s S such that d(x, s) < r/4. Now choose q Q such that r/4 < q < r/2. It follows that x Bq (s). Moreover, if y Bq (s), then d(x, y) d(x, s) + d(s, y) < r r r +q < + <r 4 4 2

and so Bq (s) Br (x). Setting Ux := Bq (s) establishes the claim. We now return to the proof of the theorem. Let {Z , I} be an open cover of X. Let x Y be arbitrary. Then, x Z for some I and, by the claim, we may infer that x Ux Z for some Ux B. Since B is countable, it follows that the set {Ux | x X} B is also countable, and so there exist a sequences (xn ) in X and (n ) in I such that {Ux | x X} = {Uxn | n N} B Therefore,
yY

and Uxn Zn n N.

Y Uy = Uyn Zn .
n=1 n=1

This completes the proof.

2.3
2.3.1

Compactness
Compact sets

Denition 2.14. A set S in a metric space is said to be compact if every open cover of S contains a nite subcover. Denition 2.15. A set S in a metric space is said to be sequentially compact if every sequence in S has a subsequence convergent to an element of S. Theorem 2.5. A set in a metric space is compact if, and only if, it is sequentially compact. Proof. In the course Metric Spaces. Lemma 2.2. A compact set in a metric space is bounded and closed. Proof. Let S be a compact subset of a metric space (X, d). Let x be an element of the closure S of S. Then there is a sequence (xn ) in S such that d(xn , x) 0 as n . By compactness of S, it follows that x S and so S is closed. For contradiction, suppose that S is unbounded. Then there exists a sequence (xn ) in S such that d(xn , x) > n for all n N, where x is some xed element of S. Clearly, every subsequence of (xn ) is unbounded and so cannot be convergent, contradicting compactness of S. Therefore, S is bounded. 2 Remark 2.1. The converse of the above result does not hold: a bounded and closed subset of a metric space is not necessarily compact. For example, consider the metric space X of square-summable real sequences x = (xn ) (with
2 nN xn

sequence which has m-th term 1, all other terms being 0 (and so d(x(m) , 0) = 1 and d(x(m) , x(n) ) = 2 for all m, n N, n = m). Then S = (x(m) ) is a bounded subset of X; S is also closed since it has no limit points and, for the same reason, S is not compact. We proceed to identify a class of metric spaces for which the converse does hold. Lemma 2.3. Let X be a metric space with the property that every bounded sequence in X has a convergent subsequence. A set in X is compact if, and only if, it is bounded and closed.

< ), with metric d(x, y) :=

nN (xn

yn )2 . For each m N, let x(m) denote the

14

Proof. Necessity. This follows by Lemma 2.2. Suciency. Let S be a closed and bounded set in X. Let (xn ) be a sequence in S. Since S is bounded, the sequence (xn ) is bounded and so, by hypothesis, has a convergent subsequence (xnk ); we denote the limit of the subsequence by x. Since S is closed, we have x S. Therefore, S is sequentially compact and so, by Theorem 2.5, is compact. 2 We know that, by Bolzano-Weierstrass, the metric space X = Rn has the property that every bounded sequence in X has a convergent subsequence. Therefore, we have the following. Corollary 2.4. A set in Rn is compact if, and only if, it is bounded and closed.

2.3.2

Relatively compact sets

Denition 2.16. A set S in a metric space is said to be relatively compact if its closure S is compact. Lemma 2.4. In a metric space a set S is relatively compact if, and only if, every sequence in S contains a convergent subsequence. [The limit of this subsequence may not be in S.] Proof See Exercise Sheet 5. 2

Denition 2.17. A set S in a metric space is said to be totally bounded if, for each > 0, it can be covered by nitely many balls of radius . These balls are said to form a nite -cover of S, and their centres to form a nite -net for S. It is clear that total boundedness but boundedness = total boundedness. To conrm the latter, consider again the metric space X of square-summable real sequences x = (xn ) with metric d(x, y) :=
nN (m)

boundedness

(xn yn )2 .

For each m N, let x denote the sequence which has m-th term 1, all other terms being 0 (and so d(x(m) , 0) = 1). Then S = (x(m) ) is a bounded subset of X: however, it is not totally bounded since d(x(m) , x(n) ) = 2 for n = m and so any ball of radius = 1/ 2 cannot contain more than one element of the sequence. A nite-dimensional setting can simplify the situation: it is easy to see that a set in the Euclidean space Rn is bounded if, and only if, it is totally bounded. Theorem 2.6. (i) Any totally bounded subset of a metric space is separable (as a metric space in its own right). (ii) Any relatively compact subset of a metric space is totally bounded. (iii) Any totally bounded subset of a complete metric space is relatively compact. Proof. (i) Let S be a totally bounded subset of a metric space (X, d). Then, for each m N, S has a nite 2m -net which we denote by {cm,1 , . . . , cm,Km }: we may assume that B2m (cm,n ) S = for n = 1, ..., Km . For each m N and each n {1, . . . , Km }, choose an element sm,n B2m (cm,n ) S. Clearly, the set = sm,n | m N, n {1, ..., KM } of all these elements is countable. Moreover, is dense in S: to see this, let > 0 and s S be arbitrary; choose m N such that 2m < /2; s B2m (cm,n ) for some n {1, ..., Km } and so d(s, sm,n ) < . 15

(ii) Let S be a relatively compact subset of a metric space (X, d). Let > 0 be arbitrary. We will construct a nite -net for S. If S = , then there is nothing to prove. Assume S = and let x1 S be arbitrary. If x B (x1 ) for all x S, then {x1 } is an -net for S and we are done: otherwise, proceed to step 2, that is, choose x2 S such that d(x1 , x2 ) . If x B (x1 ) B (x2 ) for all x S, then {x1 , x2 } is an -net for S: otherwise, proceed to step 3, that is, choose x3 S such that d(x2 , x3 ) and d(x1 , x3 ) . If x j=1,2,3 B (xj ) for all x S, then {x1 , x2 , x3 } is an -net for S: otherwise proceed to the next step. After nitely many such steps we must arrive at an -net for S: otherwise, the construction would yield a sequence (xn ) in S with the property that d(xn , xm ) for all n, m N with n = m; this sequence cannot have a convergent subsequence, which contradicts relative compactness of S. (iii) Let S be a totally bounded subset of a complete metric space (X, d). Let (xn ) be a sequence in S. By total boundedness of S, it has a nite 1-net and so there exists an element c1 of the 1-net such that the ball B1 (c1 ) contains a subsequence (xk1 (n) ) of (xn ). Similarly, S has a nite (1/2)-net and there exists an element c2 thereof such that the ball B1/2 (c2 ) contains a subsequence (xk2 (n) ) of (xk1 (n) ) and so (xk2 (n) ) is contained in 2 B1/j (cj ). Proceeding inductively, we generate a j=1 sequence (cm ) and a sequence of subsequences (xn ) (xk1 (n) ) (xk2 (n) ) (xkm (n) ) with the property that, for all m N, (xkm (n) ) is a sequence in m B1/j (cj ). Dene the sequence j=1 (ym ) = (xkm (m) ) = (xk1 (1) , xk2 (2) , xk3 (3) , ...). Let > 0 be arbitrary and x N N such that 1/N < /2. Then ym = xkm (m) B1/N (cN ) B/2 (cN ) m > N and so d(ym , yn ) < for all n, m > N . Therefore, (ym ) S is a Cauchy sequence in the complete metric space X. We may now conclude that (ym ) is a convergent subsequence of the sequence (xn ) in S. Therefore, S is relatively compact. 2

2.3.3

Arzel`-Ascoli theorem a

Cesare Arzel` (1847-1912) a (Giulio Ascoli (1843-1896): portrait unavailable)

Denition 2.18. Let (X, d) be a metric space and let D X. A set S of functions f : D R is equicontinuous if, for all > 0, there exists > 0 such that x, y D & d(x, y) < & f S = |f (x) f (y)| < .

Of course, if the set S contains only one function the above denition coincides with the denition of uniform continuity. Denote by C(D) the Banach space of bounded continuous functions f : D R with f supxD |f (x)|. 16 :=

Theorem 2.7. Let D be a compact subset of a metric space (X, d). A set of functions S C(D) is relatively compact if, and only if, it is bounded (that is, supf S f < ) and equicontinuous. Proof of Theorem 2.7 Necessity . Let S be relatively compact. Then, by Theorem 2.6, S is totally bounded and, hence, is bounded. It remains to prove equicontinuity of S. Let > 0 be arbitrary. By total boundedness of S, there exists a nite (/3)-net for S, which we denote by {g1 , ..., gN }. By compactness of D, each gn C(D) is uniformly continuous (Question 1 on Exercise Sheet 5). It follows that there exists > 0 such that x, y D & d(x, y) < x, y D & d(x, y) < = + + = . 3 3 3 = |gn (x) gn (y)| < /3 for n = 1, ..., N .

Let f S be arbitrary. Then there exists n {1, ..., N } such that f gn < /3 and so |f (x) f (y)| |f (x) gn (x)| + |gn (x) gn (y)| + |gn (y) f (y)| < Therefore, S is equicontinuous. Suciency. Let S be bounded and equicontinuous, and let (fn ) be any sequence in S. We seek a Cauchy subsequence. Let (xp ) be a countable dense set in D. Such a set exists since, by Theorem 2.6, D compact = D totally bounded = D separable. The sequence (fn (x1 )) is bounded in R and, hence, has a convergent subsequence, which we denote by (fk1 (n) (x1 )). Similarly, (fk1 (n) (x2 )) is a bounded sequence in R and, hence, has a convergent subsequence (fk2 (n) (x2 )). Note that (fk2 (x)) converges for x = x1 and x = x2 . Proceeding inductively, we generate a sequence of subsequences of (fn ): (fn ) (fk1 (n) ) (fk2 (n) ) (fkm (n) ) with the property that, for each m N, (fkm (n) (x)) converges at x = x1 , . . . , xm . Extract the diagonal sequence (gm ) := (fkm (m) ) = (fk1 (1) , fk2 (2) , ...). Then, for each p N, (gm ) is a subsequence of (fkp (n) ) and so (gm (xp )) converges for each p N. m=p Since (gm ) is a subsequence of the sequence (fn ) in the Banach space C(D), the proof is complete if we can show that (gm ) is a Cauchy sequence: this we proceed to do. Let > 0 be arbitrary. By equicontinuity of S, there exists > 0 such that p N & x D & d(x, xp ) < = |gm (x) gm (xp )| < m N. (2.4) 4 By compactness of D, there exists a nite (/2)-cover: D J B/2 (cj ) with D B/2 (cj ) = , j=1 j = 1, ..., J. For j = 1, ..., J, choose xpj (xp ) such that xpj B/2 (cj ) (such a choice is possible because (xp ) is dense in D). Recalling that (gm (xp )) converges for each p N, we may infer the existence of N N such that m, n N m, n > N = |gm (xpj ) gn (xpj )| < 4 for j = 1, ..., J. (2.5)

Let x D be arbitrary. Then, for some j {1, ..., J}, we have d(x, xpj ) < which, together with (2.4) and (2.5), gives |gm (x) gn (x)| |gm (x) gm (xpj )| + |gm (xpj ) gn (xpj )| + |gn (xpj ) gn (x)| 3 < + + = 4 4 4 4 17

m, n > N .

Thus, we have shown that, for each > 0, there exists N N such that m, n > N = |gm (x) gn (x)| <

3 3 x D = gm gn < . 4 4 Therefore, (gm ) is a Cauchy sequence in the Banach space C(D) and is a subsequence of (fn ). Therefore, S is relatively compact. 2

2.3.4

The Stone-Weierstrass theorem

We are now in a position to prove an elegant and powerful generalization of the classical Weierstrass approximation theorem (Theorem 1.4), due to Marshall H Stone.

Marshall Harvey Stone (1903-1989) Let D be a non-empty compact subset of a metric space X = (X, d). We regard D as a metric space in its own right (and so a set Y D is open if, for each y Y , there exists r > 0 such that the set {x D| d(x, y) < r} is contained in Y ). Consider the space C(D) of continuous functions D R. By continuity of f and compactness of D, it follows that f is bounded and so, equipped with the uniform norm ( f := supxD |f (x)|), C(D) is a Banach space. Let S be a non-empty subset of C(D). Recall the following equivalent denitions of uniform approximation of functions in C(D) by functions in S. Denition 2.19 (Uniform Approximation). Functions in C(D) are said to be uniformly approximated by functions in S if, for each each f C(D) and all > 0, there exists f0 S such that f0 B (f ). The following is readily veried. Proposition 2.1. The following statements are equivalent: 1. functions in C(D) are uniformly approximated by functions in S; 2. for each f C(D), there exists a sequence (fn ) in S such that f fn 0 as n ; 3. S is dense in C(D); 4. the closure of S is C(D). Lattices and the Stone Approximation Theorem For (f, g) C(D) C(D), dene the functions f g and f g by Observe that, for f, g C(D), the map D R2 , x (f (x), g(x)) is continuous; moreover, the maps R2 R, (a, b) max{a, b} and R2 R, (a, b) min{a, b} are also continuous. Therefore, the functions f g and f g are continuous, that is, f g C(D) and f g C(D). We refer to and as the lattice operations. We record that,as a map C(D) C(D) C(D), each lattice operation is continuous (you are asked to prove this in Exercise Sheet 6). A set S C(D) is said to be a lattice if it is closed under the lattice operations, that is, f, g S = f g, f g S. 18 (f g)(x) := min{f (x), g(x)}, (f g)(x) := max{f (x), g(x)}.

Remark 2.2. Since the lattice operations are continuous, it follows that, if S is a lattice, then so is its closure S. Theorem 2.8. Let S C(D) be a lattice. Then a function f C(D) is in the closure S of S if, and only if, for all x, y D and > 0, there exists a function fx,y S such that |f (x) fx,y (x)| < , & |f (y) fx,y (y)| < . Proof. Necessity of the stated condition is obvious. We proceed to prove suciency. Let f C(D) and > 0 be arbitrary. We will construct a function f0 S such that f f0 < . For each pair of points x, y D, there exists fx,y S such that |f (x) fx,y (x)| < and |f (y) fx,y (y)| < . For each xed x D, Nx (y) := {z D| fx,y (z) > f (z) } is an open set containing y. The family of these sets Nx (y)| y D covers D and so, by compactness of D, has a nite subcover which we denote by Nx (yk (x))| k = 1, ..., K . Dene hx :=
K k=1 fx,yk (x) ,

z hx (z) = max{fx,y1 (x) (z), ..., fx,yK (x) (z)}.

Since S is a lattice and fx,y S for all y D, it follows that hx S. Observe that, for each z D, there exists k {1, ..., K} such that z Nx (yk (x)) and so hx (z) fx,yk (x) (z) > f (z) . Therefore, hx (z) > f (z) z D. Moreover, since |f (x) fx,yk (x) (x)| < , k = 1, ..., K, we have fx,yk (x) (x) < f (x) + , k = 1, ..., K. Therefore, hx (x) < f (x) + . We have now shown that x D hx S : hx (z) > f (z) z D & hx (x) < f (x) + . (2.6)

For each x D, M (x) := {z D| hx (z) < f (z) + } is an open set containing x. The family of these sets M (x)| x D covers D and so has a nite subcover, which we denote by M (xj )| j = 1, ..., J . Dene f0 :=
J j=1 hxj ,

z f0 (z) = min{hx1 (z), ..., hxJ (z)}.

Since S is a lattice and hxj S, j = 1, ..., J, we have f0 S. Let z D. Then z M (xj ) for some j {1, ..., J} and so f0 (z) hxj (z) < f (z) + . By (2.6), we also have f0 (z) > f (z) and so we may conclude that |f (z) f0 (z)| < z D, whence f f0 < . 2 19

A set S C(D) is said to separate points in D if x, y D, x = y = f S : f (x) = f (y). A set S C(D) is said to separate points in D and R if a, b R, x, y D, x = y = f S : f (x) = a, f (y) = b. An immediate consequence of the above theorem is the Stone Approximation Theorem. Theorem 2.9 (Stone Approximation Theorem). Let S C(D) be a lattice that separates points in D and R. Then S is dense in C(D). Proof. Let f C(D) be arbitrary. For each pair of points x, y D, there exists fx,y S such that fx,y (x) = f (x) and fx,y (y) = f (y) (including the possibility that x = y). By Theorem 2.8, it follows that f S. Therefore, S = C(D). 2 Algebras and the Stone-Weierstrass Theorem A set S C(D) is said to be an algebra if S is a real vector subspace of C(D) and f, g S = f g S. Example 2.3. Let D = I = [a, b] be a compact interval in R. Then the space of polynomial functions S = P (I) C(I) is an algebra and S separates points in I and R. Remark 2.3. Since the algebra operation (that is, (f, g) f g) is continuous (as a map C(D) C(D) C(D)), it follows that, if S is an algebra, then so is its closure S; moreover, if S separates points in D and R, then so does S. Theorem 2.10 (Stone-Weierstrass). Let S C(D) be an algebra that separates points in D and assume that the constant function x 1 is in S. Then S is dense in C(D). We preface the proof of the Stone-Weierstrass Theorem with a proposition and a lemma, which contribute to the proof. Proposition 2.2. For n N, dene un C[1, 1] by 1) ( 1 (n 1)) 2 . n n n! 1 + t. n=1 un converges uniformly to the function [1, 1] R, t un (t) :=
1 2

tn ,

where

1 2

:=

1 1 2(2

The series 1 +

Proof. See Exercise Sheet 6. Lemma 2.5. Let a > 0 and write I := [a, a]. The absolute value function v C(I) given by v(t) := |t| can by uniformly approximated by polynomials from P (I) (the space of polynomial functions on I with real coecients). Proof. Let > 0 be arbitrary. We will show that there exists a polynomial p P (I) such that |t| p(t) < for all t I. By Proposition 2.2, we know that there exists N N such that the polynomial q P [1, 1], given by
N

q(s) = 1 +
n=1

1 2

sn ,

has the property that

1 + s q(s) < s [1, 1]. a 20

Let p P (I) be the polynomial given by p(t) = a q((t/a)2 1). Let t I be arbitrary and write s := (t/a)2 1 [1, 1]. Then |t| p(t) = a 1 + (t/a)2 1 a q((t/a)2 1) = a 1 + s q(s) < . This completes the proof. We are now in a position to prove the Stone-Weierstrass Theorem. Proof of the Stone-Weierstrass Theorem. First, we show that S separates points in D and R. Let a, b R and x, y D with x = y. Since S separates points in D, there exists f S such that f (x) = f (y). We seek , R such that the function g = 1 + f S (where 1 denotes the constant function z 1) has the requisite properties g(x) = a and g(y) = b. In particular, we seek , R such that + f (x) = a and + f (y) = b which, since f (x) f (y) = 0, has unique solution bf (x) af (y) ab = , = . f (x) f (y) f (x) f (y) Therefore, S separates points in D and R. Recalling Remark 2.3, the closure S of the algebra S is also an algebra and separates points in D and R. If it can also be established that S is a lattice, then an application of Theorem 2.9 completes the proof. We proceed to show that S is a lattice. We claim that f S = |f | S, (2.7) where |f | denotes the function x |f (x)|. If f = 0 (the zero function), then the claim is evidently true. Assume f = 0, dene a := f > 0 and I := [a, a]. Let > 0 be arbitrary. By Lemma 2.5, there is a polynomial p P (I) such that |t| p(t) < for all t I. Therefore, |(p f )(x) |f (x)| < x D and, since S is an algebra, p f S. We have now shown that, for every > 0, there exists f0 S such that f0 |f | < and so, since S is closed, it follows that |f | S. Noting that 1 1 (f + g) |f g| , f g = (f + g) + |f g| , 2 2 we may now conclude that f, g S = f g, f g S f g = 2 2

and so S is a lattice. By Theorem 2.9, S is dense in C(D) and so S = C(D).

2.3.5

The classical Weierstrass approximation theorems

In this section, we show that the two classical approximation theorems (on approximation of continuous functions by polynomials and approximation of continuous periodic functions by trigonometric polynomials), due to Weierstrass, are simple consequences of the Stone-Weierstrass Theorem. Theorem 2.11. Let I = [a, b] be a compact interval in R. The space S = P (I) of polynomial functions on I with real coecients is dense in C(D) = C(I). Proof. Clearly, p, q S = pq S and so S is an algebra. The constant function x 1 is in S. Moreover, the identity function x x is in S and so S separates points. By the Stone-Weierstrass Theorem, we conclude that S is dense in C(D). 2 The proof of the above result applies, mutatis mutandis, to establish the following. Theorem 2.12. Let D be a compact subset of Rn . The space S = P (D) of polynomial functions on D with real coecients is dense in C(D). 21

Let C2 denote the Banach space of continuous 2-periodic functions R R: C2 := {f C(R)| f (t + 2) = f (t) t R} equipped with the norm f := maxt[,] |f (t)| = maxt(,] |f (t)|. Let T P C2 denote the space trigonometric polynomials, that is, the space of functions of the form t a0 + n k=1 ak cos kt + bk sin kt , with real coecients a0 , ..., an , b1 , ..., bn . Theorem 2.13. The space T P is dense in C2 . Proof. Let f C2 and > 0 be arbitrary. We will establish the existence of f0 T P such that f f0 < . Let D be the unit circle in R2 , that is, D = {x = (x1 , x2 ) R2 | x2 + x2 = 1}. Let 1 2 S C(D) be the space of polynomial functions on D with real coecients. By Theorem 2.12, S is dense in C(D). Let h denote the continuous bijection (, ] D, t (cos t , sin t) and dene C(D) by (x) = (x1 , x2 ) := f (h1 (x1 , x2 )) = (f h1 )(x). Observe that :=
(x1 ,x2 )D

max

|(x1 , x2 )| = max |f (t)|

t(,]

and so f = (of course, the former is the norm on C2 and the latter is the uniform norm on C(D)). Since S is dense in C(D), there exists 0 S such that 0 < . Now dene f0 C(, ] by f0 (t) := 0 (cos t , sin t) = ( h)(t) and extend this function to f0 C2 by imposing periodicity: f0 (t + 2) = f0 (t) for all t R. Since 0 is a polynomial function, it follows that f0 = h T P : moreover, f f0 = 0 < . This completes the proof. 2

2.4

Completion of metric spaces

Can a metric space fail to be complete? Example 2.4. (Q, d), with metric d(p, q) = |p q|, is incomplete. To see this, simply note that, for each p Q, the set {p} is closed and has empty interior and so, since Q is countable, it is a countable union of nowhere dense sets. By Theorem 2.3, it follows that Q cannot be complete. Example 2.5. The space preLp [a, b] is incomplete. For p 1, preLp [a, b] denotes the space C[a, b] equipped with the norm
b 1/p

given by

:=
a

|f (x)| dx

To see that this space is incomplete, we construct a Cauchy sequence (fn ) in preLp [1, 1] which fails to converge to an element of preLp [1, 1]. This is done in Exercise Sheet 4, Question 2. Our next objective is to show that every incomplete metric space has a completion. Denition 2.20. A map : (X, dX ) (Y, dY ) is an isometry if dY ((x1 ), (x2 )) = dX (x1 , x2 ), It is clear that an isometry is injective and continuous. If there exists a bijective isometry : X Y we say that X and Y are isometric or congruent. 22 x1 , x2 X.

Denition 2.21. Let (X, dX ) be space (X, dX ) with a dense subset (It is usual to identify X 0 with X

a metric space. A completion of (X, dX ) is a complete metric X 0 isometric to X. and view X as a subset of X.)

Example 2.6. R is a completion of Q. Before stating the main theorem, we assemble some denitions and basic facts. Lemma 2.6. Let X be a metric space, with metric dX . |dX (p, q) dX (x, y)| dX (p, x) + dX (q, y) Proof of Lemma 2.6 See Exercise Sheet 4, Question 4. Lemma 2.6 implies, in particular, that the metric dX is continuous as a map X X R (see Exercise Sheet 4, Question 4). Corollary 2.5. Let X be a metric space with metric dX . If (xn ) and (yn ) are convergent sequences in X with respective limits p X and q X then dX (xn , yn ) dX (p, q) as n .
0 Lemma 2.7. Let X1 and X2 be metric spaces and assume that X2 is complete. Let X1 be a 0 0 0 dense subset of X1 and X2 a dense subset of X2 . Any isometry X1 X2 extends uniquely to an isometry X1 X2 .

p, q, x, y X.

Proof of Lemma 2.7. Let d1 and d2 be the metrics on X1 and X2 , respectively, and let 0 : 0 0 X1 X2 be an isometry. Let x X1 be arbitrary. By denseness, there exists a sequence (xn ) 0 in X1 converging to x. Since 0 is an isometry, it follows that 0 (xn ) is a Cauchy sequence in 0 the complete space X2 and so is convergent: denote its limit by (x) X2 . If (yn ) X1 is also 0 a sequence converging to x, then d1 (xn , yn ) 0 as n and, since is an isometry, we have d2 (0 (xn ), 0 (yn )) 0 as n . Therefore, 0 (yn ) (x) as n . Therefore, with each x X1 we associate a unique (x) X2 : in other words, the map X1 X2 , x (x) is well 0 dened and is clearly an extension of 0 (that is, |X1 = 0 ). We proceed to show that is an 0 isometry. Let x, y X1 be arbitrary and let (xn ), (yn ) be sequences in X1 converging to x, y, respectively. Invoking Corollary 2.5, we have d2 ((x), (y)) = lim d2 (0 (xn ), 0 (yn )) = lim d1 (xn , yn ) = d1 (x, y)
n n

and so is an isometry. It remains only to establish uniqueness. Let : X1 X2 be an isometry 0 extending 0 . Let x X1 be arbitrary and let (xn ) be a sequence in X1 converging to x. Then, by continuity of , we may conclude that (x) = lim (xn ) = lim (xn ) = (x),
n n

and so = . Let (X, dX ) be a metric space and let Xs be the set of Cauchy sequences from X, that is, Xs := {(xn ) in X| (xn ) is a Cauchy sequence}. On Xs , dene the relation as follows (xn ) (x ) i dX (xn , x ) 0 as n . n n Exercise:

We say that sequences (xn ) and (x ) in Xs are equivalent if (xn ) (x ). Let X be the space of n n equivalence classes of elements of Xs (the equivalence class of (xn ) Xs is {(xn )| (x ) (xn )} = n x X; any such sequence (x ) is a representative of x). n Let x, y X be arbitrary. Let (xn ) and (yn ) be any representatives of x and y, respectively. 23

Show that is an equivalence relation on Xs (Exercise Sheet 7, Question 1).

Lemma 2.8. limn dX (xn , yn ) exists. Proof. Exercise Sheet 7, Question 2.

Lemma 2.9. Let (xn ) and (x ) (respectively, (yn ) and (yn )) be any two representatives of x X n ). Then (respectively, y Y n lim dX (xn , yn ) = lim dX (x , yn ). n n

Proof. Exercise Sheet 7, Question 3. In view of Lemmas 2.8 and 2.9, we see that dX (, y ) := lim dX (xn , yn ), where (xn ) and (yn ) are any representatives of x and y , x
n

denes a map X X R. Exercise: Show that dX satises the axioms of a metric. (Exercise Sheet 7, Question 4.) We are now in a position to state and prove the main result of this section which asserts that every metric space has a completion, unique up to isometries. Theorem 2.14. (a) Any metric space has a completion. (b) Any two completions are isometric: if (X1 , dX1 ) and (X2 , dX2 ) are completions of a metric 0 0 space (X, dX ) with bijective isometries 1 : X X1 , 2 : X X2 then there exists a bijective isometry : X1 X2 such that 1 = 2 . Proof of Theorem 2.14 (a) Let X = (X, dX ) be a metric space. Let X = (X, dX ) be the metric space of equivalence 0 classes of Cauchy sequences from X. Let X be the subset of X of equivalence classes which has a representative of the form (x, x, ...), x X and dene the metric dX 0 := dX |X 0 . Consider the map : (X 0 , dX 0 ) (X, dX ), x x, where (x, x, ...) is a representative of x. It is straightforward to verify that is a bijection. Moreover, for every x, y X 0 (with represen tatives (x, x, ...) and (y, y, ...)), we have dX 0 (, y ) = dX (x, y) = dX ((), ()) x y x and so X 0 and X are isometric. We proceed to show that X 0 is dense in X. Let x X and > 0 be arbitrary. Let (xn ) be a representative of x and choose N N be such that dX (xn , xm ) < /2 for all n, m > N . Let y X 0 be the equivalence class with representative (xN +1 , xN +1 , ...). Then dX (, y ) = lim dX (xn , yn ) = lim dX (xn , xN +1 ) /2 < , x
n n

and so we may infer that X 0 is dense in X. It remains to show that (X, dX ) is complete. Let ((n) ) be a Cauchy sequence in X. Since X 0 is x we may choose, for each n N, and element xn X such that d ((n) , xn ) < 1/n, dense in X, X x where xn X 0 is the equivalence class with representative (xn , xn , ...). Let > 0 be arbitrary. Since ((n) ) is Cauchy, there exists N N such that x dX ((n) , x(m) ) < /3 n, m > N. x

24

Therefore, we have dX (xn , xm ) = dX (n , xm ) dX (n , x(n) ) + dX ((n) , x(m) ) + dX ((m) , xm ) x x x x < 1 1 + + n 3 m

3 n, m > max N ,

and so (xn ) is a Cauchy sequence in X. Now, let x be the element of X with representative (xn ). Let > 0 be arbitrary. Since (xn ) is Cauchy, there exists N N such that dX (n , xm ) = x dX (xn , xm ) < /2 for all n, m > N . For each m > N , we have dX (, xm ) = lim dX (xn , xm ) x
n

. 2

Therefore, dX (, x(m) ) dX (, xm ) + dX (m , x(m) ) < x x x 1 + 2 m m > max N, 2 .

We may now conclude that dX (, x(m) ) 0 as m and so the Cauchy sequence ((n) ) in X x x is convergent with limit x X.

(b) Let (X1 , dX1 ) and (X2 , dX2 ) be two completions of (X, dX ) with associated bijective isometries 0 , 2 : X X 0 . Dene 0 : X 0 X 0 by 0 := 2 1 . Clearly, 0 is bijective; 1 : X X1 2 1 2 1 moreover, dX 0 ( 0 (), 0 ()) = dX 0 (2 (1 ()), 2 (1 ())) = dX (1 (), 1 ()) = dX 0 (, y ) x y x y x 1 y x 1 1 1
2 2 1

and so 0 is an isometry. By Lemma 2.7, 0 extends uniquely to an isometry : X1 X2 and 0 1 0 X 0 extends uniquely to an isometry : X2 X1 . Thus, we have ( ) : X2 1 ( )
0 X1

= ( 0 )1 0 = idX 0 ,
1

( )

0 X2

= 0 ( 0 )1 = idX 0 .
2

0 0 As the maps are continuous and as X1 is dense in X1 and X2 is dense in X2 , the above identities extend to = idX1 , = idX2 , which implies that a bijection (with inverse 1 = ). 2

25

Chapter 3

Real Hilbert spaces

3.1 Inner product spaces

Denition 3.1. A real inner product (prehilbert) space is a real vector space X equipped with the function , : X X R satisfying the following axioms: x, x 0 x X, and x, x = 0 i x = 0; x, y = y, x x, y X; x, y, z X , R. x + y, z = x, z + y, z (a) x, y (b) (c) (d)
2

Theorem 3.1. Let X be an inner product space. x, x y, y

2

x, y X (the CauchySchwarz inequality).

2

x := x+y

x, x denes a norm on X. + xy =2 x
2

+2 y

x, y X (the parallelogram law).

, : X X R is continuous.

Proof of Theorem 3.1 (a) For y = 0 the assertion is true, since x, 0 = x, z z = x, z x, z = 0 = x, x 0, 0 . Assume y = 0. Let t R be a parameter. We have 0 x + ty, x + ty = x, x + 2t x, y + t2 y, y = x, x + y, y t + ( x, y / y, y ) Setting t = x, y / y, y gives the result. (b) Clearly, x 0 for all x X and x = 0 if, and only if, x = 0. Also, x = || x for all x X and all R. Furthermore, x+y
2 2

x, y 2 / y, y

t R.

= x + y, x + y = x, x, + x, y + y, x + y, y = x
2

+ 2 x, y + y

+2 x

y + y

= ( x + y )2 x, y X, =2 x
2

whence the triangle inequality. (c) x + y

2

+ xy

= x

+ 2 x, y + y

+ x

2 x, y + y

+2 y

x, y X.

26

(d) Let (xn ) and (yn ) be convergent sequences in (X, ) with respective limits x, y (X, Then we have xn , yn x, y = xn , yn xn , y + xn , y x, y = xn , yn y + xn x, y , whence | xn , yn x, y | xn yn y + xn x y 0 as n .

).

2 Part (b) of Theorem 3.1 implies that an inner product space is a special case of a normed space.

3.2

Hilbert spaces

Complete inner product spaces are called Hilbert spaces.

David Hilbert (1862-1943)

Example 3.1 (Finite-dimensional Euclidean space). The space Rn is the vector space of n-tuples of real numbers equipped with the inner product
n

x, y :=
j=1

xj yj ,

x = (x1 , . . . , xn ),

y = (y1 , . . . , yn ).

and induced norm

n

x =
(m) (m)

x, x =
j=1

x2 . j

Let (xm ) = (x1 , ..., xn ) be a Cauchy sequence in Rn . Let > 0 be arbitrary. Then there exists N N such that
n

xm xk whence, |xj
(m)

=
j=1

xj

(m)

xj

(k) 2

< 2

m, k > N ,

xj | <

(k)

m, k > N,

j = 1, ..., n.

Therefore, the sequences (xj ), j = 1, ..., n, are Cauchy sequences in R and so, by completeness of R, are convergent with limits xj R, j = 1, ..., n. It follows that the sequence (xm ) Rn is convergent with limit x = (x1 , ..., xn ) Rn . Therefore, Rn is complete and so is a Hilbert space. Example 3.2 (Square-summable real sequences). Denote, by l2 , the space of square-summable real sequences, that is, the space of sequences x = (xn ) with the property that nN x2 < . With the obvious notions of addition, x+y = (xn )+(yn ) := n 27

(m)

(xn + yn ), and scalar multiplication, x = (xn ) := (xn ), l2 is a real vector space. For every 2 x = (xn ) l2 and y = (yn ) l2 , we have 2|xn yn | x2 + yn for all n N and so n 1 1 x2 + y2 |xn yn | 2 n=1 n 2 n=1 n n=1 and so the series is well dened
nN N

N N

xn yn is absolutely convergent. Therefore, for every x, y l2 , the following

x, y :=
n=1

xn yn ,

x = (xn ), y = (yn )

and the inner product axioms hold. We equip l2 with this inner product, which induces the norm given by x = x, x =
nN

x2 , n

and proceed to show that it is a complete inner product space. Theorem 3.2. l2 is a Hilbert space. Proof of Theorem 3.2 Let (x(n) ) = (xk ) be a Cauchy sequence in l2 . For every k N we (n) (m) (n) have |xk xk | x(n) x(m) and so (xk ) is Cauchy sequence in R, hence, convergent to some xk R. Set x := (xk ). We will prove that x l2 and that x(n) x 0 as n . Let > 0 be arbitrary. Since (x(n) ) is a Cauchy sequence, there exists N N such that x(n) x(m) < /2 for all n, m > N . Therefore, we have
k (n)

xj
j=1

(n)

xj

(m) 2

<

, 2

k N,

n, m > N.

Passing to the limit as m , we obtain

k

xj
j=1

(n)

xj

, 2

k N,

n>N.

Passing to the limit as k , yields x(n) x < , 2 n > N.

Therefore, x(n) x 0 as n . Moreover, since > x(N +1) + x(N +1) x x(N +1) + x x(N +1) , it follows that x l2 2 The space l2 is the simplest innite-dimensional example of a Hilbert space.

3.3

Convexity and the projection lemma

x, y K = (1 t)x + ty K, t [0, 1].

Denition 3.2. A subset K of a vector space is said to be convex if

In words: K contains the line segment joining any two of its points. 28

K x y

Denition 3.3. In a metric space (with metric d), the distance from a point p to a set K is dist(p, K) := inf d(p, q)
qK

The following is the Projection Lemma. Lemma 3.1. Let K be a closed, convex, non-empty subset of a real Hilbert space H. Then, for all p H, there exists unique q K such that p q = dist(p, K).

Proof of Lemma 3.1 Existence. For brevity, write := dist(p, K). There exists a sequence (qn ) in K such that qn p . Now invoke the parallelogram law (Theorem 3.1(c), with x = qn p, y = qm p ) to obtain qn + qm 2p
2

+ qn qm

= 2 qn p

+ 2 qm p

n, m N.

By convexity of K, (qn + qm )/2 is in K and so, by the denition of , we have qn + qm 2p We may now infer that qn qm
2 2

=4

qn + qm p 2

42 n, m N .

2 qn p

+ 2 qm p

42 n, m N

and so, for each > 0, there exists N N such that qn qm < n, m > N. Therefore, (qn ) is a Cauchy sequence in K and so converges to some q H. By closedness of K, we have q K and, by continuity of the norm, q p = . Uniqueness. Let q, q K be such that pq = pq = . By the parallelogram law (Theorem 3.1(c), with x = q p, y = q p ), we have q + q 2p
2

+ q q

=2 qp

+ 2 q p
2

= 42 .

By convexity of K and the denition of , q + q 2p Therefore, which implies q = q . q q

2 2

=4

q + q p 2

42

42 42 = 0 2 29

3.4

Orthogonality and the projection theorem

Denition 3.4. (i) Elements x and y of an inner product space are said to be perpendicular or orthogonal if x, y = 0 (in which case, we write x y). (ii) Let Y , Z be subspaces of a vector space X. We say that X is a direct sum of Y and Z and write X = Y Z if every x X can be written in a unique way as x = y + z, y Y, z Z.

(iii) For a subspace Y of an inner product space X the orthogonal complement Y is the subspace given by Y := {x X| x y, y Y }. The following is the Projection Theorem. Theorem 3.3. Let Y be a closed vector subspace of a Hilbert space H. Then H = Y Y . Proof of Theorem 3.3 Let x H be arbitrary. Noting that Y is closed, convex, and non-empty (0 Y ), we may invoke the Projection Lemma 3.1 to conclude the existence of a unique point y Y nearest to x. Dene z := x y. We will show that z Y . Suppose that z Y . Then there exists v Y such that z, v = 0. For each R, we have y + v Y and so z
2

= (dist(x, Y ))2 x (y + v)

= z v = z
2

= z
2

+ v

( z, v / v 2 )

2 z, v + 2 v
2

z, v / v

Setting = z, v / v 2 , yields the contradiction 0 z, v v

2

< 0.

Let y Y and z Y be such that x = y + z . Then y y = z z, Therefore, and so y = y which, in turn, implies z = z . y y
2

Therefore, there exist y Y and z Y such that x = y + z. y y Y, z z Y .

= y y, y y = y y, z z = 0 , 2

3.5

Orthonormal basis

Let {x }I be an indexed set in an inner product space X. The elements of this set are assumed to be indexed without repetition, that is, x = x whenever = . Here, the set I is the index set. Possible index sets are: I = {1, . . . , n} (nite), or I = N (countable), or I may be uncountable. Denition 3.5. A set {x }I is said to be linearly independent if, for every nite subset {x1 , . . . , xn },
n

cj xj = 0
j=1

c1 = = cn = 0.

30

Denition 3.6. The space of all nite linear combinations of elements from {x }I is called the span of this set and denoted span{x }I . Denition 3.7. A set {x }I is said to be complete in X if span{x }I is dense in X. Denition 3.8. A set {x }I is said to be orthogonal in X if x x , , I, = . Denition 3.9. A set {x }I is said to be orthonormal in X if it is orthogonal in X and x , x = 1, I. Let {x }I be orthonormal in X and let {x1 , ..., xn ) be any nite subset. Assume c1 x1 + + cn xn = 0 for some scalars c1 , ..., cn . Taking inner product with xj yields cj = 0, j = 1, ..., n. This proves the following: Lemma 3.2. An orthonormal set in an inner product space is linearly independent. Given a countable linearly independent set {x1 , x2 , ...} in X, there is a standard inductive procedure, called the Gram-Schmidt process, for generating an orthonormal set {y1 , y2 , ...} with the property that span{x1 , ..., xn } = span{y1 , ..., yn } for all n N. The initial step in the process is to set y1 := x1 / x1 . The inductive step is as follows: assume given an orthonormal set n {y1 , ..., yn } with span{x1 , ..., xn } = span{y1 , ..., yn }, then dene vn+1 := xn+1 j=1 xn+1 , yj yj and, observing that vn+1 = 0 because xn+1 is not in span{x1 , ..., xn } = span{y1 , ..., yn }, we set yn+1 := vn+1 / vn+1 ; noting that vn+1 , yj = 0 for j = 1, ...n, we see that {y1 , ..., yn+1 } is an orthonormal set and it is clear that span{y1 , ..., yn+1 } = span{x1 , ..., xn+1 }; this completes the inductive step. We summarize the above in the following theorem. Theorem 3.4. Let {x1 , x2 , . . .} be a countable linearly independent set in an inner product space. Then there exists an orthonormal set {y1 , y2 , . . .} such that, for all n N, span{x1 , ..., xn } = span{y1 , ..., yn }. In particular, the sets {x1 , x2 , . . .} and {y1 , y2 , . . .} have the same spans, and so one set is complete if, and only if, the other set is complete. Denition 3.10. A complete orthonormal set in an inner product space X is called an orthonormal basis for X.

3.6

Approximation in Hilbert spaces

In the remainder of this chapter, we will use the Projection Lemma 3.1 and the Projection Theorem 3.3 in studying the approximation problem in Hilbert spaces. Let {x1 , . . . , xn } be a nite orthonormal set in the inner product space X. Consider the Gauss approximation problem: given x X, nd c1 , . . . , cn R such that
n

x is minimal.

ck xk
k=1

Theorem 3.5. In a Hilbert space, the Gauss approximation problem has the unique solution ck = x, xk , k = 1, . . . , n. The element z := x n ck xk is orthogonal to span{x1 , . . . , xn }. Moreover, k=1
n

= x

c2 k
k=1

(Bessels equality)

(3.1)

31

Johann Carl Friedrich Gauss (1777-1855)

and

k=1

c2 x k

(Bessels inequality).

(3.2)

Proof of Theorem 3.5 Let H be a Hilbert space and let {x1 , . . . , xn } be a nite orthonormal set in H. Let x H be arbitrary. The subspace Y := span{x1 , . . . , xn } is nite dimensional and so is complete. Let (ym ) be a convergent sequence in Y with limit y H. Then (ym ) in Cauchy in H and so is Cauchy in Y . By completeness of Y , it follows that y Y and so Y is closed. By the Projection Theorem 3.3, there is a unique y Y and a unique z Y such that x = y + z and, furthermore, x y is minimal. Since y Y , we have y = c1 x1 + + cn xn for some c1 , ..., cn R and, since z Y , it follows that x, xk = y + z, xk = y, xk + z, xk = y, xk = ck , Moreover, x
2 n

k = 1, ..., n.

= y+z

= y

+ z

= z

+
k=1

c2 , k 2

whence Bessels equality and Bessels inequality.

32