Introduction To Functional Analysis PDF

Introduction to Hilbert Space
and Applications
Assoc. Prof. Waroth Kuhirun
February 20, 2017

2
Preface
This textbook includes two parts; the first part presents theory corresponding to
the course whereas the second part presents related applications. The material
is intended for engineering students, not for mathematics students. The author
ignores some material which is considered to be too complicated for engineer-
ing students. Especially in the second part, the author attempts to show the
relationship between theory and engineering applications rather than to show
rigorous proofs for the aforementioned engineering applications. The material
included in this textbook is referenced from documents including books, papers,
etc, listed in bibliography.
Finally, the author would like to thank Assoc. Prof. Vutipong Areekul and
Dr. Phunsak Thiennviboon for providing digital signal processing notes and
adaptive filtering notes, respectively, and Assoc. Prof. Yotsanan Meemark for
advice.
Assoc. Prof. Waroth Kuhirun
i
ii PREFACE
Contents
Preface i
1 Mathematical Fundamentals 1
1.1 Supremum, Maximum, Infemum and Minimum . . . . . . . . . . 1
1.2 Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Countable and Uncountable Sets . . . . . . . . . . . . . . . . . . 3
1.4 Neighborhood, Points and Sets . . . . . . . . . . . . . . . . . . . 4
1.5 Sequences and Limits . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Vector Spaces 7
3 Normed Spaces and Banach Spaces 15

3.1 Norm and Normed Spaces . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.2 Examples of Norms . . . . . . . . . . . . . . . . . . . . . 17
3.1.3 Normed Spaces . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.4 Sets in Normed Spaces . . . . . . . . . . . . . . . . . . . . 20
3.1.5 Examples of Normed Spaces . . . . . . . . . . . . . . . . . 21
3.2 Completion and Banach Spaces . . . . . . . . . . . . . . . . . . . 23
3.3 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4 Inner Product Spaces and Hilbert Spaces 25

4.1 Inner Products and Inner Product Spaces . . . . . . . . . . . . . 25
4.1.1 Examples of Inner Product Spaces . . . . . . . . . . . . . 28
4.2 Completion and Hilbert Spaces . . . . . . . . . . . . . . . . . . . 29
4.3 Orthogonality and Orthonormality . . . . . . . . . . . . . . . . . 32
4.4 Best Approximation and Orthogonal Projection . . . . . . . . . . 34
4.5 Generalized Fourier Series . . . . . . . . . . . . . . . . . . . . . . 40
4.6 Gram-Schmidt Orthononormalization or Process . . . . . . . . . 41
4.7 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5 Operators and Functionals 43

5.1 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.2 Kernels, Null Spaces and Range Spaces . . . . . . . . . . . . . . 47
iii
iv CONTENTS
5.3 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.4 Hilbert-Adjoint Operators . . . . . . . . . . . . . . . . . . . . . . 53
5.5 Self-Adjoint Operators and Formally Self-Adjoint Operators . . . 57
5.6 Positive Operators . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.7 Compact Operators . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.8 Contraction Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.9 Minimization of Functionals . . . . . . . . . . . . . . . . . . . . . 64
5.10 Sturm - Liouville Operator . . . . . . . . . . . . . . . . . . . . . 65
5.11 Hilbert-Schmidt Operator . . . . . . . . . . . . . . . . . . . . . . 67
6 Eigenvalues, Eigenvectors, Eigenfunctions 69

6.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . 69
6.1.1 Properties related to Eigenvalues and Eigenvectors . . . . 69
6.1.2 Singular Value Decomposition . . . . . . . . . . . . . . . . 71
6.2 Eigenvalues and Eigenfunctions . . . . . . . . . . . . . . . . . . . 73
7 Examples of Applications 77
7.1 Finite Element Method . . . . . . . . . . . . . . . . . . . . . . . 77
7.2 Method of Moments . . . . . . . . . . . . . . . . . . . . . . . . . 78
A Matrices 79
B Integrals 81
B.1 Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
B.2 Riemann-Stieltjes Integral . . . . . . . . . . . . . . . . . . . . . . 81
B.3 Lebesgue Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
C Holder, Cauchy-Schwarz and Minkowski Inequalities 85

C.1 Holder Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
C.2 Cauchy-Schwarz Inequality . . . . . . . . . . . . . . . . . . . . . 85
C.3 Minkowski Inequality . . . . . . . . . . . . . . . . . . . . . . . . . 85
D Complex Analysis 87
E Gateaux Differential 91
Chapter 1
Mathematical
Fundamentals
1.1 Supremum, Maximum, Infemum and Mini-

mum
Definition 1 (Supremum). Let a nonempty set S Ă R and S be bounded above.
Suppose that the exists α P R with the following properties:
1. α is an upper bound of S.
2. For β P R, if β ă α, β is not an upper bound of S.
Therefore, α is called the supremum (least upper bound) of S and we write
α “ sup S
Definition 2 (Maximum). Let a nonempty set S Ă R and S. If α P S and

α ě β for all β P S, α is called the maximum of S and we write
α “ max S
Definition 3 (Infemum). Let S Ă R and S be bounded below. Suppose that the

exists α P R with the following properties:
1. α is a lower bound of S.
2. For β P R, if β ą α, β is not a lower bound of S.
Therefore, α is called the infemum (greatest lower bound) of S and we write
α “ inf S
1
2 CHAPTER 1. MATHEMATICAL FUNDAMENTALS
Definition 4 (Minimum). Let a nonempty set S Ă R and S. If α P S and

α ď β for all β P S, α is called the minimum of S and we write
α “ min S
By definition, if maximum exists, supremum equals maximum and if mini-

mum exists, infemum equals minimum. In contrast, if supremum exists, max-
imum may not exist and if infemum exists, minimum may not exist. The dif-
ference between supremum and maximum and the difference between infemum
and minimum show in examples 1 and 2, respectively.
1
Example 1. Let an “ 1 ´ , n P N. Find suptan u and maxtan u
n nPN nPN
Solution
It is trivial that an ď 1 for all n P N. In addition, we can show that, for any
c ă 1, there exists n such that an ą c. Therefore, suptan u “ 1.
nPN
Moreover, since
ˆ ˙ ˆ ˙
1 1 1 1
an`1 ´ an “ 1´ ´ 1´ “ ´ ą 0,
n`1 n n n`1
then maxtan u does not exist.

nPN
1
Example 2. Let an “ 1 ` , n P N. Find inf tan u and mintan u
n nPN nPN
Solution
It is trivial that an ě 1 for all n P N. In addition, we can show that, for any
c ą 1, there exists n such that an ă c. Therefore, inf tan u “ 1.
nPN
Moreover, since
ˆ ˙ ˆ ˙
1 1 1 1
an`1 ´ an “ 1` ´ 1` “ ´ ă 0,
n`1 n n`1 n
then mintan u does not exist.

nPN
1.2 Mappings
Definition 5 (Functions or Mappings). Consider sets X and Y . Suppose that
for each x P X, there is an associated element, y P Y , which is denoted by f pxq.
Then, f is called a function or mapping from X into Y , we write f : X Ñ Y .
X is called the domain of f whereas ty | y “ f pxq for some x P Xu is called the
range of f .
Definition 6 (1-to-1 Functions or Mappings). Consider a function f : X Ñ Y .
Suppose that, for all x1 , x2 P X, f px1 q “ f px2 q, then x1 “ x2 . Therefore, f is
called a 1 ´ 1 function or mapping.
1.3. COUNTABLE AND UNCOUNTABLE SETS 3
Definition 7 (Onto Functions or Mappings1 ). Consider f : X Ñ Y . Suppose

that, for all y P Y , there exists x P X such that y “ f pxq. Therefore, f is called
a function or mapping from X onto Y .
Definition 8 (Bijective Functions or Mappings). A function or mapping f is

called bijective function or mapping iff f is both 1 ´ 1 and onto.
Definition 9 (Continuity of Functions or Mappings). Consider a function or

mapping f : X Ñ Y . Suppose that, for all ą 0, there exists δ ą 0 such that if
|x ´ x0 | ă δ, then |f pxq ´ f px0 q| ă
Therefore, f is said to be continuous at x0
Definition 10 (Indicator Functions). A function IA : R Ñ R is said to be an

indicator function of A iff
#
1, x P A
IA pxq “
0, x R A
Definition 11 (Positive and Negative Parts of f ). Let f be a function defined

on R. The positive part f ` and the negative part f ´ of f is defined as
f ` pxq “ maxtf pxq, 0u and f ´ pxq “ maxt´f pxq, 0u.
Definition 12 (Simple Functions). A simple function f is a function of the

form
N
ÿ
f pxq “ an fn pxq
n“1
#
1, x P An
where an P R for n “ 1, 2, 3, . . . and fn pxq “ IAn pxq “ provided
0, x R An
that A1 , A2 , A3 , . . . are disjoint sets.
1.3 Countable and Uncountable Sets

Definition 13 (Cardinal Numbers). Let A and B be sets. If there is a 1-1
mapping of A onto B, A and B are said to have the same cardinal numbers
(cardpAq “ cardpBq). In other words, A and B are equivalent.
Definition 14 (Countable and Uncountable Sets). Let Jn “ 1, 2, 3, . . . , n and

J “ 1, 2, 3, . . ..
1. A set S is said to be finite or finitely countable iff, for some n P N, there

is a bijective mapping f : Jn Ñ S.
1 sometimes called Surjective functions or mappings
2. A set S is said to be infinitely countable iff, for some n P N, there is a

bijective mapping f : Jn Ñ S.
3. A set S is said to be countable iff S is finite or infinitely countable.
4. A set S is uncountable iff S is not countable.
Example 3. Show that S “ t0, 2, 4, . . .u is a countable set.
Solution Let f : J Ñ S be defined as
f pnq “ 2pn ´ 1q for n P J
where J “ t1, 2, 3, . . .u
It is obvious that f is a bijective mapping. Therefore, S is countable. In
particular, S is infinitely countable.
1.4 Neighborhood, Points and Sets

Definition 15 (Neighborhood). A neighborhood of x with radius r denoted by
Nr pxq is defined by
Nr pxq “ ty P R | |x ´ y| ă r, r ą 0u
Definition 16 (Limit Points). A point x0 is called a limit point of S Ă R iff

Nr px0 q contains a point x P S
Definition 17 (Closed Sets). A set S P R is called a closed set iff every x P S

is a limit point in S.
Definition 18 (Interior Points). A point x0 is called an interior point of S Ă R

iff there exists Nr px0 q Ă S.
Definition 19 (Open Sets). S is an open set iff every point x P S is an interior

point
Definition 20 (Denseness). S is dense in X iff every point in X is a limit

point in S.
Definition 21 (Closure). Let S Ă R and S 1 be the set of all limit points of S

in R. Therefore, the closure of S, denoted by S, is defined as
S “ S Y S1
Definition 22 (Bounded Sets). A subset M ‰ H of R is said to be a bounded

set iff its diameter
δpM q “ sup |x ´ y| ă 8
x,yPM
1.5. SEQUENCES AND LIMITS 5
1.5 Sequences and Limits

Definition 23 (Bounded Sequences). A sequence txn u8 n“1 in R is said to be
bounded iff there exists M such that |xn | ă M for n “ 1, 2, 3, . . ..
Definition 24 (Cauchy Sequence). A sequence txn , n P Nu is said to be a
Cauchy sequence in R iff for all ą 0 there exist an integer N ą 0 such that
|xm ´ xn | ă for all integers m, n ą N
Definition 25 (Limit, Convergent and Divergent Sequences). A sequence txn , n P

Nu is said to be a convergent sequence to a limit2 x in R iff an element x in
X and for all ą 0 there exist an integer N ą 0 such that |x ´ xn | ă for all
integer n ą N .
In addition, a sequence which is not a convergent sequence is said to be a diver-
gent sequence.
Theorem 1 (Bolzano-Weierstrass Theorem). Every bounded sequence in R has
a convergent subsequence.
Proof see the proof for a more general case in[68]
2 The proof for the uniqueness of the limit of a convergent sequence in a metric space pX, dq
is shown in example ??
Chapter 2
Vector Spaces
Definition 26 (Fields). Field F is a nonempty set satisfying the following prop-

erties
1. α ` β P F and α.β P F for α and β P F
2. α ` β “ β ` α and α.β “ β.α for α and β P F
3. pα ` βq ` γ “ α ` pβ ` γq and pα.βq.γ “ α.pβ.γq for α, β and γ P F
4. α.pβ ` γq “ α.β ` α.γ for α, β and γ P F
5. For all α P F there exists 0 P F such that α`0 “ α and, for all α P F, there
exists 1 P F such that 1.α “ α
6. For all α P F there exists β P F such that α ` β “ 0
7. For all α P F provided that α ‰ 0 there exists γ P F such that α.γ “ 1
Example 4. Show that the set of all real numbers R is a field.
Solution Since the set of all real numbers R satisfies the following properties
1. α ` β P R and α.β P R for α and β P R
2. α ` β “ β ` α and α.β “ β.α for α and β P R
3. pα ` βq ` γ “ α ` pβ ` γq and pα.βq.γ “ α.pβ.γq for α, β and γ P R
4. α.pβ ` γq “ α.β ` α.γ for α, β and γ P R
5. For all α P R there exists 0 P R such that α`0 “ α and, for all α P R, there
exists 1 P R such that 1.α “ α
6. For all α P R there exists β P R such that α ` β “ 0
7. For all α P R provided that α ‰ 0 there exists γ P R such that α.γ “ 1
7
8 CHAPTER 2. VECTOR SPACES
it follows that the set of all real numbers R is a field.
Example 5. Show that the set of all complex numbers C is a field
Solution
Since the set of all complex numbers C satisfies the following properties
1. α ` β P C and α.β P C for α and β P C
2. α ` β “ β ` α and α.β “ β.α for α and β P C
3. pα ` βq ` γ “ α ` pβ ` γq and pα.βq.γ “ α.pβ.γq for α, β and γ P C
4. α.pβ ` γq “ α.β ` α.γ for α, β and γ P C
5. For all α P C there exists 0 P C such that α`0 “ α and, for all α P C, there
exists 1 P C such that 1.α “ α
6. For all α P C there exists β P C such that α ` β “ 0
7. For all α P C provided that α ‰ 0 there exists γ P C such that α.γ “ 1
it follows that the set of all complex numbers C is a field.
Example 6. Show that the set of all integers I is not a field
Solution
Since the set of all integers I does not satisfy the following property (For
all α P I provided that α ‰ 0 there exists γ P F such that α.γ “ 1), it follows
that the set of all integers I is not a field.
Definition 27 (Vector Spaces). Vector Space V over F is a nonempty set sat-

isfying the following properties
1. x ` y P V for x and y P V
2. x ` y “ y ` x for x and y P V
3. px ` yq ` z “ x ` py ` zq for x, y and z P V
4. For all x P V there exists 0 P V such that x ` 0 “ x
5. For all x P V, there exists ´x P V such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P F and x P V
7. pα ` βqx “ αx ` βx for α and β P F and x P V
8. αpx ` yq “ αx ` αy for α P F and x, y P V
9. 1x “ x for x P V
Example 7. Show that R is a vector space over R.

9
Solution
Since R satisfies the following properties
1. x ` y P R for x and y P R
2. x ` y “ y ` x for x and y P R
3. px ` yq ` z “ x ` py ` zq for x, y and z P R
4. For all x P R there exists 0 P R such that x ` 0 “ x
5. For all x P R, there exists ´x P R such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P R and x P R
7. pα ` βqx “ αx ` βx for α and β P R and x P R
8. αpx ` yq “ αx ` αy for α P R and x, y P R
9. 1x “ x for x P R
it follows that R is a vector space over R.
Example 8. Show that C is a vector space over C.
Solution
Since C satisfies the following properties
1. x ` y P C for x and y P C
2. x ` y “ y ` x for x and y P C
3. px ` yq ` z “ x ` py ` zq for x, y and z P C
4. For all x P C there exists 0 P C such that x ` 0 “ x
5. For all x P C, there exists ´x P C such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P C and x P C
7. pα ` βqx “ αx ` βx for α and β P C and x P C
8. αpx ` yq “ αx ` αy for α P C and x, y P C
9. 1x “ x for x P C
it follows that C is a vector space over C.
Example 9. Show that, for n P N, Rn is a vector space over R.
Solution
Since Rn satisfies the following properties
1. x ` y P Rn for x and y P Rn
2. x ` y “ y ` x for x and y P Rn
3. px ` yq ` z “ x ` py ` zq for x, y and z P Rn
4. For all x P Rn there exists 0 P Rn such that x ` 0 “ x
5. For all x P Rn , there exists ´x P Rn such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P R and x P Rn
7. pα ` βqx “ αx ` βx for α and β P R and x P Rn
8. αpx ` yq “ αx ` αy for α P R and x, y P Rn
9. 1x “ x for x P Rn
it follows that Rn is a vector space over R.
Example 10. Show that, for n P N, Cn is a vector space over C.
Solution
Since Cn satisfies the following properties
1. x ` y P Cn for x and y P Cn
2. x ` y “ y ` x for x and y P Cn
3. px ` yq ` z “ x ` py ` zq for x, y and z P Cn
4. For all x P Cn there exists 0 P Cn such that x ` 0 “ x
5. For all x P Cn , there exists ´x P Cn such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P C and x P Cn
7. pα ` βqx “ αx ` βx for α and β P C and x P Cn
8. αpx ` yq “ αx ` αy for α P C and x, y P Cn
9. 1x “ x for x P Cn
it follows that Cn is a vector space over C.
Example 11. Show that a set of all polynomial functions in the interval ra, bs
is a vector space over R
Solution
Let P ra, bs be a set of all polynomial functions in the interval ra, bs. Since R
satisfies the following properties
1. f ` g P P ra, bs for f and g P P ra, bs
2. f ` g “ g ` f for f and g P P ra, bs
3. pf ` gq ` h “ f ` pg ` hq for f, g and h P P ra, bs
11
4. For all f P P ra, bs there exists 0 P P ra, bs such that f ` 0 “ f

5. For all xf P P ra, bs, there exists ´f P P ra, bs such that f ` p´f q “ 0
6. αpβf q “ pαβqf for α and β P R and f P P ra, bs
7. pα ` βqf “ αf ` βf for α and β P R and f P P ra, bs
8. αpf ` gq “ αf ` αg for α P R and f, g P P ra, bs
9. 1f “ f for f P P ra, bs
it follows that P ra, bs is a vector space over R.
Definition 28 (Subspaces). Let a nonempty set S is a subset of a vector space
V over a field F. S is said to be a subspace of the vector space V iff S is itself
a vector space over the same field F under the same operations.
Theorem 2. Let S ‰ H be a subset of a vector space V over a field F. There-
fore, S is a subspace of the vector space V iff
1. x ` y P S for all x, y P S
2. αx P S for all α P F and x P V
Proof
Since S ‰ H is a subset of a vector space V over the same field F, most
properties of the vector space V over the field F are automatically satisfied.
The proof is left for exercise.
Definition 29 (Linear Combination). Let x1 , x2 , . . . , xn belong to a vector space
n
ÿ
over a field F and α1 , α2 , . . . , αn belong to the field F. x “ αi xi is said to be
i“1
a linear combination of x1 , x2 , . . . , xn
Definition 30 (Independence). Let x1 , x2 , . . . , xn belong to a vector space over
a field F. x1 , x2 , . . . , xn are independent iff
n
ÿ
αi xi “ 0
1“1
has only one solution, that is,
α1 “ α2 “ . . . “ αn “ 0.
Otherwise, x1 , x2 , . . . , xn are dependent.

It should be noted that if αi ‰ 0, that is, xi is dependent from xj , j ‰ i,
then we can express xi as
n
ÿ αj
xi “ ´ xj
α
j“1,j‰i i
Example 12. Let A ‰ 0 be n ˆ n symmetric matrix and B ‰ 0 be an anti-

symmetric matrix. Show that A and B are independent.
Solution Let aij be the element of ith row and j th column of matrix A and
bij be the element of ith row and j th column of matrix B. We can determine
whether A and B are independent by solving the following equation
αA ` βB “ 0 (2.1)
Consequently,
αaij ` βbij “ 0 (2.2)
and
αaji ` βbji “ 0 (2.3)
By using the fact that A and B are symmetric and anti-symmetric, respectively.
We obtain
aij “ aji and bij “ ´bji
Consequently,
αaji ´ βbji “ 0 (2.4)
Regarding to equations 2.2 and 2.4, we obtain
αaij “ 0 and βbij “ 0
It should be noted that, for some i and j, there exist aij ‰ 0. This is also
applicable for bij . Consequently, α “ 0 and β “ 0. Therefore, A and B are
independent.
Definition 31 (Spanning Sets). Let x1 , x2 , . . . , xn belong to a vector space V
over a field F. The spanning set of S “ tx1 , x2 , . . . , xn u is defined in the follow-
ing
ÿn
SpanpSq “ tx | x “ αi xi u
i“1
Definition 32 (Hamel Basis). Let S be a linear independent subset of a vector

space V. Therefore, S is a Hamel basis of V iff
V “ spanpSq
It follows that
Lemma 1. If S is a (Hamel) basis for a vector space V over field F, then, for
a nonzero x P V has a unique representation as a linear combination of finitely
many n elements of S with nonzero coefficients. That is, x can be expressed
uniquely as
n
ÿ
x“ αi xi
i“1
where αi P F and xi P S for i “ 1, 2, 3, . . . , n
13
Theorem 3. Every vector space V ‰ t0u has a Hamel basis.

Proof
The proof is beyond the scope of this text. See the proof in [11].
Definition 33 (Dimension). Let S be a basis of a vector space V over a field
F. The dimension of the vector space V over the field F is
dimpVq “ cardpSq
Definition 34 (Direct Sum). Let X and Y be subspaces of a vector space V.

V is said to be a direct sum of X and Y , written
à
V “X Y
iff for each v P V has a unique representation
v “ x ` y, x P X and y P Y
Y is an algebraic complement of X in V
Definition 35 (Convex Sets). Let S be a subset of a vector space V. S is called
a convex set iff αx1 ` p1 ´ αqx2 for any α P r0, 1s and x1 , x2 P S
Chapter 3
Normed Spaces and Banach

Spaces
3.1 Norm and Normed Spaces

3.1.1 Norms
Definition 36 (Norm). Let a vector space V be a vector space over field F1
equipped with a mapping }.} : V Ñ R satisfying the following properties
1. }αx} “ |α|}x} for all α P F and x P V
2. (Non-negativity) }x} ě 0 for all x P V
3. (Triangular Inequality) }x ` y} ď }x} ` }y} for all x, y P V
4. (Non-degeneracy) }x} “ 0 iff x “ 0
Lemma 2. |}x1 } ´ }y 1 }| ď }x1 ´ y 1 }
Proof
According to triangular inequality
}x ` y} ď }x} ` }y}. (3.1)
replace x and y with x1 ´ y 1 and y 1 , respectively, we obtain
}x1 } ´ }y 1 } ď }x1 ´ y 1 } (3.2)
Similarly, replace y and x in equation 3.1 with y 1 ´ x1 and x1 , respectively.

respectively, we obtain
}y 1 } ´ }x1 } ď }x1 ´ y 1 } (3.3)

1 In this definition, the field F can be only R and C
15
16 CHAPTER 3. NORMED SPACES AND BANACH SPACES
By inequalities 3.2 and 3.3, we obtain
|}x1 } ´ }y 1 }| ď }x1 ´ y 1 }
Definition 37 (Equivalent Norms). }.}1 and }.}2 are said to be equivalent iff
for any sequence pxn q and x in a vector space V,
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8
Theorem 4. Let }.}1 and }.}2 be norms in a vector space V. }.}1 and }.}2 are
equivalent iff there exist α ą 0 and β ą 0 such that
α}x}1 ď }x}2 ď β}x}1
Proof
First of all, suppose that there exist α ą 0 and β ą 0 such that α}x}1 ď
}x}2 ď β}x}1 . Clearly, it follows that
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8.
Conversely, we prove that if
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8.
then, there exist α ą 0 and β ą 0 such that α}x}1 ď }x}2 ď β}x}1 by contra-
diction.
Suppose that there exists no α ą 0 such that
α}x}1 ď }x}2
for all x P V
Therefore, for all n, there exists xn P V such that
1
}xn }1 ą }xn }2
n
if we define
1 xn
yn “ ? ,
n }xn }2
then
1 }xn }2 1
}yn }2 “ ? “ ? Ñ 0.
n }xn }2 n
and thus ?
}yn }1 ě n}y}2 ě n.
It follows that there exists α satisfying the required property. The existence of
β can be proved in a similar way.
Theorem 5. For a finite-dimensional vector space, all norms are equivalent.
Proof
See the proof in [11].
3.1. NORM AND NORMED SPACES 17
3.1.2 Examples of Norms

1. Norms in Rn
Let x P Rn . Examples of norms are in the following
n
ÿ
• }x}1 “ |xi |
i“1
• }x}8 “ max |xi |

1ďiďn
n
ÿ 1
• }x}p “ p |x1 |p q p , p ě 1
i“1
2. Norms in Rmˆn Let A P Rmˆn . Examples of norms in the following are

induced by }.}1 , }.}8 and }.}p , respectively, defined in Rn .
}Ax}1
• }A}1 “ max
}x}1 “1 }x}1
}Ax}8
• }A}8 “ max
}x}8 “1 }x}8
}Ax}p
• }A}p “ max ,p ě 1
}x}p “1 }x}p
Lemma 3. Let A P Rmˆm and }.} be an induced norm. Then,
}An } ď }A}n
Proof
Let xp‰ 0q P Rm . It follows that
}An x} ď }A}}An´1 x}
By mathematical induction, we obtain
}An x} ď }A}n }x}
Consequently,
}An x}
ď }A}n
}x}
That is,
}An x}
}An } “ sup ď }A}n
x‰0 }x}
3.1.3 Normed Spaces

Definition 38 (Normed Spaces). A vector space V equipped with a norm }.}
called a normed space.
Example 13. Show that the set of all polynomial functions is not a normed
vector space over R equipped with a norm }.}8 .
Solution Consider a polynomial function
N
ÿ
f pxq “ an xn
n“0
where an ě 0.
Consequently, if }f }8 exists, }f }8 can be evaluated in the following.
}f }8 “ sup |f pxq|
N
ÿ
“| an xn |
n“0
N
ÿ
ď| |an ||xn |
n“0
“8
Consequently, }f }8 does not exist. That is, the set of all polynomial functions
is not a normed vector space over R equipped with a norm. }.}8 .
Definition 39 (Bounded Sequences). A sequence txn u8 n“1 in a normed space
V is said to be bounded iff there exists M such that }xn } ă M for n “ 1, 2, 3, . . ..
Definition 40 (Cauchy Sequences). Let txn u8 n“1 be a sequence in a vector space
V equipped with norm }.}. Therefore, txn u8 n“1 is a Cauchy sequence iff, for all
ą 0. there exist N ą 0 such that, for all m ą N and n ą N , }xm ´ xn } ă
Definition 41 (Convergent and Divergent Sequences). Let txn u8 n“1 be a se-
quence in a vector space V equipped with norm }.}. Therefore, txn u8 n“1 is a
convergent sequence iff, for all ą 0, there exist N ą 0 such that, for all
n ą N , there exists a fixed x P V such that }x ´ xn } ă . Moreover, x is called
a limit of sequence txn u8n“1 . In addition, a sequence which is not convergent is
said to be a divergent sequence.
The limit mentioned in definition 41 can be proved to be unique in the
following theorem.
Theorem 6. If x is a limit of convergent sequence txn u8
n“1 then x is unique.
Proof. Suppose that x and x1 are both limits of convergent sequence txn u8
n“1 .
1
By definition, for all “ ą 0, there exists an integer N ą 0 such that
2
}xn ´ x} ă 1 and }xn ´ x1 } ă 1
for all integers n ą N .

This follows that for all 1 “ ą 0, there exists an integer N ą 0 such that
2

}x ´ x1 } ď }x ´ xn } ` }x1 ´ xn } ď 1 ` 1 “ ` “
2 2
for all integers n ą N . Consequently, x “ x1 . That is, if x is a limit of convergent
sequence txn u8 n“1 then x is unique.
The concept of convergence can be used to define the so-called “(Schauder)

basis” in the following
Definition 42 (Schauder Basis). If a normed space V contains a sequence
txn u8 8
n“1 with the property that there is a unique sequence tαn un“1 such that

ÿN
x ´ αn xn Ñ 0 as N Ñ 8

n“1

then txn u8
n“1 is called “(Schauder) basis”.
Theorem 7. If xn Ñ x as n Ñ 8, }xn } Ñ }x} as n Ñ 8

Proof
Since xn Ñ x as n Ñ 8, for all ą 0, there exists N such that }x ´ xn } ă
for all n ą N .
By using the fact that |}x} ´ }xn }| ď }x ´ xn }, we obtain that, for all ą 0,
there exists N such that |}x} ´ }xn }| ă for all n ą N . That is, }xn } Ñ }x} as
n Ñ 8.
Theorem 8. A convergent sequence txn u8
n“1 is bounded.
Proof
Suppose that txn u8n“1 is a convergent sequence. Therefore, by definition,
there exists N ą 0 such that
}x ´ xn } ă
for all ą 0
Since
}xn } ´ }x} ď }x ´ xn },
we obtain,for all ą 0, there exists N ą 0 such that, for all n ą N ,
}xn } ď }x ´ xn } ` }x} ď }x} `
Consequently, there exists N ą 0 such that
}xn | ď supt|x1 |, . . . , |xN |, |x| ` u
for all ą 0
That is, txn u8
n“1 is bounded.
3.1.4 Sets in Normed Spaces

Definition 43 (Open Sets). Let a nonempty S be a subset of a normed space
V. S is an open set iff for every x0 P S, there exists ą 0 such that Bpx, q “
tx | }x ´ x0 } ă u Ă S.
Definition 44 (Closed Sets). Let a nonempty S be a subset of a normed space
V. S is a closed set iff its complement S c “ V ´ S is open.
Definition 45 (Dense Sets). S is a dense set in a normed space V iff every
point in a normed space V is a limit point in S.
Definition 46 (Closure). Let S Ă R and S 1 be the set of all limit points of S
in a normed space V. Therefore, the closure of S, denoted by S, is defined as
S “ S Y S1
Definition 47 (Compact Sets). A subset S of a normed space V is compact iff,
every sequence in S has a convergent subsequence in S
Lemma 4. Compact sets are closed and bounded.
Proof
Suppose that S is a compact subset of a normed space V. By definition,
every sequence of x1 , x2 , . . . P S has a subsequence convergent to an element
x P S. Consequently, every sequence of x1 , x2 , . . . P S convergent to x P V has a
subsequence convergent to y P S. Hence, y “ x P S. That is, compact sets are
closed.
Suppose that a compact set S is unbounded. Therefore, there exists a di-
vergent sequence of x1 , x2 , . . . P S such that, for some N, M ą 0, }xn } ą M for
all n ą N . Consequently, there exists no convergent subsequence. That is,by
contradiction, S is bounded.
Theorem 9. Let a nonempty set S be a subset of a normed space V. Therefore,
S is closed iff every sequence of x1 , x2 , x3 , . . . P S convergent in V, has a limit
in S.
Proof
Suppose that S is a subset of a normed space V. At first show that if S is
closed, then every sequence of x1 , x2 , . . . P S convergent in V, has a limit in S.
Suppose that a sequence x1 , x2 , . . . P S convergent in V, has a limit in V but
not in S. Since S is closed, V ´ S is open. Therefore, for every x P S, there
exist Bpx, q Ă V ´ S. In other words, for all ą 0, there exists N ą 0 such
that }x ´ xn } ă for every x P S. Therefore, by contradiction, x P S.
Now show that if every sequence of x1 , x2 , . . . P S convergent in V, has
a limit in S, then S is closed. Suppose that S is not closed. Consequently,
V ´ S is not open and thus there exists x P V ´ S such that, for all ą 0,
Bpx, q “ ty | dpy, xq ă u contains elements of S. Consequently, we can
construct a sequence x1 , x2 , x3 , . . . P S such that xn P BpX, q for all ą 0.
Therefore, xn Ñ x where x P X ´ S. This proves by contradiction that if every
sequence of x1 , x2 , . . . P S convergent in X, has a limit in S, then S is closed.
3.1.5 Examples of Normed Spaces

Definition 48 (lp spaces, p ě 1). lp is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi |p ă 8u
i“1
with associated norm defined by

8
ÿ 1
}x}p “ p |xi |p q p
i“1
Particularly, for p “ 1,
Definition 49 (l1 space). l1 is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi | ă 8u
i“1

8
ÿ
}x}1 “ |xi |
i“1
and for p “ 2
Definition 50 (l2 space). l2 is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi |2 ă 8u
i“1

8
ÿ 1
}x}2 “ p |xi |2 q 2
i“1
Definition 51 (Spaces of Lebesgue Integrable Functions defined on R). Let

1 ď p ă 8. Spaces of Lebesgue Integrable Functions are defined on R as
" ˇż *
ˇ
p
L pRq “ f ˇˇ |f pxq|p dx ă 8
R
equipped with the norm

ˆż ˙ p1
p
}f }Lp pRq “ |f pxq| dx
R
For Let p “ 8. Spaces of Lebesgue Integrable Functions are defined on R as

" ˇ *
ˇ
L8 pRq “ f ˇˇess sup}f pxq} ă 8
}f }L8 pRq “ ess sup}f pxq}
It should be noted that the essential supremum of f is the smallest value that
bound f almost everywhere.
Definition 52 (Spaces of Lebesgue Integrable Functions defined on an open

subset Ω of Rn ). Let 1 ď p ă 8. Spaces of Lebesgue Integrable Functions are
defined on an open subset Ω of Rn as
" ˇż *
p
ˇ p
L pΩq “ f ˇ |f pxq| dx ă 8
ˇ
Ω

ˆż ˙ p1
p
}f } Lp pΩq “ |f pxq| dx
Ω
For p “ 8. Spaces of Lebesgue Integrable Functions are defined on an open

subset Ω of Rn as
" ˇ *
8
ˇ
L pΩq “ f ˇess sup}f pxq} ă 8
ˇ
}f }L8 pΩq “ ess sup}f pxq}
Definition 53 (Sobolev Space of order k). Let 1 ď p ă 8, Sobolev Spaces are

defined on an open subset Ω of Rn as
" ˇ *
ˇ
Wpk pΩq “ f P L pΩqˇˇDα f P Lp pΩq, |α| ď k
p
equipped with the Sobolev norm

¨ ˛
ÿ
}v}Wpk pΩq “ ˝ }Dα v}pLp pΩq ‚
|α|ďk
For p “ 8, Sobolev Spaces are defined on an open subset Ω of Rn as

" ˇ *
k
ˇ
W8 pΩq “ f P L pΩqˇˇDα f P L8 , |α| ď k
8
3.2. COMPLETION AND BANACH SPACES 23
3.2 Completion and Banach Spaces

Definition 54 (Completion). A normed space V is said to be complete iff every
Cauchy sequence txn u8
n“1 is a convergent sequence.
Definition 55 (Banach Space). A complete normed space is said to be a Banach

space.
Theorem 10. A closed subspace of a Banach space is itself a Banach space.
Proof
Since a closed subspace of a Banach space is a closed set. Therefore, by
definition, every convergent sequence in a closed subspace has a limit point in
itself. That is, a closed subspace of a Banach space is complete. Hence, it is a
Banach space.
3.3 Exercise
1. Show that lp , p ě 1 is a Banach space.
2. Show that l8 is a Banach space.
3. Show that Lp , p ě 1 is a Banach space.

4. Show that L8 pR is a Banach space.
Chapter 4
Inner Product Spaces and

Hilbert Spaces
4.1 Inner Products and Inner Product Spaces

Definition 56 (Inner Product). Let a vector space V be a vector space over
field1 F equipped with a mapping x., .y : V ˆ V Ñ F satisfying the following
properties
1. xx, xy ě 0 for x P V
2. xx, xy “ 0 iff x “ 0 for x P V
3. xx, yy “ xy, xy for x, y P V
4. xx ` y, zy “ xx, zy ` xy, zy for x, y, z P V
5. xαx, yy “ αxx, yy for α P F and x, y P V
Definition 57 (Inner Product Spaces). A vector space V equipped with an inner
product x., .y called a inner product space.
Example 14. Let A P Rnˆn be positive-definite2 , i.e., xT Ax ą 0 for all non-
zero vectors x P Rn . Show that x., .y where xx, yy “ xT Ay for all x, y P Rn is an
inner product.
Solution
1. xx, xy “ xT Ax ě 0 for x P Rn (by positive-definiteness of A)
2. xx, xy “ 0 iff x “ 0 for x P Rn (by positive-definiteness of A)
3. xx, yy “ xT Ay “ y T Ax “ xy, xy “ xy, xy for x, y P Rn
1 In this definition, the field F can be only R and C
2 see definition 107
25
26 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
4. xx ` y, zy “ px ` yqT Az “ xT Az ` y T Az “ xx, zy ` xy, zy for x, y, z P Rn
5. xαx, yy “ pαxqT Ay “ αxT Ay “ αxx, yy for α P R and x, y P Rn
That is,x., .y where xx, yy “ xT Ay for all x, y P Rn is an inner product.
Theorem 11 (Continuity of Inner Products). Let convergent sequences pxn q8 n“1

and pyn q8 n“1 in an inner product converge to x and y, respectively. Therefore
xxn , yn y Ñ xx, yy.
Proof
Since txn u8 8
n“1 and tyn un“1 be convergent sequences in H such that xn Ñ x
as n Ñ 8 and yn Ñ y as n Ñ 8, by triangular inequality, we obtain
|xx, yy ´ xxn , yn y| “ |xx, yy ´ xx, yn y ` xx, yn y ´ xxn , yn y|

ď |xx, yy ´ xx, yn y| ` |xx, yn y ´ xxn , yn y|
“ |xx, y ´ yn y| ` |xx ´ xn , yn y|
By using Cauchy-Schwarz Inequality, we obtain
|xx, y ´ yn y| ď }x}}y ´ yn } and |xx ´ xn , yn y| ď }x ´ xn }}yn }.
Consequently,
|xx, yy ´ xxn , yn y| ď }x}}y ´ yn } ` |xx ´ xn , yn y| ď }x ´ xn }}yn }
Since xn Ñ x and yn Ñ y as n Ñ 8, }yn } Ñ }y|, }x ´ xn } Ñ 0 and }y ´ yn } Ñ 0

as n Ñ 8
Consequently,
|xx, yy ´ xxn , yn y| Ñ 0 as n Ñ 8
That is, xxn , yn y Ñ xx, yy.
Theorem 12 (Induced Norm). We can define the so-called induced norm by

using the following equation.
1
}x} “ xx, xy 2
Proof Note that x is any vector in a vector space V equipped with an inner
product x¨, ÿ and }x} is defined by the following equation
1
}x} “ xx, xy 2
By the definition of the inner product defined earlier, we obtain

4.1. INNER PRODUCTS AND INNER PRODUCT SPACES 27
1. If x is a vector in the vector space V, then

1
}α x} “ xαx, αxy 2
1
“ pαx x, αxyq 2
´ ¯ 21
“ αxαx, xy
´ ¯ 12
“ ααxx, xy
1
“ |α| xx, xy 2
1
“ |α| xx, xy 2
“ |α| }x}
2. By the definition of the inner product, we obtain xx, xy ě 0. That is, }x} ě
0
3. Let x and y be any vectors in the vector space V. Therefore,
}x ` y}2 “ xx ` y, x ` yy
“ xx, x ` yy ` xy, x ` yy
“ xx ` y, xy ` xx ` y, yy
“ xx, xy ` xy, xy ` xx, yy ` xy, yy
“ xx, xy ` xx, yy ` xx, yy ` xy, yy
Since xx, xy “ xx, xy, xy, yy “ xy, yy and xx, yy “ xx, yy,
}x ` y}2 “ xx, xy ` xx, yy ` xx, yy ` xy, yy
Since 2< pxx, yyq “ xx, yy ` xx, yy, }x}2 “ xx, xy and }y}2 “ xy, yy,
}x ` y}2 “ }x}2 ` 2< pxx, yyq ` }y}2
Since < pxx, yyq ď |xx, yy|,
}x ` y}2 “ }x}2 ` 2< pxx, yyq ` }y}2
ď }x}2 ` 2 |xx, yy| ` }y}2
By Cauchy-Schwarz Inequality C.1, we obtain
}x ` y}2 ď }x}2 ` 2}x}}y} ` }y}2
2
“ p}x} ` }y}q
That is,
}x ` y} ď }x} ` }y}
1
4. Let x be a vector in the vector space V. Therefore, }x} “ xx, xy 2 “
0 iff x “ 0.
4.1.1 Examples of Inner Product Spaces

Definition 58 (l2 Space). l2 is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi |2 ă 8u
i“1
with associated inner product defined by

8
ÿ
xx, yy “ xi yi
i“1
8
ÿ
1 1
and induced norm defined by }x}2 “ xx, xy 2 “ p |xi |2 q 2 . It follows that
i“1
8
ÿ 1
}x}2 “ p |xi |2 q 2
i“1
Definition
“ ‰ 59 (Space of Square Lebesgue Integrable Functions defined“ ‰ on
a, b ). Spaces of square Lebesgue integrable functions are defined on a, b as
" ˇż b *
“ ‰
L2 p a, b q “ f ˇˇ |f pxq|2 dx ă 8
ˇ
a
equipped with the inner product

żb
xf, gy “ f pxqgpxq dx
a
and the induced norm

˜ż ¸ 12
b
2
}f }L2 pRq “ |f pxq| dx
a
Definition 60 (Space of Square Lebesgue Integrable Functions defined on R).

Spaces of square Lebesgue integrable functions are defined on R as
" ˇż *
ˇ
L2 pRq “ f ˇˇ |f pxq|2 dx ă 8
R

ż
xf, gy “ f pxqgpxq dx
R
and the induced norm

ˆż ˙ 21
2
}f }L2 pRq “ |f pxq| dx
R
4.2. COMPLETION AND HILBERT SPACES 29
Definition 61 (Hilbertian Sobolev Spaces). Hilbertian Sobolev space of order

k defined on R is defined as
" ˇ *
k k 2
ˇ
H “ W2 “ f P L pRqˇ}f }H k pRq ă 8
ˇ
where
¨ ˛ 21
ÿ
}f }H k “ ˝ }Dα f }2L2 pRq ‚
|α|ďk

¨ ˛ 12
ÿ ż
xf, gy “ ˝ pDα f qpxqpDα gqpxq dx‚
|α|ďk R
4.2 Completion and Hilbert Spaces

Definition 62 (Completion). An inner product space V is said to be complete
iff every Cauchy sequence txn u8
n“1 is a convergent sequence.
Definition 63 (Hilbert Spaces). A complete inner product space is said to be

a Hilbert space.
Hilbert questioned by himself the example as follows:
Example 15. Let V “ Cr0, 1s be a vector space over R equipped with the inner
product x., .y defined by
ż1
xf, gy “ f pxqgpxq dx.
0
Prove or disprove that V is a Hilbert space.
Solution Consider a sequence tfn , n P Nu where fn defined as follows:

$ “ 1 1
‰
’0,
’ ˆ ˙ x P 0, 2 ´ 2n
& 1 1
fn pxq “ n x ´ 1 1
` , x P p 21 ´ 2n ,2` 1
2n q
’
’ 2 2 “1 ‰
% 1
1, x P 2 ` 2n ,1
It follows that ż1
}fn ´ fm }2 “ pfn pxq ´ fm pxqq2 dx Ñ 0
0
as m, n Ñ 8
That is, tfn u is a Cauchy sequence.
If there is a function f P V such that fn Ñ f in }.}2 , that is,

ż1
pf pxq ´ fn pxqq2 dx Ñ 0 as n Ñ 8,
0
f can be defined by one of the following equations.

# “ ˘
0, x P 0, 21
f“ “ ‰,
1, x P 12 , 1
# “ ‰
0, x P 0, 21
f“ ` ‰
1, x P 21 , 1
and
# “ ‰
0, x P 0, 21
f“ “ ‰
1, x P 12 , 1
However, f , defined by one of the above equations, is not a continuous function.

It follows that the Cauchy sequence tfn u has no limit point in V. That is, V is
not a Hilbert space.
Theorem 13. The closure of a subspace of a Hilbert space is itself a Hilbert
space.
Proof
This follows theorem 10 by using the fact that every Hilbert space is a
Banach space.
Theorem 14. Every finite dimensional inner product space is a Hilbert space.
Proof
Consider a Cauchy sequence tvn u in a finite-dimensional inner product space
V. That is, for all real number ą 0, there exists an integer N ą 0 such that
}vn ´ vm } ă for all integers n, m ą N
For a vector v in an finite-dimensional inner product space V, v can be of the

form
N
ÿ
v“ αi ui
i“1
where α is the field (either R or C) associated with the inner product space
V and tu1 , u2 , . . . , uN u is an orthonormal basis of the finite-dimensional inner
product space V. Therefore, vn and vm can be written as
N
ÿ N
ÿ
vn “ αi,n ui and vm “ αi,m ui , respectively.
i“1 i“1
4.2. COMPLETION AND HILBERT SPACES 31
Consequently,

ÿN ÿN
}vn ´ vm } “ αi,n ui ´ αi,m ui

i“1 i“1

ÿN
“ pαi,n ui ´ αi qui,m

i“1
g
fN
fÿ
“ e |αi,n ´ αi,m |2 }ui }2
i“1
g
fN
fÿ
“ e |α i,n ´ αi,m |2
i“1
and
|αi,n ´ αi,m | ď }vn ´ vm } ď
That is, the sequence tai,n u8n“1 is also a Cauchy sequence. By using the fact

the field F (either R or C) is complete, that is, for some αi for all 1 “ ą 0,
N
there exists an integer N ą 0 such that

|αi,n ´ αi | ă 1 “
N
N
ÿ
By letting v “ αi ui , we obtain
i“1

ÿN N
ÿ
}vn ´ v} “ αi,n ui ´ αi ui

i“1 i“1
N
ÿ
ď pαi,n ui ´ αi qui,m
i“1
N
ÿ
“ |αi,n ui ´ αi | ui,m
i“1
N
ÿ
“ |αi,n ui ´ αi |
i“1
ă

where “ N 1 “ N
N

That is, for all 1 “ ą 0, there exists an integer N ą 0 such that
N
}vn ´ v} ă
Consequently, for all n ă N . That is, the Cauchy sequence tvn u8 n“1 is also a
convergent sequence. That is, the finite-dimensional inner product space V is
complete. This proves the fact that every finite dimensional inner product space
is a Hilbert space.
4.3 Orthogonality and Orthonormality

Definition 64 (Orthogonality and Orthonormality). Let x and y be vectors in
an inner product space associated with an inner product x., .y. Therefore, x and
y are said to be orthogonal iff xx, yy “ 0. In addition, x and y are said to be
orthonormal iff xx, yy “ 0 and }x} “ }y} “ 1.
Example 16. Let A “ raij s P Rnˆn and B “ rbij s P Rnˆn be symmetric and
antisymmetric. Show that A and B are orthogonal. In addition, the associated
inner product x., .y is defined by the following equation
n ÿ
ÿ n
xA, By “ aij bij
i“1 j“1
Solution
n ÿ
ÿ n
xA, By “ aij bij
i“1 j“1
ÿn ÿ n
“ aji bji
j“1 i“1
ÿn ÿ n
“ aji bji
i“1 j“1
(4.1)
By using the fact that A and B are symmetric and antisymmetric, respectively.
That is,
aji “ aij and bji “ ´bij
Consequently,
n ÿ
ÿ n n ÿ
ÿ n
aij bij “ ´ aij bij
i“1 j“1 i“1 j“1
and, then,
n ÿ
ÿ n
xA, By “ aij bij “ 0
i“1 j“1
Definition 65 (Orthogonal Complement). Let S ‰ H be a subset??? of a

Hilbert space H. The orthogonal complement of S denoted by S K is defined as
" ˇ *
K
ˇ
S “ x P Hˇˇxx, yy “ 0 for all y P S
4.3. ORTHOGONALITY AND ORTHONORMALITY 33
Theorem 15. If S ‰ H is a subset of a Hilbert space H, S K is a closed subspace

of H.
Proof
First of all, prove that S K is a subspace of H. Now, suppose that x, y P S K
and z P S. It follows that
xx, zy “ 0 and xy, zy “ 0
Moreover,
xαx ` βy, zy “ αxx, yy ` βxy, zy “ 0
for α, β P F
That is, S K is a subspace of H
Next, prove that S K is closed. Let txn u be a Cauchy sequence such that
xn Ñ x for some x P H. By using theorem 11 (Continuity of Inner peoduct), it
follows that
xxn , yy Ñ xx, yy
for any y P S
Moreover, since
xxn , yy “ 0,
it follows that
xx, yy “ 0
K
That is, S is closed.
Lemma 5. Let S ‰ H be a subspace of a Hilbert space H. Therefore,
S X S K “ t0u
Proof
Let x P S X S K . That is, x P S and x P S K . It follows that, by definition of
orthogonal complement,
xx, xy “ 0
and then
x“0
Therefore, S X S K “ t0u
Lemma 6. Let S ‰ H be a subset of a Hilbert space H. spantSu is dense in H
iff S K “ t0u
Proof
We first prove that if spantSu is dense in H, then S K “ t0u. Let x P S K .
Since spantSu is dense in H, that is, spantSu “ H, x P spantSu and there exists
a sequence txn u8
n“1 P spantSu such that lim xn “ x. In addition, since x P S
K
nÑ8
and S K KspantSu, we obtain
xx, xn y “ 0
By continuity of x., .y, we obtain

lim xx, xn y “ xx, xy “ 0
nÑ8
Consequently,
xx, xy “ }x}2 “ 0
That is, x “ 0. Therefore, S K “ t0u.
Conversely, we prove that if S K “ t0u, then spantSu is dense in H (or
spantSu “ H). Suppose that S K “ t0u. Let xKspantSu. It implies that xKS.
Consequently, x P S K and thus x “ 0 and thus spantSuK “ t0u. It follows that
K
spantSu “ t0u. Since spantSu is a closed subspace of H, we obtain
à K
H “ spantSu spantSu
K
Since spantSu “ t0u, we obtain
à
H “ spantSu t0u
That is, H “ spantSu

Theorem 16 (Pythagorean Law). Let x1 and x2 be orthogonal vectors in a
vector space V equipped with an inner product x., .y. In addition, }.} is defined
1
by }x} “ xx, xy 2 , x P V. Therefore,
}x1 ` x2 }2 “ }x1 }2 ` }x2 }2
Lemma 7 (Extension of Pythagorean Law). Let x1 , x2 , . . . , xn be orthogonal
vectors in a vector space V equipped with an inner product x., .y. In addition,
1
}.} is defined by }x} “ xx, xy 2 , x P V
n 2
ÿ ÿn
}x1 }2
x
i “
i“1 i“1
Theorem 17 (Paralellogram Law). Let x1 and x2 be vectors in a vector space

V over field R equipped with an inner product x., .y. In addition, }.} is defined
1
by }x} “ xx, xy 2 , x P V. Therefore,
}x1 ` x2 }2 ` }x1 ´ x2 }2 “ 2}x1 }2 ` 2}x2 }2
4.4 Best Approximation and Orthogonal Pro-

jection
Definition 66 (Best Approximation). Let S be a nonempty subset of an inner
product space V. v 1 and v belong to S and V, respectively. Therefore, v0 is a
best approximation of v from subset S iff v0 satisfies
}v ´ v0 } “ inf
1
}v ´ v 1 }
v PS
4.4. BEST APPROXIMATION AND ORTHOGONAL PROJECTION 35
Theorem 18 (Existence and Uniqueness of Best Approximation). Let S be a

closed convex subset of a Hilbert space H. For v P H, there exists a unique v0
in S such that
}v ´ v0 } “ inf
1
}v ´ v 1 }
v PS
Proof
At first, we need to prove the existence of best approximation. Let tvn u8
n“1
be a sequence in S such that
dn Ñ d (4.2)
or
d “ lim dn
nÑ8
where dn “ }v ´ vn } and d “ }v ´ v0 }
By setting δn “ vn ´ v, we obtain
dn “ }δn }
and
}δn ` δm } “ }vn ` vm ´ 2v}

v ` vm

“ 2 n ´ v

2
ě 2d
vn ` vm
Since S is convex, P S. Moreover, we have vn ´ vm “ δn ´ δm . Also,
2
by parallelogram law, we obtain
}vn ´ vm }2 “ }δn ´ δm }2
` ˘
“ ´}δn ` δm }2 ` 2 }δn }2 ` }δm }2
ď ´p2dq2 ` 2pd2n ` d2m q
and, by equation 4.2, dn Ñ d, it follows that tvn u8 n“1 is a cauchy sequence.

Since S is closed, tvn u8n“1 is a convergent sequence. That is, vn Ñ v0 . Since
v0 P S, it follows that }v ´ v0 } ě d. Also, by
}v ´ v0 } ě }v ´ vn } ` }vn ´ v0 } “ dn ` }vn ´ v0 } Ñ d
it implies }v ´ v0 } “ d.
Next, we need to prove the uniqueness of the best approximation. Suppose
that v0 , v01 P S satisfy
}v ´ v0 } “ d and }v ´ v01 } “ d
By parallelogram law,
}v0 ´ v01 }2 “ }pv0 ´ vq ´ pv01 ´ vq}2

“ 2}v0 ´ v}2 ` }pv01 ´ vq}2 ´ }pv0 ´ vq ` pv01 ´ vq}2
2
v0 ` v 1
“ 2d2 ` 2d2 ´ 4

0
2 ´ v

v0 ` v01
Since S is convex, P S. It follows that
2

1
v ´ v0 ` v0 ě d

2
That is,
}v0 ´ v01 } ď 0
However, by the definition of norms, we have
}v0 ´ v01 } ě 0
That is,
}v0 ´ v01 } “ 0 or v0 “ v01
Theorem 19 (Direct Sum). Let M be a closed subspace of a Hilbert space H.
Therefore, à K
H“M M
That is, for every x P H, x is uniquely represented by
x“y`z
where y P M and z P M K
Proof
It is trivial that, for x P M , x “ x ` 0. Suppose that x R H. Further suppose
that x can be both represented by
x “ y1 ` z1
and
x“y`z
where y, y 1 P M and z, z 1 P M K By recognizing that M is a closed subspace of
H, it follows that M is a closed convex subset of H. Since M is a closed convex
subset of H, by using theorem 18, we can choose y such that
}x ´ y} “ inf
1
}x ´ y 1 }
y PM
Since y, y 1 P M and z, z 1 P M K , it follows that y ´ y 1 P M and z ´ z 1 P M K and
xy ´ y 1 , z ´ z 1 y “ 0
It follows that, by using y ´ y 1 “ z ´ z 1 ,
xy ´ y 1 , y ´ y 1 y “ 0
Consequently,
y “ y1 and z “ z 1
That is, x can be uniquely represented by
x“y`z
where y P M and z P M K
Definition 67 (Orthogonal Projection). Let M be a closed subspace of a Hilbert
space H. The orthogonal projection of H onto M is the function PM : H Ñ H
such that for all x P H such that
x ´ PM pxqKM
Theorem 20. Let PM be the orthogonal projection of a Hilbert space H onto a

closed subspace M . It follows that
1. PM is linear
2
2. PM “ PM
Proof
1. Let x1 , x2 P H and α1 , α2 P F. It follows that
PM px1 q P M and PM px2 q P M.
Consequently, since M is a closed subspace of H,
α1 PM px1 q ` α2 PM px2 q P M.
and
pα1 x1 `α2 x2 q´pα1 PM px1 q`α2 PM px2 qq “ α1 px1 ´PM px1 qq`α1 px1 ´PM px1 qq
and
α1 px1 ´ PM px1 qq ` α1 px1 ´ PM px1 qq P M K
That is,
PM px1 ` x2 q “ PM px1 q ` PM px2 q
2. Suppose that x P H and PM pxq “ y. It follows that
x ´ PM pxq “ x ´ y and x ´ yKM.
Consequently,
2
PM pxq ´ PM pPM pxqq “ PM pxq ´ PM pxq “ y ´ PM pyqKM
and then
y ´ PM pyq P M K
2
Moreover, since y “ PM pxq P M and PM pxq “ PM pyq P M , PM pxq ´
2
PM pxq P M . That is,
2
PM pxq ´ PM pxq P M X M K
2
By lemma 5, PM pxq ´ PM pxq “ 0. Therefore, for all x P H,
2
PM pxq “ PM pxq
That is,
2
PM “ PM
Theorem 21. Let M be a closed subspace of a Hilbert space H and ten u be an
orthonormal basis of M . Therefore, the projection of x onto S is given by
ÿ
PM pxq “ xx, en yen
n
Proof
We can separate the proof into two cases:
1. M is of finite dimension
For x P H, ÿ ÿ
x “ xx, en yen ` px ´ xx, en yen q
n n
ÿ ÿ
It is obvious that xx, en yen P S and x ´ xx, en yen P M K
À K n n
Since M M , there uniquely exist y P M and y 1 P M K . That is,
x “ y ` y1
Consequently, ÿ
y“ xx, en yen
n
2. M is of infinite dimension
For x P H, ÿ ÿ
x“ xx, en yen ` px ´ xx, en yen q
nďN nďN
Since M is closed, as N Ñ 8,
ÿ
xx, en yen P M
nďN
À
Since H “ M M K , there uniquely exist y P M and y 1 P M K . That is,
x “ y ` y1
Consequently, ÿ
y“ xx, en yen
n
Definition 68 (Orthonormal Sequences). A sequence txn u in a Hilbert space

H is called an orthonormal sequence iff
#
1, m“n
xxm , xn y “ δmn “
0, m‰n
Definition 69 (Complete Orthonormal Sequence). An orthonormal sequence

txn u8
n“1 in an inner product space V is said to be complete iff
8
ÿ
x“ xx, xn yxn
n“1
Theorem 22 (Bessel’s Inequality). Let tφn u8

n“1 be an orthonormal sequence
in a Hilbert space H then
N
ÿ N
ÿ
}x ´ xx, φn yφn }2 “ }x}2 ´ |xx, φn y|2 (4.3)
n“1 n“1
and
8
ÿ
|xx, φn y|2 ď }x}2 for any x P H
n“1
Proof
N
ÿ
0 ď }x ´ xx, φn yφn }2
n“1
N
ÿ
“ }x}2 ´ 2Re pxx, φn yxφn , xyq ` |xx, φn y|2
n“1
N
ÿ
“ }x}2 ´ |xx, φn y|2
n“1
It follows that equation 4.3 holds and
N
ÿ
|xx, φn y|2 ď }x}2
n“1
By taking N Ñ 8, we obtain
8
ÿ
|xx, φn y|2 ď }x}2
n“1
4.5 Generalized Fourier Series

Theorem 23 (Generalized Fourier Series). Let tφn u8
n“1 be an orthonormal basis
of a Hilbert space H then
8
ÿ
x“ xx, φn yφn for any x P H
n“1
In particular, for f P L2 ra, bs, it follows that
Theorem 24 (Generalized Fourier Series). Let f be a function in L2 ra, bs and

2
tφi u8
n“1 be an orthonormal basis for L ra, bs. f can be described by the following
equation.
8
ÿ
f“ xf, φn yφn (4.4)
n“1
Theorem 25 (Parseval Theorem). Let tφn u8

n“1 be an orthonormal sequence of
a Hilbert space H is complete iff
8
ÿ
}x}2 “ |xx, φn y|2 for any x P H
n“1
Proof
For x P H, by using equation 4.3, we obtain
N
ÿ N
ÿ
}x ´ xx, φn yφn }2 “ }x}2 ´ |xx, φn y|2
n“1 n“1
for every positive integer N

If txn u8
n“1 is a complete sequence, then
2
ÿN
x ´ xx, xn yxn Ñ 0 as N Ñ8

n“1

and, hence,
N
ÿ
|xx, φn y|2 Ñ }x}2 as N Ñ8
n“1
It follows that
8
ÿ
n“1
Conversely, if
8
ÿ
n“1
4.6. GRAM-SCHMIDT ORTHONONORMALIZATION OR PROCESS 41
then
N
ÿ
}x}2 ´ |xx, φn y|2 Ñ 0 as N Ñ8
n“1
2
ÿN ÿN
Since xx, φn yφn “ |xx, φn y|2 , we have

n“1 n“1
2
ÿN
2
xx, φn yφn Ñ 0 as N Ñ8

}x} ´
n“1
and thus 2
ÿN
x ´ xx, xn yxn Ñ 0 as N Ñ8

n“1

It follows that txn u8

n“1 is a complete sequence.
4.6 Gram-Schmidt Orthononormalization or Pro-

cess
Let tu1 , u2 , u3 , . . . , un u be a basis. We can obtain an orthonormal basis tq1 , q2 , q3 , . . . , qn u
as follows:
u1
q1 “
}u1 }
u2 ´ xu2 , q1 yq1
q2 “
}u2 ´ xu2 , q1 yq1 }
..
.
n´1
ÿ
un ´ xu2 , qi yqi
i“1
qn “ n´1
ÿ
}un ´ xu2 , qi yqi }
i“1
4.7 Exercise
1. Show that l2 is a Hilbert space.
2. Show that L2 pRq is a Hilbert space.
Chapter 5
Operators and Functionals
5.1 Operators
Definition 70 (Operators). An operator is a mapping of a vector space1 into
a vector space2 .
Definition 71 (Domain and Range). Let T : V Ñ W be an operator. The
domain of T denoted by DpT q is defined as
DpT q “ tx | T pxq is defined u
That is,
DpT q “ V
The range of T denoted by RpT q is defined as
RpT q “ tT pxq | T pxq is defined u
It should be noted that RpT q can possibly be W or a subset of W
Definition 72 (Linear Operators). Let T : V Ñ W be an operator (mapping
from a vector space V into a vector space W). T is said to be a linear operator
iff
1. T pu ` vq “ T puq ` T pvq for u, v P V
2. T pαuq “ αT puq for α P F and u P V
Definition 73 (Isomorphism). Let T : V Ñ W be a bijective linear opera-
tor where V and W are inner product spaces over the same field. T is called
isomorphism iff
xT x, T yyW “ xx, yyV for all x, y P V and T x, T y P W
where x., .yV and x., .yW are inner products over V and W, respectively.
1 in particular, a normed space or inner product space
2 in particular, a normed space or inner product space
43
44 CHAPTER 5. OPERATORS AND FUNCTIONALS
Example 17. Let T : V Ñ V1 where V onto V1 are R2 over the same field R.
T is defined as
ˆ„ ˙ „ „ 
x cos θ ´ sin θ x
T “
y sin θ cos θ y
and inner product x., .y is defined as
B„  „ F
x1 x
, 2 “ x1 x2 ` y1 y2 .
y1 y2
Show that T is an isomorphism.
Solution
ˆ„ ˙ „ „ 
x1 cos θ ´ sin θ
x1
T “
y1 sin θ cos θ
y1
„ 
x cos θ ´ y1 sin θ
“ 1
x1 sin θ ` y1 cos θ
and
ˆ„ ˙ „ „ 
x2 cos θ ´ sin θ x2
T “
y2 sin θ cos θ y2
„ 
x2 cos θ ´ y2 sin θ
“
x2 sin θ ` y2 cos θ
Consequently,
B ˆ„ ˙ ˆ„ ˙F B„ „  „  „ F
x1 x2 cos θ ´ sin θ x1 cos θ ´ sin θ x2
T ,T “ ,
y1 y2 sin θ cos θ y1 sin θ cos θ y2
B„  „ F
x1 cos θ ´ y1 sin θ x cos θ ´ y2 sin θ
“ , 2
x1 sin θ ` y1 cos θ x2 sin θ ` y2 cos θ
“ x1 y1 pcos2 θ ` sin2 θq ` x2 y2 pcos2 θ ` sin2 θq
By using the fact that cos2 θ ` sin2 θ “ 1, we obtain

B ˆ„ ˙ ˆ„ ˙F
x1 x2
T ,T “ x1 y1 ` x2 y2
y1 y2
In addition,
B„  „ F
x1 x
, 2 “ x1 y1 ` x2 y2
y1 y2
Therefore,
B„  „ F B ˆ„ ˙ ˆ„ ˙F
x1 x x1 x2
, 2 “ T ,T
y1 y2 y1 y2
5.1. OPERATORS 45
Definition 74 (Bounded Linear Operators). Let V and W are normed spaces

equipped with }.}V and }.}W , respectively, and T : DpT q Ñ W a linear operator
where DpT q is a subset of the normed space V. The operator T is a bounded
linear operator iff there exists c ą 0 such that
}T pxq}W ď c}x}V (5.1)
for some c ą 0

It follows that, by setting δ “ ,
c
We can show that if we choose c in equation 5.1 to be the supremum of
}T pxq}
the operator T . That is, }T } “ sup , it will satisfy the definition of
xPDpT q }x}
x‰0
norms3 . That is, we can define a norm of a bounded linear operator in the
following
Definition 75 (Norm of a Bounded Linear Operator). Let T : DpT q Ñ W. }T }
can be defined in the following
}T pxq}
}T } “ sup
xPDpT q }x}
x‰0
}T pxq}
Example 18. Let T be a bounded linear operator. Prove that }T } “ sup
xPDpT q }x}
x‰0
satisfies the definition of norms.
Solution We can show that }T } satisfies the definition of norms as follows:
1. }αT } “ |α|}T } for all α P F and x P V By using the fact that
|αT pxq| “ |α||T pxq|
Consequently,
|αT pxq| |T pxq|
“ |α| ,x ‰ 0
|x| |x|
2. (Non-negativity) }T } ě 0 for all T P V
3. (Triangular Inequality) }x ` y} ď }x} ` }y} for all x, y P V
4. (Non-degeneracy) }x} “ 0 iff x “ 0
Definition 76 (Continuous Operator). Let T : V Ñ W where V and W are
normed spaces equipped with }.}V and }.}W , respectively. Then, T is continuous
at x “ x0 iff, for all ą 0, there exists δ ą 0 such that
}x ´ x0 }V ă δ Ñ }T pxq ´ T px0 q}W ă
Moreover, if f is continuous at every x0 P V, then f is said to be continuous
3 The proof is left in example 18
Definition 77 (Uniformly Continuous Operator). Let T : V Ñ W where V

and W are normed spaces equipped with }.}V and }.}W , respectively. Then, T is
continuous iff, for all ą 0, there exists δ ą 0 such that
}x ´ y}V ă δ Ñ }T pxq ´ T pyq}W ă
for all x, y P V
Theorem 26. A bounded linear operator is continuous.
Proof
Let T : V Ñ W where V and W are normed spaces equipped with }.}V and
}.}W , respectively be a linear operator. It follows that
}T pxq ´ T px0 q} “ }T px ´ x0 q} ď c}x ´ x0 }V

and then, by setting δ “ , it implies that, for all ą 0, there exists δ ą 0 such
c
that
}x ´ x0 }V ă δ Ñ }T pxq ´ T px0 q}W ă
for all x0 P V
That is, T is continuous.
Theorem 27. Let an operator T : V Ñ W where V and W are normed spaces
equipped with norms }.}V and }.}W , respectively, be continuous at a fixed x
T p lim xn q “ lim T pxn q

nÑ8 nÑ8
Proof
Let a sequence txn u8 n“1 converge to a fixed x. That is, x “ nÑ8
lim xn . There-
fore, for all ą 0, there exists δ ą 0 such that
}y ´ x}V ă δ Ñ }T pyq ´ T pxq}W ă
Let N ą 0 be such that

}xn ´ x}V ă δ
for all n ą N then
}T pyq ´ T pxq}W ă
for all n ą N
It follows that
T p lim xn q “ lim T pxn q
nÑ8 nÑ8
Definition 78 (Zero Operators or Null Operators). A zero operator or null

operator 0 : V Ñ W is defined as
0pxq “ 0
for all x P V
5.2. KERNELS, NULL SPACES AND RANGE SPACES 47
Definition 79 (Identity Operators). An identity operator I : V Ñ V is defined

as
Ipxq “ x
for all x P V
Definition 80 (Inverse Operators). Let T : DpT q Ñ RpT q be an operator from

DpT q onto RpT q. An operator T ´1 defined on RpT q is called an inverse of T
iff
T T ´1 pxq “ x for all x P RpT q
and
T ´1 T pxq “ x for all x P DpT q
5.2 Kernels, Null Spaces and Range Spaces

Definition 81 (Kernels). Let T : V Ñ W be a mapping from a vector space V
into W. The kernel of T is defined as
kerpT q “ tv P V | T pvq “ 0W u
Example 19. Let T be a linear operator Prove that kerpT q is a subspace of T
Solution
1. Let u, v P kerpT q. We obtain
T puq “ 0 and T pvq “ 0.
Consequently, by linearity, we obtain
T pu ` vq “ T puq ` T pvq “ 0 ` 0 “ 0
Therefore, u ` v P kerpT q.
2. Let α P F and v P kerpT q. By linearity, we obtain
T pαuq “ αT puq “ α0 “ 0
Therefore, αu P kerpT q
That is, kerpT q is a subspace of T
Definition 82 (Null Space). Let T : V Ñ W be a linear operator. The null

space4 of T denoted by N pT q is defined as
N pT q “ tx | T pxq “ 0u
4 For a linear operator T , the null space of T is actually the kernel of T . It follows that
N pT q is a subspace of V
Definition 83 (Range Space). Let T : V Ñ W be a linear operator. The range

space of T denoted by RpT q is defined as
RpT q “ tT pxq | x P Vu
Theorem 28. The range space RpT q of a linear operator T : V Ñ W is a

subspace of W (a vector space with the same field)
Proof
Let y1 , y2 P RpT q Ă W. It implies that there exists x1 , x2 P V such that
y1 “ T px1 q and y2 “ T px2 q
Consequently,
α1 y1 ` α2 y2 “ α1 T px1 q ` α2 T px2 q “ T pα1 x1 ` α2 x2 q
for α1 , α2 P F
Since α1 x1 ` α2 x2 P V, α1 y1 ` α2 y2 “ T pα1 x1 ` α2 x2 q P RpT q. That is, RpT q
is a subspace of W.
Theorem 29. Let T : H Ñ H be a bounded linear operator. It follows that
N pT ‹ qK “ RpT q and N pT q “ RpT ‹ qK
Proof
Let x P RpT q. It follows that there exists y P H such that
x “ T pyq
For all z P N pT ‹ q,
xy, T ‹ pzqy “ 0 and xy, T ‹ pzqy “ xT pyq, zy “ xx, zy
That is,
xx, zy “ 0
Consequently, RpT q Ă pN pT ‹ qqK . Since, by theorem 15, N pT ‹ q is closed,
RpT q Ă pN pT ‹ qqK
Now, let x P pRpT qqK . Then, for all y P H, we have
xT pyq, xy “ 0 and xT pyq, xy “ xy, T ‹ pxqy
Therefore,
xy, T ‹ pxqy “ 0
for all y P H
It follows that
T ‹ pxq “ 0
5.3. FUNCTIONALS 49
and
pRpT qqK Ă N pT ‹ q
Consequently, by taking orthogonal complement,
pN pT ‹ qqK Ă pRpT qqKK “ RpT q
Since RpT q Ă pN pT ‹ qqK and pN pT ‹ qqK Ă RpT q, it implies that
N pT ‹ qK “ RpT q
and thus
N pT q “ RpT ‹ qK
5.3 Functionals
Definition 84 (Functionals). A functional is an operator whose range lies in
R or C
Definition 85 (Linear Functionals). A linear functional is a linear operator

whose range lies in R or C
Definition 86 (Bounded Functionals). A bounded functional is a bounded op-

erator whose range lies in R or C
Definition 87 (Sesquilinear Form). Let s : V ˆ W Ñ F where V and W are

vector spaces over the same field F (R or C) be a mapping such that
1. spx1 ` x2 , yq “ spx1 , yq ` spx2 , yq
2. spx, y1 ` y2 q “ spx, y1 q ` spx, y2 q
3. spαx, yq “ αspx, yq
4. spx, βyq “ βspx, yq
for all x P V, y P W and α, β P F

In addition, if there exists c P R such that |spx, yq| ď c}x}}y}, s is said to be
bounded and }.} is defined by
spx, yq spx, yq
}s} “ sup “ sup
x‰0 }x}}y} }x}“1 }x}}y}
y‰0 }y}“1
Theorem 30 (Riez Representation Theorem for Bounded Linear functionals).

Suppose that f is a bounded linear functional on a Hilbert space H. There exists
a unique z P H such that f pxq “ xx, zy. Also, }f } “ }z} where }f } “ sup |f pxq|.
}x}“1
Proof
First of all, prove that there exists a z P H such that f pxq “ xx, zy. It is
trivial for a bounded linear functional f pxq “ 0 for all x P H that the functional
f can be of the form
f pxq “ xx, zy
where z “ 0
For a bounded linear functional f such that f pxq ‰ 0 for some x P H, RpT q Ă H
contains a z0 ‰ 0. Let
y “ f pxqz0 ´ f pz0 qx
We obtain
f pyq “ f pf pxqz0 ´ f pz0 qxq “ f pxqf pz0 q ´ f pz0 qf pxq “ 0
It implies that y P N pf q. Moreover, since RpT q “ N pT qK , z0 P N pT qK . It

follows that
xy, z0 y “ xf pxqz0 ´ f pz0 qx, z0 y “ f pxqxz0 , z0 y ´ f pz0 qxx, z0 y “ 0
Since z0 ‰ 0, xz0 , z0 y ‰ 0. It follows that
xx, z0 y f pz0 q
f pxq “ f pz0 q “ xx, z0 y
xz0 , z0 y xz0 , z0 y
f pz0 q
By setting z “ z0 , we obtain
xz0 , z0 y
f pxq “ xx, zy
Secondly, prove that z is unique. Suppose that there exist z and z 1 such that
f pxq “ xx, zy and f pxq “ xx, z 1 y
It implies that
xx, z ´ z 1 y “ 0
It follows that, by setting x “ z ´ z 1 , we obtain
xz ´ z 1 , z ´ z 1 y “ 0
and then
z “ z1
That is, by contradiction, z is unique.
Thirdly, }f } “ }z} where }f } “ sup |f pxq|. It is trivial for f “ 0. Suppose
}x}“1
that f ‰ 0. We have
f pxq “ xx, zy
By setting x “ z,
|xz, zy| “ }z}2 “ |f pzq|
5.3. FUNCTIONALS 51
Since we suppose that f ‰ 0, z ‰ 0. It implies that

z
}z}2 “ |f pzq| “ |f p q|}z} ď }f }}z}
}z}
and then
}f } ě }z}
Moreover, by Cauchy-Schwarz inequality,
|f pxq| “ |xx, zy| ď }x}}z}
and then
}f } ď }z}
That is,
}f } “ }z}
Theorem 31 (Riez Representation Theorem for Sesquilinear Forms). Let s :
V ˆ W Ñ F be a bounded sesquilinear form. Then s is of the form
spx, yq “ xSx, yy
where S is unique and equipped with the norm
}S} “ }s}
Proof
For a fixed x, we can define a functional f as
f pyq “ spx, yq
By using definition of sesquilinear forms, spx, y1 ` y2 q “ spx, y1 q ` spx, y2 q and

spx, βyq “ βspx, yq, we have
f pαy1 ` βy2 q “ spx, αy1 ` βy2 q “ αspx, y1 q ` βspx, y2 q “ αf py1 q ` βf py2 q
It follows that f is linear. Since s is bounded, it implies that f is a bounded

linear functional. By using Riez representation theorem for bounded linear
functionals, f can be of the form
f pyq “ xy, zy
That is,
spx, yq “ xy, zy
or
spx, yq “ xz, yy
where z P W is unique but z depends on x.
Suppose that z “ Sx. We have
spx, yq “ xSx, yy
We can first prove that S is linear in the following.

Consider that
xSpαx1 ` βx2 q, yy “ spαx1 ` βx2 , yq
By definition of sesquilinear forms,
spαx1 ` βx2 , yq “ spαx1 , yq ` spβx2 , yq

“ αspx1 , yq ` βspx2 , yq
It implies that S is linear.

Next, by setting y “ Sx, we prove that S is bounded in the following
|spx, yq|
|spx, y| “ sup
x‰0 }x}}y}
y‰0
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
|xSx, Sxy|
ě sup
x‰0 }x}}Sx}
}Sx}
“ sup
x‰0 }x}
“ }S}
It follows that S is linear and bounded.

We next prove that }S} “ }s}. It is trivial for the case where S = 0. Now,
suppose that }S ‰ }0. By setting y “ Sx, we have
|spx, yq|
}s} “ sup
x‰0 }x}}y}
y‰0
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
||xSx, Sxy|
ě sup
x‰0 }x}}Sx}
Sx‰0
|}Sx}
“ sup
x‰0 }x}
“ sup }Sx}
}x}‰1
“ }S}
5.4. HILBERT-ADJOINT OPERATORS 53
In addition, by Cauchy-Schwarz inequality,

|spx, yq|
}s} “ sup
x‰0 }x}}y}
y‰0
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
}Sx}}y}
ď sup
x‰0 }x}}y}
y‰0
}Sx}
“ sup
x‰0 }x}
“ sup }Sx}
}x}“1
It follows that
}s} “ }S}
Finally, we prove that S is unique. Suppose that there exists a bounded linear
operator T : V Ñ W such that
spx, yq “ xSx, yy “ xT x, yy
for all x P V and y P W
It follows that
xSx ´ T x, yy “ 0
and then
Sx “ T x
for all x P V
It implies that S “ T and then S is unique.
5.4 Hilbert-Adjoint Operators

Suppose that L : H Ñ H is bounded linear operator on a Hilbert Space H. By
using Riez representation theorem (theorem 30), for a bounded linear functional
f on H, there exists a unique y0 P H such that f can be represented by
f pxq “ xLx, y0 y for all x P H
This implies that, for a fixed y P H, a bounded linear functional fy “ xLx, yy
can be represented by
fy pxq “ xx, zy
for a unique z P H.
If we define an operator L‹ such that z “ L‹ y, we obtain
xLx, yy “ xx, L‹ yy
This leads to the following definition.
Definition 88 (Hilbert - Adjoint Operators5 ). Let L : H1 Ñ H2 be a bounded

linear operator where H1 and H2 are Hilbert spaces. L‹ is called the Hilbert-
adjoint operator of L iff
xLx, yy “ xx, L‹ yy for x P H2 and y P H1
Theorem 32 (Existence and Uniqueness of Adjoint Operators). The Hilbert-
adjoint operator T ‹ of a bounded linear operator T exists and is unique. More-
over,
}T } “ }T ‹ }
Proof
Let
spx, yq “ xx, T yy
equipped with norm
x y
}s} “ sup |sp , q| “ sup |spx, yq|
x‰0 }x} }y} }x}“1
y‰0 }y}“1
By Cauchy-Schwarz inequality,
|spx, yq| “ |xx, T yy| ď }x}}T y}
By using the fact that
}T } ď }T }}y},
it implies that
|spx, yq| ď }T }}x}}y}
and then
}s} ď }T }
Moreover, by replacing x with T y
x y
}s} “ sup |sp , q|
x‰0 }x} }y}
y‰0
1
“ sup |xx, T yy|
x‰0 }x}}y}
y‰0
1
ě sup |xT y, T yy|
T y‰0 }T y}}y}
y‰0
1
“ sup }T y}
y‰0 }y}
“ sup }T y}
}y}“1
“ }T }
5 sometimes called ”Adjoint operators”
5.4. HILBERT-ADJOINT OPERATORS 55
It follows that
}s} “ }T }
It follows directly from definition 88 that
Theorem 33. Let L1 : V Ñ W and L2 : V Ñ W be
1. pL1 ` L2 q‹ “ L‹1 ` L‹2
2. pL‹ q‹ “ L
3. pαLq‹ “ αL‹
4. pL1 L2 q‹ “ L‹2 L‹1
Proof
1.
xpL1 ` L2 qx, yy “ xx, pL1 ` L2 q‹ yy

“ xx, L‹1 yy ` xx, L‹2 yy
(5.2)
That is,
pL1 ` L2 q‹ “ L‹1 ` L‹2
2.
xLx, yy “ xx, L‹ yy
“ xL‹ y, xy
“ xy, pL‹ q‹ xy
“ xpL‹ q‹ x, yy
It follows that
xLx, yy ´ xpL‹ q‹ x, yy “ 0
and
xpL‹ q‹ x ´ Lx, yy ´ x, yy “ 0
It implies that
pL‹ q‹ x ´ Lx “ 0 or pL‹ q‹ x “ Lx (5.3)
That is,
pL‹ q‹ “ L
3.
xpαLqx, yy “ αxLx, yy
“ αxx, L‹ yy
“ αxL‹ y, xy
“ αxL‹ y, xy
“ xαL‹ y, xy
“ xx, αL‹ yy
(5.4)
Moreover, we have
xpαLqx, yy “ xx, pαLq‹ yy (5.5)
It implies that
xx, αL‹ yy “ xx, pαLq‹ yy
and then
xx, pαLq‹ y ´ αL‹ yy “ 0 (5.6)
It follows that
pαLq‹ y ´ αL‹ y “ 0
or
pαLq‹ “ αL‹
4.
xpL1 L2 qx, yy “ xpL1 pL2 xq, yy
“ xL2 x, L‹1 yy
“ xx, L‹2 L‹1 yy
(5.7)
Moreover, we have
xpL1 L2 qx, yy “ xx, pL1 L2 q‹ yy
It implies that
xx, pL1 L2 q‹ y ´ L‹2 L‹1 yy “ 0
(5.8)
It follows that
pL1 L2 q‹ y “ L‹2 L‹1 y
or
pL1 L2 q‹ “ L‹2 L‹1
5.5. SELF-ADJOINT OPERATORS AND FORMALLY SELF-ADJOINT OPERATORS57
5.5 Self-Adjoint Operators and Formally Self-

Adjoint Operators
Definition 89 (Self-Adjoint Operators). Let L : H Ñ H be a bounded linear
operator. The adjoint operator L‹ of L is the self-adjoint operator of L iff
L “ L‹ . That is,
xLx, yy “ xx, Lyy for x, y P H
By definition 89, for a self-adjoint operator L, L “ L‹ and DpLq “ DpL‹ q.
Definition 90 (Formally Self-Adjoint Operators). A differential operator L is
formally self-adjoint if L “ L‹ .
Example 20. Let L be a differential operator defined by
d2 upxq
Lupxq “ .
dx2
Consider a boundary value problem
Lupxq “ 0
with upaq “ u1 paq “ 0. Determine whether this operator is self-adjoint or for-
mally self-adjoint.
Solution
Consider that
B F
d2 u
xLu, vy “ ,v
dx2
żb 2
d upxq
“ vpxq dx
a dx2
ż x“b
“ vpxq du1 pxq
x“a
ż x“b
“ ‰b
“ vpxqu1 pxq a
´ u1 pxqdvpxq
x“a
By noting that dvpxq “ v 1 dx, we obtain

żb
“ 1
‰b
xLu, vy “ vpxqu pxq a
´ u1 pxqv 1 pxq dx (5.9)
a

ż x“b ż x“b
“ ‰b
u1 pxqv 1 pxq dx “ upxqv 1 pxq a ´ upxq dv 1 pxq
x“a x“a
1 2
By noting that dv pxq “ v dx, we obtain
żb ż x“b
“ ‰b
u1 pxqv 1 pxq dx “ upxqv 1 pxq a ´ upxqv 2 pxq dx (5.10)
a x“a
By using equations 5.9 and 5.10, we obtain

ż x“b
“ ‰b
xLu, vy “ vpxqu1 pxq ´ upxqv 1 pxq a ` upxqv 2 pxq dx
x“a
That is,
“ ‰b
xLu, vy “ vpxqu1 pxq ´ upxqv 1 pxq a ` xu, Lvy
b
It follows that rvpxqu1 pxq ´ upxqv 1 pxqsa “ 0 for all u P L iff
vpbq “ 0 and ´ v 1 pbq “ 0
That is, L is formally self-adjoint but not self-adjoint since DpLq ‰ DpL‹ q
Example 21. Let L be a differential operator defined by
dupxq
Lupxq “ .
dx
Consider a boundary value problem
Lupxq “ 0
with upaq “ 0. Determine whether this operator is self-adjoint or formally self-

adjoint.
Solution
Consider that
B F
du
xLu, vy “ ,v
dx
żb
dupxq
“ vpxq dx
a dx
ż x“b
“ vpxq dupxq
x“a
ż x“b
b
“ rvpxqupxqsa ´ upxqdvpxq
x“a
That is,
b
xLu, vy “ rvpxqupxqsa ´ xu, Lvy (5.11)
Equation 5.11 show that it is impossible that
xLu, vy “ xu, Lvy
for all u P DpLq. That is, L is neither self-adjoint nor formally self-adjoint.
5.6. POSITIVE OPERATORS 59
5.6 Positive Operators

Definition 91 (Positive Operators). Let H be a Hilbert space. An operator T
on H is said to be a positive operator iff T is self-adjoint and xT x, xy ě 0 for
all x P H. Also, if T is positive, we write T ě 0.
Definition 92. Let H be a Hilbert space. If positive operator T1 and T2 satisfies
xT1 x, xy “ xT2 x, xy
for all x P H, we write

T1 ě T2
By definitions 91 and 92, if T1 ´ T2 ě 0, it implies that
xpT1 ´ T2 qx, xy ě 0
and thus
xT1 x, xy ě xT2 x, xy
Therefore, we write T1 ě T2
Theorem 34. Let H be a Hilbert space and T be a bounded operator on H. It

follows that
1. T ‹ T is positive.
2. T T ‹ is positive.
Proof
Let x P H. It follows that
1. xT ‹ T x, xy “ xT x, T xy “ }T x}2 ě 0
That is, T ‹ T is positive.
2. xT T ‹ x, xy “ xT ‹ x, T ‹ xy “ }T ‹ x}2 ě 0
That is, T T ‹ is positive.
5.7 Compact Operators

Definition 93 (Compact Operators). An operator T on a normed space V is
called a compact operator6 iff, for every bounded sequence txn u8
n“1 in V, the
sequence tT pxn qu8
n“1 contains a convergence subsequence.
Theorem 35. Let T : H1 Ñ H2 be a compact linear operator where H1 and H2

are Hilbert spaces. Therefore, T is bounded and continuous.
6 also called a completely continuous operator
Proof
Let S “ Bp0, q “ tx | }x} ă u, ą 0. Since Bp0, q is bounded and T
is compact, there exists a sequence tT pxn qu8
n“1 which contains a convergence
subsequence. Suppose that T pSq is unbounded. Then, there exists a strictly
monotone and unbounded sequence tT pxn qu8 8
n“1 . It implies that tT pxn qun“1
does not contain a convergence subsequence. By contradiction, for all ą 0
T pSq is bounded. That is, T is bounded. It follows that, by theorem 26, T is
continuous.
Theorem 36. Let tTn u8 n“1 be a convergent sequence of compact operators on
a Banach Space V. If Tn Ñ T as n Ñ 8, T is also a compact operator.
Proof
Let txn u8n“1 be a bounded sequence in V. Since T1 is a compact operator
on a Banach Space V, it implies that there exists a subsequence tx1,n u8 n“1 of
txn u8
n“1 such that tT px qu
1 1,n n“1
8
is a convergent subsequence. Moreover, since
tTn u8
n“1 is a convergent sequence of compact operators on a Banach Space V,
it implies that there exists a subsequence txk,n u8
n“1 , k “ 2, 3, . . . of tx u 8
k´1,n n“1
such that tTk pxk,n qu8
n“1 is a convergent subsequence. If we let
xan “ xn,n ,
it follows that tTk pxan qu8 `

n“1 is a convergence sequence for every k P I .
8
Furthermore, we will prove that tT pxan qun“1 is a convergent sequence. Let
}xn } ď M . Since Tn Ñ T as n Ñ 8, for all ą 0, there exist k 1 such that

}Tk pxan q ´ Tk pxan q} ď }T ´ Tk }}xan } ď }T ´ Tk }M ă
3
Since tTk pxan qu8n“1 is convergent and thus cauchy, for all ą 0, there exists k
1
such that

}T pxan q ´ Tk pxan q} ď
3
for all n ą k 1 .
It implies that, for all ą 0, there exist k such that
}T pxan q ´ T pxam q} ď }T pxan q ´ Tk pxan q} ` }Tk pxan q ´ Tk pxam q} ` }Tk pxam q ´ T pxam q}

ă ` `
3 3 3
“
for all n, m ą k
That is, tT pxan qu8
n“1 is also convergent and thus T is also compact.
Theorem 37. Let T : V Ñ W be a linear operator where V and W are normed

spaces. Therefore,
1. If dimpVq is finite, T is compact.
2. If T is bounded and dimpRpT qq is finite, T is compact.
5.7. COMPACT OPERATORS 61
Proof
1. Since dimpVq is finite, for a bounded sequence txn u8
n“1 , xn can be written
in terms of e1 , e2 , . . . , eN as
N
ÿ
xn “ αi,n ei
i“1
where te1 , e2 , . . . , eN u is a basis of V.

Since txn u8n“1 is bounded, there exists M such that
N
ÿ
|αi,n |}ei } ď } αi,n ei } ď }x} ă M
i“1
for all n “ 1, 2, . . ., it follows that tαi,n u8

n“1 is also bounded. By Bolzano-
Weierstrass theorem (theorem 1), tαi,n u8 n“1 has a convergence subse-
quence tαi,an u8n“1 . Let
ÿN
x“ αi ei
i“1
where αi,an Ñ αi as n Ñ 8
Consequently, by linearity,
N
ÿ N
ÿ
}T pxq ´ T pxan q} “ } αi T pei q ´ αi,an T pei q}
i“1 i“1
N
ÿ
“} pαi ´ αi,an qT pei q}
i“1
N
ÿ
ď| pαi ´ αi,an q}T pei q}
i“1
N
ÿ
ď |αi ´ αi,an |}T pei q}
i“1
Since αi,an Ñ αi as n Ñ 8, there exists N ą 0 such that, for all 1 ą 0,

|αi ´ αi,an | ă 1
for all n ą N 1
It implies that there exists N 1 ą 0 such that, for all “ 1 M N ą 0,
N
ÿ
}T pxq ´ T pxan q} ď |αi ´ αi,an |}T pei q} ă
i“1
for all n ą N 1
That is, for every bounded sequence txn u8 8
n“1 , tT pxn qun“1 has a convergent
subsequence. Therefore, T is compact.
2. Consider a bounded sequence txn u8 n“1 . It is trivial for dimpRpT qq “ 0

that tT pxn qu8 n“1 has a convergent subsequence tT pxan qu8n“1 . Let E be a
basis of V. For 0 ă dimpRpT qq “ N ă 8, T pxn q can be written in terms
of e1 , e2 , . . . , eN P E (but R N pT q) in the following equation
N
ÿ N
ÿ
T pxn q “ T p αi,n ei q “ T pαi,n ei q
i“1 i“1
By linearity, T pαi,n ei q “ αi,n T pei q. It follows that

N
ÿ
T pxn q “ αi,n T pei q
i“1
By triangular inequality,
N
ÿ N
ÿ N
ÿ
}T pxn q} “ } αi,n T pei q} ď |αi,n |}T pei q} ď |αi,n |}T }}ei }
i“1 i“1 i“1
Since
N
ÿ
|αi,n |}ei } “ }αi,n ei } ď } αi,n ei } ď }xn }
i“1
8
and txn u8
n“1 is bounded, it implies that tαi,n un“1 is bounded. By Bolzano-
Weierstrass theorem, tαi,n un“1 has a convergent subsequence tαi,an u8
8
n“1 .
Consequently, for all 1 ą 0, there exists N 1 such that
|αi ´ αi,an | ă 1
for all n ą N 1 .
It implies that, for all “ 1 }T }maxt}ei }u ą 0, there exists N 1 such that
N
ÿ N
ÿ N
ÿ
}T pxq´T pxan q} “ } αi T pei q´ αi,an T pei q} ď |αi ´αi,an |}T }}ei } ă
i“1 i“1 i“1
for all n ą N 1 .
That is, tT xn u8 n“1 has a convergent subsequence. That is, T is compact.
5.8 Contraction Mapping

Definition 94 (Contraction Mapping). An operator T from a Banach space
(complete normed vector space) V into a Banach space V is said to be a con-
traction mapping iff there exists a constant 0 ď c ă 1 for which
}T pxq ´ T pyq}V ď c}x ´ y}V
for all x, y P V
5.8. CONTRACTION MAPPING 63
Theorem 38. Let T : V Ñ V, defined on a complete normed vector space V,

be a contraction mapping. Therefore, there exists a unique solution for
T pxq “ x
Proof
We define a sequence txn u8
n“1 in V by
xn`1 “ T xn , n ě 0
Consequently,
xn “ T n px0 q
where T n is the nth iteration of T .
Consequently, for n ě m ě 1,
}xn ´ xm }V “ }T n px0 q ´ T m px0 q}V
By using }T pxq ´ T pyq}V ď c}x ´ y}V , we obtain
}T n px0 q ´ T m px0 q}V ď cm }T n´m px0 q ´ x0 }V
That is,
}xn ´ xm }V ď cm }T n´m px0 q ´ x0 }V
By using triangular inequality and }T pxq ´ T pyq}V ď c}x ´ y}V , it follows that
n´m´1
ÿ
}T n´m px0 q ´ x0 }V ď }T i`1 px0 q ´ T i px0 q}V
i“0
n´m´1
ÿ
ď ci }T px0 q ´ x0 }V
i“0
n´m´1
ÿ
ď ci }x1 ´ x0 }V
i“0
Consequently,
}xn ´ xm }V ď cm }T n´m px0 q ´ x0 }V
n´m´1
ÿ
ď cm ci }x1 ´ x0 }V
i“0
8
ÿ
ď cm ci }x1 ´ x0 }V
i“0
(5.12)
By definition 94, 0 ď c ă 1, we obtain
8
ÿ 1
ci “
i“0
1ć
Thus,
cm
}xn ´ xm }V ď }x1 ´ x0 }V
1ć
It follows that txn u8
n“1 is a Cauchy sequence. Since V is a Banach space,
txn u8
n“1 is a convergent sequence. That is, there exists x P V such that
lim xn “ x
nÑ8
Since T is continuous,
T pxq “ T p lim xn q “ lim T pxn q “ lim T n px0 q
nÑ8 nÑ8 nÑ8
5.9 Minimization of Functionals

Theorem 39. Suppose that L : H Ñ H is a linear, self-adjoint and positive
operator on a real Hilbert space H. The function φ that minimizes the functional
F pφq “ xLφ, φy ´ 2xf, φy
is the solution to
Lφ “ f
Proof
Suppose that the functional F is minimized at φ “ φ0
F pφq ´ F pφ0 q “ rxLφ, φy ´ 2xf, φys ´ rxLφ0 , φ0 y ´ 2xf, φ0 ys
Since φ “ φ0 is the solution to Lφ “ f , Lφ0 “ f . Consequently,
F pφq ´ F pφ0 q “ rxLφ, φy ´ 2xLφ0 , φys ´ rxLφ0 , φ0 y ´ 2xLφ0 , φ0 ys
“ xLφ, φy ´ 2xLφ0 , φy ` xLφ0 , φ0 y
By using the fact that L is a linear and self-adjoint operator on a real Hilbert
space,
xLφ, φ0 y “ xLφ0 , φy
and, consequently,
F pφq ´ F pφ0 q “ xLφ, φy ´ xLφ, φ0 y ´ xLφ0 , φy ` xLφ0 , φ0 y
“ xLφ, φ ´ φ0 y ´ xLφ0 , φ ´ φ0 y
“ xLpφ ´ φ0 q, φ ´ φ0 y
ě0 (5.13)
Conversely, by assuming that φ “ φ0 ` tν, we obtain
F pφq ´ F pφ0 q “ F pφ0 ` tνq ´ F pφ0 q
“ pxLpφ0 ` tνq, φ0 ` tνy ´ 2xf, φ0 ` tνyq ´ pxLφ0 , φ0 y ´ 2xf, φ0 yq
(5.14)
5.10. STURM - LIOUVILLE OPERATOR 65
By using the fact that L is a linear and self-adjoint operator on a real Hilbert
space,
` ˘
F pφq ´ F pφ0 q “ xLφ0 , φ0 y ` 2txLφ0 , νy ` t2 xLν, νy ´ 2xf, φ0 ` tνy ´ pxLφ0 , φ0 y ´ 2xf, φ0 yq
“ 2txLφ0 , νy ´ 2txf, νy ` t2 xLν, νy
“ 2txLφ0 ´ f, νy ` t2 xLν, νy
Consequently,
F pφq ´ F pφ0 q
“ 2xLφ0 ´ f, νy ` txLν, νy
t
As t Ñ 0, Gateaux differential dF pφ0 , νq can be expressed as follows
F pφq ´ F pφ0 q
dF pφ0 , νq “ lim
tÑ0 t
“ 2xLφ0 ´ f, νy
for all ν P H
Since F has a local minimum at φ “ φ0 , F pφ0 , νq “ 0, for all ν P H. That
is, Lφ0 ´ f “ 0. This proves that φ0 is a solution to Lφ “ f .
Theorem 40. Suppose that L is a linear, self-adjoint and positive operator on
a complex Hilbert space H . The function φ that minimizes the functional
1 1 1
F pφq “ xLφ, φy ´ xφ, f y ´ xf, φy
2 2 2
is the solution to
Lφ “ f
5.10 Sturm - Liouville Operator

Definition 95 (Sturm -Liouville Operator).
„ 
1 d d
L“´ ppxq ` qpxq
wpxq dx dx
Example 22. Consider the following differential equation of second order
d2 upxq dupxq
a0 pxq ` a1 pxq ` a2 pxqupxq ´ λupxq “ f pxq, a ă x ă b
dx2 dx
Show that the operator
d2 d
L “ a0 pxq 2
` a1 pxq ` a2 pxq (5.15)
dx dx
can be of the form „ 
1 d d
L“´ ppxq ` qpxq
wpxq dx dx
Solution
Consider
„ 
1 d d
L“´ ppxq ` qpxq (5.16)
wpxq dx dx
By rearranging the above equation, we get
ppxq d2 p1 pxq d
L“´ ´ ` qpxq
wpxq dx2 wpxq dx
Comparing equation 5.15 with equation 5.16, we obtain
ppxq
a0 pxq “ ´ (5.17)
wpxq
p1 pxq
a1 pxq “ ´ (5.18)
wpxq
a2 pxq “ qpxq (5.19)
By using equations 5.17 and 5.18, we get
a1 pxq p1 pxq
“
a0 pxq ppxq
Consequently,
ż t“x żx
dppxq a1 pxq dpptq a1 ptq
“ and “ dt
ppxq a0 pxq t“0 pptq 0 a0 ptq
That is, ˆż x ˙
a1 ptq
ppxq “ pp0qexp dt
0 a0 ptq
By choosing pp0q “ 1, we get
ˆż x ˙
a1 px1 q 1
ppxq “ exp dx
0 a0 px1 q
In addition, by using equation 5.18, we get
p1 pxq
wpxq “ ´
a1 pxq
Finally, by using equation 5.19, we get
qpxq “ a2
5.11. HILBERT-SCHMIDT OPERATOR 67
d2 d
In conclusions, L “ a0 pxq 2
` a1 pxq ` a2 pxq can be written in the form of
dx dx „ 
1 d d
Sturm - Liouville operator L “ ´ ppxq ` qpxq by choosing
wpxq dx dx
ˆż x ˙
a1 px1 q 1
ppxq “ exp dx
0 a0 px1 q
p1 pxq
wpxq “ ´
a1 pxq
qpxq “ a2
Theorem 41 (Lagrange’s Identity). Let L is a Sturm-Liouville operator defined

by
„ 
1 d d
L“´ ppxq ` qpxq
wpxq dx dx
and u, v P DpLq. It follows that

„ ˆ ˙
1 d dv du
uLv ´ vLu “ p u ´v (5.20)
wpxq dx dx dx
Proof By definition,
„  „ 
1 d du 1 d dv
Lu “ ´ ppxq ` qpxqu and Lv “ ´ ppxq ` qpxqv
wpxq dx dx wpxq dx dx
Consequently,
ˆ „  ˙ ˆ „  ˙
1 d dv 1 d du
uLv ´ vLu “ u ´ ppxq ` qpxqv ´ v ´ ppxq ` qpxqu
ˆ „ ˙ ˆ „ ˙
1 d dv 1 d du
“u ´ ppxq ´v ´ ppxq
„ ˆ „ ˙ ˆ „ ˙
1 d du d dv
“ v ppxq ú ppxq
wpxq dx dx dx dx
„ ˆ ˙
1 d dv du
“ p u ´v
wpxq dx dx dx
5.11 Hilbert-Schmidt Operator

Definition 96 (Hilbert-Schmidt Kernels and Operator). Let D Ă Rn , n P I` .
A function k : D ˆ D Ñ R satisfying
ż ż
kps, tq dsdt ă 8
D D
is called a Hilbert-Schmidt kernel7 .

In addition, the integral operator K on L2 pDq defined by
ż
rKuspxq “ kpx, yqupyqdy, u P L2 pDq
D
is called a Hilbert-Schmidt operator.

Lemma 8. A Hilbert-Schmidt operator K is a bounded linear operator.
Proof Suppose that K on L2 pDq defined by
ż
rKuspxq “ kpx, yqupyqdy, u P L2 pDq
D
where k is a Hilbert-Schmidt kernel.

It suffices to prove that a Hilbert-Schmidt operator K on L2 pDq is linear and
bounded.
1. K is linear.
For u1 , u2 P L2 pDq,
ż ż
rKpu1 ` u2 qspxq “ kpx, yqpu1 ` u2 qdy
żD żD ż ż
“ kpx, yqu1 dy ` kpx, yqu2 dy
D D D D
“ Ku1 pxq ` Ku2 pxq
2. K is a bounded operator.
For any u P L2 pDq,

ż
}Ku}2LDˆD “ |rKuspxq|2 dx
D
ż ż
“ | kpx, yqupyqdy|2 dx
D D
By using Cauchy-Schwarz Inequality,

ż ż ż
| kpx, yqupyqdy|2 ď |kpk, yq|2 dy |upyq|2 dy,
D D D
Consequently,
ż ż ż
}Ku}2LDˆD ď p |kpx, yq|2 dyqp |upyq|2 dyqdx ă 8
D D D
That is,
}Ku}2L2 pDq ď }k}2L2 pDˆDq }u}L2 pDq ă 8
It follows that K is a bounded operator.
7 It should be noted that k P L2 pD ˆ Dq
Chapter 6
Eigenvalues, Eigenvectors,
Eigenfunctions
6.1 Eigenvalues and Eigenvectors

Definition 97 (Eigenvalues and Eigenvectors). Let A be an n ˆ n matrix.
Suppose that an n ˆ 1 matrix x and a number λ satisfy the following equation.
Ax “ λx, x ‰ 0
The vector x is said to be an eigenvector associated with an eigenvalue λ of the

matrix A.
6.1.1 Properties related to Eigenvalues and Eigenvectors

Theorem 42. Let A be a square matrix. If λ is an eigenvalue of A associated
with an eigenvector v of A, λ ´ µ is an eigenvalue of A ´ µI associated with an
eigenvector v.
Proof Since λ is an eigenvalue of A associated with an eigenvector v,
Av “ λv, v ‰ 0.
Consequently,
Av ´ µv “ λv ´ µv, v ‰ 0
or
pA ´ µIqv “ pλ ´ µqv, v ‰ 0
Therefore, λ ´ µ is an eigenvalue of A ´ µI associated with an eigenvector v.
Theorem 43. Let A and n be a square matrix and a positive integer, respec-
tively. λ is an eigenvalue of A associated with an eigenvector v of A. Therefore,
λn is an eigenvalue associated with an eigenvector v of An .
69
70 CHAPTER 6. EIGENVALUES, EIGENVECTORS, EIGENFUNCTIONS
Proof Since λ is an eigenvalue of A associated with an eigenvector v of A,
Av “ λv, v ‰ 0.
Consequently,
A2 v “ Apλqv
“ λAv
“ λpλvq
“ λ2 v
By iteration, we obtain
An v “ λn v
Theorem 44. Let A be a square matrix. If λ is an eigenvalue of A associated

with an eigenvector v, ppλq is an eigenvalues associated with an eigenvector v
of ppAq.
Proof
N
ÿ
Suppose that ppAq “ an An .
n“0
Since
An v “ λ n v
In particular,
A0 v “ Iv “ λ0 v,
N
ÿ N
ÿ
an An v “ an λn v
n“0 n“0
That is,
ppAqv “ ppλqv
Theorem 45. Let A be a nˆn matrix. If λ1 and λ2 are distinct eigenvalues of A

associated with eigenvectors v1 and v2 , respectively, v1 and v2 are independent.
Proof
To determine if v1 and v2 are independent, consider
c1 v1 ` c2 v2 “ 0 (6.1)
Since
Av1 “ λ1 v1 and Av2 “ λ2 v2 ,
it follows that
Apc1 v1 ` c2 v2 q “ C1 Av1 ` c2 Av2 “ 0
and
c1 λ1 v1 ` c2 λ2 v2 “ 0
6.1. EIGENVALUES AND EIGENVECTORS 71
By using equation 6.1, we obtain

c1 λ1 v1 ` c2 λ1 v2 “ 0 (6.2)
and
c1 λ2 v1 ` c2 λ2 v2 “ 0 (6.3)
By using equations 6.1 and 6.2, we obtain
c2 pλ2 ´ λ1 qv2 “ 0
Similarly, by using equations 6.1 and 6.3, we obtain
c1 pλ1 ´ λ2 qv1 “ 0
Since λ1 and λ2 are distinct and eigenvectors v1 and v2 are not zero vectors, it
follows that c1 “ c2 “ 0
6.1.2 Singular Value Decomposition

Theorem 46 (Singular Value Decomposition for Real-Valued Matrices). Let A
be an m ˆ n real-valued matrix. Therefore, the matrix A can be expressed in the
following equation.
A “ U ΣV T
where
U “ ru1 u2 . . . um s and V “ rv1 v2 . . . vn s are and orthogonal m ˆ m matrix and
an orthogonal n ˆ n matrix, respectively.
Σ “ diagpσ1 , σ2 , . . . , σminpm,nq q is a diagonal m ˆ n matrix such that σ1 ě σ2 ě
. . . ě σminpm,nq ě 0.
Theorem 47 (Singular Value Decomposition for Complex-Valued Matrices).
Let A be an m ˆ n complex-valued matrix. Therefore, the matrix A can be
expressed in the following equation.
A “ U ΣV H
where
U and V are and unitary mˆm matrix and an unitary nˆn matrix, respectively.
Σ “ diagpσ1 , σ2 , . . . , σminpm,nq q is a diagonal m ˆ n matrix such that σ1 ě σ2 ě
. . . ě σminpm,nq ě 0.
Example 23. By singular value decomposition discussed earlier, show that a
m ˆ n matrix A can be expressed as
minpm,nq
ÿ
A“ σi ui viT if A is a real-valued matrix.
i“1
Also, a m ˆ n matrix A can be expressed as

minpm,nq
ÿ
A“ σi ui viH if A is a complex-valued matrix.
i“1
Proof By singular value decomposition for real-valued matrices,
A “ U ΣV T
where“ ‰ “ ‰
U “ u1 u2 . . . un and V “ v1 v2 . . . vn are orthogonal matrices.
Σ “ diagpσ1 , σ2 , . . . , σminpm,nq q where σ1 ě σ2 ě . . . ě σminpm,nq ě 0 is a
diagonal matrix.
That is,
“ ‰ “ ‰T
A “ u1 u2 ... um diagpσ1 , σ2 , . . . , σminpm,nq q v1 v2 ... vn
1. m ď n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
m
ÿ
“ σi ui viT
i“1
2. m “ n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
m
ÿ
“ σi ui viT
i“1
3. m ą n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
n
ÿ
“ σi ui viT
i“1
6.2. EIGENVALUES AND EIGENFUNCTIONS 73
That is,
minpm,nq
ÿ
A“ σi ui viT
i“1
Example 24. Let A be a matrix with m rows and n columns. Prove that
g
fminpm,nq
f ÿ
}A}F “ e σi2
i“1
where σi is the ith singular value.
Solution
1. A is a real-valued matrix
minpm,nq
ÿ
A“ σi ui viT
i“1
By using the fact that, for i, j “ 1, 2, . . . , minpm, nq,

#
T T 1, i “ j
xui vi , uj vj y “ δij “
0, i ‰ j
and by Pethagorean law,

g
fminpm,nq
f ÿ
}A}F “ e σi2
i“1
2. A is a complex-valued matrix
6.2 Eigenvalues and Eigenfunctions

Definition 98 (Eigenvalues and Eigenfunctions). Let L : D Ñ D be a linear
operator. Also, a function φ P D is corresponding to a complex number λ P Z
such that
Lφ “ λφ ‰ 0
where D
v is called an eigenfunction corresponding to an eigenvalue λ
Theorem 48. Let φ1 and φ2 be eigenfunctions associated with a linear operator

L and corresponding to distinct eigenvalues λ1 and λ2 , respectively. φ1 and φ2
are independent.
Proof To determine whether v1 and v2 are independent, consider equation

6.4 in the following
α1 φ1 ` α2 φ2 “ 0 (6.4)
Since
Lφ1 “ λ1 φ1 and Lφ2 “ λ2 φ2
This follows that, by linearity,
Lpα1 φ1 ` α2 φ2 q “ α1 Lφ1 ` α2 Lφ2 “ L0 “ 0

Lφi “ λi φi , i “ 1, 2
we obtain
α1 λ1 φ1 ` α2 λ2 φ2 “ 0
By multiplying equation 6.4 with λ1 and λ2 , we obtain
α1 λ1 φ1 ` α2 λ1 φ2 “ 0 (6.5)
and
α1 λ2 φ1 ` α2 λ2 φ2 “ 0, (6.6)
respectively. By using equations 6.4 and 6.6, we obtain
α1 pλ1 ´ λ2 qφ1 “ 0
and, by using equations 6.4 and 6.5, we obtain
α2 pλ2 ´ λ1 qφ2 “ 0
Consequently, since λ1 ‰ λ2 and φ1 , φ2 ‰ 0, then
α1 “ α2 “ 0
That is, φ1 and φ2 are independent.

Theorem 48 can be extended to theorem 49 in the following
Theorem 49. Let φ1 , φ2 , . . . φn be eigenfunctions associated with a linear op-

erator L and corresponding to distinct eigenvalues λ1 , λ2 , . . . , λn , respectively.
φ1 , φ2 , . . . , φn are independent.
Theorem 50. Let φ is an eigenfunction corresponding to an eigenvalue λ as-

sociated with a linear self-adjoint operator L. Therefore, λ is real.
6.2. EIGENVALUES AND EIGENFUNCTIONS 75
Proof Since φ is an eigenfunction corresponding to an eigenvalue λ associated

with a linear self-adjoint operator L, then
Lφ “ λφ
and
xLφ, φy “ xφ, Lφy
Consequently,
xLφ, φy “ xλφ, φy and “ xφ, Lφy “ xφ, λφy
That is,
xλφ, φy “ xφ, λφy
By Definition 56,
xλφ, φy “ λxφ, φy and xφ, λφy “ λxφ, φy
Consequently,
λxφ, φy “ λxφ, φy
Since φ ‰ 0, then xφ, φy ‰ 0. we obtain λ “ λ. That is, λ is real.
Theorem 51. Let φ1 , φ2 , . . . , φn be eigenfunctions corresponding to distinct
eigenvalues λ1 , λ2 , . . . , λn associated with a linear self-adjoint operator L on a
Hilbert space H. Therefore, φ1 , φ2 , . . . , φn are orthogonal.
Proof
First of all, it should be noted by that, by theorem 50, an eigenvalue λ
corresponding to an eigenfunction φ associated with a linear self-adjoint operator
L is real.
Next, let λi , i “ 1, 2, . . . , n are distinct eigenvalues corresponding to eigen-
functions φi , i “ 1, 2, . . . , n, respectively, associated with a linear self-adjoint
operator L. This follows that, for i ‰ j,
xLφi , φj y “ λi xφi , φj y
and, by using the fact that the adjoint operator L‹ of the linear self-adjoint
operator L is L.
xLφi , φj y “ xφi , L‹ φj y “ xφi , Lφj y “ xφi , λj φj y “ λj xφi , φj y “ λj xφi , φj y
That is,
λi xφi , φj y “ λj xφi , φj y
Consequently,
pλi ´ λj qxφi , φj y “ 0
By using the fact that λi and λj are distinct, we obtain
xφi , φj y “ 0
That is, φ1 , φ2 , . . . , φn are orthogonal.
Lemma 9. Let A P Rnˆn . For all ą 0, there exists an induced norm }.}1
such that
}A} ď ρpAq `
Proof
See the proof in [72]
1 }.} depends on
Chapter 7
Examples of Applications
7.1 Finite Element Method

First of all, consider the following ordinary differential equation
B2 u
“ f, x P p0, 1q
Bx2
with boundary condition up0q “ up1q “ 0 We can construct weak formulation
in the following equation.
ż1 ż1
B2 u
v dx “ f v dx
0 Bx2 0
Consequently,
ˇ1 ż 1 ż1
ˇ
u1 v ˇˇ ´ u1 v 1 dx “ f v dx
0 0 0
Divide the interval r0, 1s into n subintervals; each of which is rxi´1 , xi s for
B2 u
i “ 1, 2, . . . , n. We now want to approximate the solution to “ f, x P
Bx2
p0, 1q with boundary condition up0q “ up1q “ 0 by a function uh P Vh “
spantφ1 , φ2 , φ3 , . . . , φn u
where $
x ´ xi
’
’ , x P rxi´1 , xi s
& h
’
φi pxq “ x i`1 ´ x
, x P rxi , xi`1 s
’
’
’ h
%0, otherwise
Suppose that
n
ÿ
uh “ αi φi pxq.
i“1
77
78 CHAPTER 7. EXAMPLES OF APPLICATIONS
It is important to note that the approximate solution satisfies the boundary

condition up0q “ up1q “ 0. That is,
ż1 ż1
´ u1 vh1 dx “ f vh dx, v P Vh
0 0
n
ż1 ÿ ż1
´ p αi φi pxqq1 φ1j dx “ f φj dx for j “ 1, 2, 3, . . . , n
0 i“1 0
By rearranging the above equation, we can rewrite the above equation in matrix
form as follows:
» ż1 ż1 fi »ż 1 fi
1 1 1 1
— ´ φ φ
1 1 dx . . . ´ φ φ
1 n dx »
ffi α1
fi
— f φ 1 dx ffi
— 0 0 ffi — 0 ffi
.. .. .
.. . .
ffi – .. fl “ — ..
— ffi — ffi — ffi
—
— . . ffi —
ffi
ffi
– ż1 ż1 fl αn –ż 1 fl
1 1 1 1
´ φn φ1 dx . . . ´ φn φn dx f φn dx
0 0 0
7.2 Method of Moments

Consider
Lu “ f
N
ÿ
Suppose that the solution u can be approximated by uh “ αn φn and fh “
n“1
N
ÿ
xf, wm ywm , respectively, where tφ1 , φ2 , φ3 , . . . , φN u and tw1 , w2 , w3 , . . . , wN u
m“1
are an orthonormal bases. Consequently,
xLuh , wm y “ xf, wm y for m “ 1, 2, 3, . . . , N

N
ÿ
By replacing uh by αn φn , we obtain
n“1
N
ÿ
αn xφn , wm y “ xf, wm y for m “ 1, 2, 3, . . . , N
n“1
or, in matrix form

» fi » fi » fi
xφ1 , w1 y ... xφN , w1 y α1 xf, w1 y
— .. .. .. ffi — .. ffi — .. ffi
– . . . fl – . fl “ – . fl
xφ1 , wN y . . . xφN , wN y αN xf, wN y
Appendix A
Matrices
Definition 99 (Matrices). A matrix is a rectangular array of numbers. A

matrix of m rows and n columns is said to have dimension of m ˆ n.
Definition 100 (Real-Valued and Complex-Valued Matrices). A real-valued
matrix is a rectangular array of real numbers whereas a complex-valued matrix
is a rectangular array of complex numbers
Definition 101 (Square Matrices). A square matrix is a matrix of the same
rows and columns.
Definition 102 (Diagonal Matrices). Let D “ rdij smˆn be an m ˆ n matrix.
The matrix D is said to be a diagonal matrix1 iff dij “ 0, i ‰ j. In addition,
the matrix D is denoted by “ diagpd1 , d2 , . . . , dminpm,nq q
Definition 103 (Symmetric Matrices). Let A “ raij snˆn be a square matrix.
The matrix A is said to be a symmetric matrix iff aij “ aji for all i “ 1, 2, . . . , n
and j “ 1, 2, . . . , n
Definition 104 (Transposition and Hermittian of matrices). Let A “ raij smˆn
be an m ˆ n matrix. AT “ rbij snˆm is said to be the transposition of A iff
bij “ aji . In addition, AT “ rbij snˆm is said to be the hermittian of A iff
bij “ aji .
Definition 105 (Orthogonal Matrices). Let Q be a real-valued square matrix.
Q is said to be an orthogonal matrix iff QQT “ QT Q “ I.
Definition 106 (Unitary Matrices). Let Q be a complex-valued square matrix.
Q is said to be an orthogonal matrix iff QQH “ QH Q “ I.
Definition 107 (Positive Definiteness and Semi-Positive Definiteness). Let A
be a n ˆ n symmetric matrix. If the matrix A holds the following property
v T Av ě 0 for all v ‰ 0,
1 It should be noted that a diagonal matrix could not be a square matrix.
79
80 APPENDIX A. MATRICES
then A is called a semi-positive-definite matrix. In addition, If the matrix A

holds the following property
v T Av ą 0 for all v ‰ 0,
then A is called a positive-definite matrix.
Definition 108 (Negative Definiteness and Semi-Negative Definiteness). Let

A be a n ˆ n symmetric matrix. If the matrix A holds the following property
v T Av ď 0 for all v ‰ 0,
then A is called a semi-negative-definite matrix. In addition, If the matrix A

holds the following property
v T Av ď 0 for all v ‰ 0,
then A is called a negative-definite matrix

Appendix B
Integrals
B.1 Riemann Integral

Consider a function f defined over ra, bs. Suppose that we want to find the Rie-
żb
mann integral f ptq dt. First of all, divide the interval ra, bs into subintervals
a
rti´1 , ti s for i “ 1, 2, 3, . . . , n and a “ t0 ă t1 ă t2 ă . . . ă tn “ b. In addition,
t˚i is an evaluation point in rti´1 , ti s.
n
ÿ
Sn pf q “ f pt˚i qpti ´ ti´1 q
i“1
B.2 Riemann-Stieltjes Integral

Suppose that g is a monotonically increasing function. Consider a function f
defined over ra, bs. Suppose that we want to find the Riemann-Stieltjes integral
żb
f ptq dgptq. Similar to the case of Riemann Integral, first of all, divide the
a
interval ra, bs into subintervals rti´1 , ti s for i “ 1, 2, 3, . . . , n and a “ t0 ă t1 ă
t2 ă . . . ă tn “ b. In addition, t˚i is an evaluation point in rti´1 , ti s.
n
ÿ
Sn pf q “ f pt˚i qpgpti q ´ gpti´1 qq
i“1
B.3 Lebesgue Integral

We can define the so-called Lebesque Integral as described below.
1. Let g be an indicator function of A defined as
#
1, x P A
gpxq “
0, x R A.
81
82 APPENDIX B. INTEGRALS
The Lebesgue integral of g is defined as

ż
g dµ0 “ µ0 pAq
R
2. Let g be a simple function of the form

N
ÿ
g“ an gn
n“1
#
1, x P An
where gn pxq “ χAn pxq “ provided that A1 , A2 , A3 , . . . are
0, x R An
disjoint sets.
The Lebesgue integral of g is defined as

ż N
ż ÿ
g dµ0 “ an gn dµ0
R R n“1
N
ÿ ż
“ an gn dµ0
n“1 R
N
ÿ ż
“ an χAn dµ0
n“1 R
N
ÿ
“ an µ0 pAn q
n“1
3. Let f be a nonnegative function defined on R. The Lebesgue integral is

defined as
ż "ż *
f dµ0 “ sup g dµ0 , g is a simple function such that gpxq ď f pxq
R R
4. Let f be a function defined on R. Therefore f can be decomposed into

positive part f ` and negative part f ´ shown as follows:
f “ f ` ´ f ´.
The Lebesgue integral of f is defined as

ż ż ż
f dµ0 “ f ` dµ0 ´ f ´ dµ0
R R R
In particular, dµ0 “ dx,

ż ż ż
f dx “ f ` dx ´ f ´ dx
R R R
B.3. LEBESGUE INTEGRAL 83
5. The Lebesgue integral of a function f defined on R over A is defined as

ż ż
f dµ0 “ χA f dµ0
A A
#
1, x P A
where χA pxq “
0, x R A
In particular, dµ0 “ dx, we obtain
ż ż
f dx “ χA f dx
A A
#
1, xPA
where χA pxq “
0, xRA
Definition 109 (Locally Integrable Functions). Let f be a function defined on

żb
R. f is said to be locally integrable iff f exists for all ´8 ă a ă b ă 8
a
Definition 110 (Convergence in Norm). A sequence tfn u converges to f in

norm1 iff }f ´ fn } Ñ 0.
Lemma 10. Let tfn u8 n“1 be a sequence of integrable functions converges to an
integrable function f in norm. If tgn u8 8
n“1 is a subsequence of tfn un“1 , then
8
tgn un“1 converges to f in norm.
Lemma 11. Let tgn u8 n“1 be a subsequence of a sequence of integrable functions
tfn u8
n“1 . If tf u 8
n n“1 converges to an integrable function f in norm, then tgn u8
n“1
also converges to f in norm.
Before we can define equality almost everywhere, we should define null func-
tions and null sets in the following
Definition 111 (Null
ż Functions). f is said to be a żnull function iff a function
f is integrable and |f | “ 0 It should be noted that |f | is an alternative form
ż
of |f | dx.
R
Definition 112 (Null Set). Let Ω Ď R. The set Ω is called a null set iff its
characteristic function is a null function.
Definition 113 (Equality Almost Everywhere). Let functions f and g be de-
fined on R. If tx P R | f pxq ‰ gpxqu is a null set, f and g are said to be equal
almost everywhere (f “ g a.e.).
As a result, we can show that
ż
1 The norm }.} is defined in terms of Lebesgue integral by }f } “ |f | dx
R
84 APPENDIX B. INTEGRALS
ż
Theorem 52. |f ´ g| “ 0 iff f “ g a.e.
Definition 114 (Convergence Almost Everywhere). A sequence tf1 , f2 , f3 , . . .u

where f1 , f2 , f3 , . . . are defined on R is said to converge to a function f (denoted
by fn Ñ f a.e.) iff fn pxq Ñ f for every x except a null set.
2
Theorem 53 (Monotone Convergence Theorem). Let tfn u8 n“1ż be a monotone

sequence of integrable functions and there exists M such that fn ă M for all
n P N. Therefore, there existsżan integrable function f such that fn Ñ f i.n.

and fn Ñ f a.e.. In addition, f ď M .
Lemma 12 (Fatou’s Lemma). Let tfn u8 n“1 be a sequence

ż of non-negative in-
tegrable functions such that there exists M such that fn ă M for all n “
ż
1, 2, 3, . . .. If fn Ñ f a.e., then f is integrable and f ă M .
Theorem 54 (Dominated Convergence Theorem). If a sequence of integrable

functions tfn u8
n“1 converges almost everywhere (a.e.) to a function f and there
exists an integrable function g P L1 pRq such that |fn | ă g for all n P N, then f
is integrable and fn Ñ f in norm (i.n.)
Theorem 55. A sequence of integrable functions tfn u8

n“1 converges to an in-
tegrable function f in norm. Therefore, tfn u8
n“1 converges to an integrable
function g in norm iff f “ g a.e.
Definition 115" (Square
ż Integrable
* Functions). Let f be a square integrable
2
function iff f P f | |f | ă 8
R
Definition 116 (Locally integrable Complex-Valued Functions). A complex-

valued function f is a locally integrable complex-valued function iff its real part
Retf u and imaginary part Imtf u are both locally integrable.
2 non-increasing or non-decreasing
Appendix C
Holder, Cauchy-Schwarz
and Minkowski Inequalities
C.1 Holder Inequality

1 1
For ` “ 1 and p ą 1, q ą 1,
p q
˜ ¸ p1 ˜ ¸ q1
8
ÿ 8
ÿ 8
ÿ
|xi yi | ď |xi |p |yi |q
i“1 i“1 i“1
C.2 Cauchy-Schwarz Inequality

˜ ¸ 21 ˜ ¸ 21
8
ÿ 8
ÿ 8
ÿ
|xi yi | ď |xi |2 |yi |2 (C.1)
i“1 i“1 i“1
C.3 Minkowski Inequality

For x “ pxi q P lp and y “ pyi q P lp and p ě 1,
˜ ¸ p1 ˜ ¸ p1 ˜ ¸ p1
8
ÿ 8
ÿ 8
ÿ
p p p
|xi ` yi | ď |xi | ` |yi |
i“1 i“1 i“1
85
86APPENDIX C. HOLDER, CAUCHY-SCHWARZ AND MINKOWSKI INEQUALITIES
Appendix D
Complex Analysis
Definition 117 (Derivatives).
f pz ` ∆zq ´ f pzq
f 1 pzq “ lim
∆zÑ0 ∆z
Definition 118 (Derivatives of Order n).
f n´1 pz ` ∆zq ´ f n´1 pzq

f n pzq “ lim
∆zÑ0 ∆z
Definition 119 (Differentiable Functions). A function f of the complex variable
z is differentiable at z0 iff f 1 pz0 q is well-defined.
Definition 120 (Analytic Functions). A function f of the complex variable z

is analytic at z0 iff there exists ą 0 such that f is differentiable at z for all
z P tz | |z ´ z0 | ă u.
Definition 121 (Integration).

ż zt
f pz 1 q dz 1 “ lim f pzi‹ q∆zi (D.1)
z0 N Ñ8
Definition 122 (Antiderivatives). The antiderivative F of a complex-valued

function f is a function which satisfies
dF
“ f pzq
dz
for an open set D
Definition 123 (Contours). A contour1 is an arc consisting of a finite number

of smooth arcs connected end to end.
1 piecewise smooth arc
87
88 APPENDIX D. COMPLEX ANALYSIS
Definition 124 (Closed Contours). A closed contour C is a contour iff the

start point and the endpoint are the same.
Definition 125 (Simple Contours). A simple contour is a contour which does

not intersect itself.
Definition 126 (Contour Integral). Suppose that
z “ zptq, a ď t ď b
represent a contour C, i.e., C “ tz | z “ zptq, a ď t ď bu. Let f be a piecewise

continuous on ra,ż bs, i.e., F ptq “ f pzptqq is piecewise continuous on ra, bs. The
contour integral f pzq dz is defined as:
C
ż żb
f pzq dz “ f pzptqqz 1 ptq dt
C a
Theorem 56 (Cauchy-Goursat Theorem). Let a function f be analytic in and

on a simple closed contour C. Therefore,
¿
f pzq “ 0
C
Theorem 57 (Cauchy’s Integral Formula). If a function f is analytic in and

on a simple closed contour in the counterclockwise direction, then
¿ #
f pzq 2πıf pz0 q, z0 is inside C
dz “
z ´ z0 0, z0 is outside C
C
Consider Cauchy’s integral formula

¿ #
f pzq 2πıf pz0 q, z0 is inside C
dz “
z ´ z0 0, z0 is outside C
C
Suppose that z0 is inside C. Then, replace z and z0 with s and z, respectively,

we obtain ¿
1 f psq
f pzq “ ds
2πı s ´ z
C
By differentiating f pzq with respect to z, we obtain

¿
1 B f psq
f pzq “ ds (D.2)
2πı Bz s´z
C
89
and then
¿
n 1 B f psq
f pzq “ ds
2πı Bz s ´ z
C
¿
n! f psq
“ ds
2πı ps ´ zqn`1
C
That is,
f n pzq
¿
1 f psq
“ ds
n! 2πı ps ´ zqn`1
C
Theorem 58 (McClaurin’s Series). Let f be analytic in and on a simple closed

contour C “ t z| |z| “ R, R ą 0u in the counterclockwise direction. Therefore,
8
ÿ
f pzq “ an z n
n“0
Theorem 59 (Taylor’s Series). Let f be analytic in and on a simple closed

contour C “ t z| |z ´ z0 | “ R, R ą 0u in the counterclockwise direction for a
fixed z0 . Therefore,
8
ÿ
f pzq “ an pz ´ z0 qn
n“0
Theorem 60 (Laurent’s Series). Let f be analytic in the region tz | R1 ă

|z ´ z0 | ă R2 u for a fixed z0 . Therefore,
8
ÿ
f pzq “ cn pz ´ z0 qn
n“´8
Lemma 13. Let f pzq “ pz ´ z0 qn , n P I and on C “ tz | |z ´ z0 | “ R, R ą 0u

which is in the counterclockwise direction. Therefore,
¿ #
2πı, n “ ´1
f pzq dz “
0, otherwise
C
8
ÿ
Lemma 14. Let f pzq “ cn pz ´ z0 qn and on C “ tz | |z ´ z0 | “ R, R ą 0u
n“´8
which is in the counterclockwise direction. Therefore,
¿
f pzq dz “ 2πıRes f pzq
z“z0
C
90 APPENDIX D. COMPLEX ANALYSIS
Lemma 15 (Contour Deformation). Let C1 and C2 be simple closed contours

in the counterclockwise direction where C2 is interior to C1 . Suppose that f
is analytic in the closed region consisting of both contours and region between
them. Therefore, ż ż
f pzq dz “ f pzq dz
C1 C2
Theorem 61 (Residue Theorem). Consider a simple closed contour C in the

counterclockwise direction. Let f be analytic in and on C except for a finite
number n of interior points. Therefore,
¿ n
ÿ
f pzq dz “ 2πı Resf pzq
z“zi
i“1
C
Appendix E
Gateaux Differential
Definition 127 (Gateaux Differential). Suppose that x is a fixed vector in

x P B1 . The operator T : B1 Ñ B2 is Gateaux differentiable1 at x iff the exist a
continuous linear operator G such that

T px ` thq ´ T pxq
lim
“ 0 for all h P B1
´ Gphq
tÑ0 t
where B1 and B2 are Banach spaces over a field F and t is in F
Theorem 62 (Uniqueness Theorem). Let T : B1 Ñ B2 where B1 and B2 are
a Banach space and a field, respectively. If the Gateaux differential of T exists,
then it is unique.
Theorem 63 (Mean Value Theorem). Suppose that a functional f has a Gateaux

differential G at every x P B where Gphq “ df px, hq. For any x and h in B,
there exists η P p0, 1q such that
f px ` hq ´ f pxq “ df px ` ηh, hq
1G is called the Gateaux differential of T where Gphq “ dT px, hq
91
92 APPENDIX E. GATEAUX DIFFERENTIAL
Bibliography
[1] F. Hirsch and G. Lacombe (translated by Silvio Levy), Elements of Func-

tional Analysis, translated ed., Springer 1999.
[2] J. K. Hunter, Measure Theory, Lecture notes at department of Mathemat-

ics, University of California at Davis, 2011.
[3] R. Durrett, Probability: Theory and Examples, 4th ed., Cambridge Univer-
sity Press, 2010.
[4] M. Vetterli and J. Kovacevic, Wavelets and Subband Coding, 2th ed., Pren-
tice Hall, 2007.
[5] M. Vetterli, J. Kovacevic and V. K. Goyal, Foundations of Signal Process-

ing, 1st ed., 2014.
[6] J. Kovacevic, V. K. Goyal and M. Vetterli, Fourier and Wavelet Signal

Processing, 1st ed., 2013.
[7] R. L. Allen, Signal Analysis Time, Frequency, Scale and Structure, 1st ed.,
IEEE Press Wiley-Interscience, 2004.
[8] C. Valens, “A Really Friendly Guide to Wavelets,” 1999.
[9] N. Young, An Introduction to Hilbert Space, reprinted ed., Cambridge Uni-

versity Press, 1992.
[10] L. Debnath and P. Mikusinski, Introduction to Hilbert Spaces with Appli-

cations, 2nd ed., Academic Press, 1999.
[11] E. Kreyszig, Introductory Functional Analysis with Applications, 1st ed.,

John Wiley & Sons, 1989.
[12] Hal Daume III, “From Zero to Reproducing Kernel Hilbert Spaces in
Twelve Pages or Less,” 2004.
[13] J. T. Oden and J. N. Reddy, An Introduction to The Mathematical Theory

of Finite Elements, reprinted ed., Dover, 2011.
[14] D. A. Salamon, Measure and Integration,lecture notes at ETH Zurich, 2015.
93
94 BIBLIOGRAPHY
[15] H. Zhang, Y. Xu and J. Zhang,“Reproducing Kernel Banach Spaces for

Machine Learning,” Journal of Machine Learning Research, vol. 10, pp.
2741-2775, 2009.
[16] F. Deutsch, H. Poulisse, and L. Zikatanov, “Best Approximation to an

Element of an Inner Product Space from the Range of a Linear Operator
over a Polyhedron,”
[17] H. Brezis, Functional Analysis, Sobolev Spaces and Partial Differential

Equations, 1st ed., Springer, 2010.
[18] L. Wang, Karhunen-Loeve Expansions and their Applications, PhD. Thesis,

The London School of Economics and Political Science, 2008.
[19] T. Hastie, R, Tibshirani and J. Friedman, The Elements of Statistical

Learning, 2nd ed., Springer, 2009.
[20] J. L. Kirley Jr., “Introduction to Electric Power Systems,” Open Course-

ware at Department of Electical Engineering and Computer Science, Mas-
sachusetts Institute of Technology, 2011.
[21] C. E. Heil,“Real Analysis II,” Lecture notes at School of Mathematics,

Georgia Institute of Technology, 2008.
[22] E. Cohen, R. F. Riesenfeld and G. Elber, Geometric Modeling with Splines:

An Introduction, 1st ed., AK Peters, 2001.
[23] S. M. Rao, D. R. Wilton, “Electromagnetic Scattering by Surfaces of Ar-

bitrary Shape,”IEEE Trans. Antennas Propagat., vol. AP-30, no. 3, pp.
409-418, May 1982.
[24] M. Grigoriu, Stochastic Calculus: Applications in Science and Engineering,

1st ed., Birkhauser, 2002.
[25] K. Trivisa, “A Very Short Summary of the Theory of Distributions”, Lec-

ture Notes at University of Maryland at College Park.
[26] E. Sulli, Lecture Notes on Finite Element Methods for Partial Differential
Equations, Mathematical Instistute, University of Oxford, 2012.
[27] Y. Zhu and A. C. Cangellaris, Multigrid Finite Element Methods for Elec-
tromagnetic Field Modeling, 1st ed., IEEE Press, 2006.
[28] G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns
Hopkins University Press Baltimore and London, 1996.
[29] H. Wu and A. C. Cangellaris,“Model-Order Reduction of Finite-Element

Approximations of Passive Electromagnetic Devices Including Lumped
Electricsl-Circuit Models,”IEEE Transactions on Microwave Theory and
Techniques, vol. 52, no. 9, pp.2305-2312, September 2004.
BIBLIOGRAPHY 95
[30] A. Wibisono, “Functional Analysis Review,” Massachusetts Institute of

Technology, 2010.
[31] P. B. Kronheimer, “Weak Convergence,” Lecture Notes at Harvard Univer-
sity, 2010.
1
[32] J. R. Shewchuk, “An Introduction to the Conjugate Gradient Method,” 1
4
ed., School of Computer Science, Carmegie Mellon University, 1994.
[33] A. Bondeson, T. Rylander, and P. Ingelstrom, Computational Electromag-
netics, 1st ed., Springer, 2005.
[34] D. J. Silvester,“A Finite Element Primer,” 1.3 ed., School of Mathematics,
University of Manchester, 2012.
[35] Y. Chen, Wavelets and Mltirate Filter Banks-Theory, Structure, Design,
and Applications, PhD. Thesis, Massachusetts Institute of Technology,
2004.
[36] M. Vetterli and C. Herley,“Wavelets and Filter Banks: Theory and De-
sign,”IEEE Transactions on Signal Processing, vol. 40, no. 9, pp.2305-
2312, September 1992.
[37] P. P. Vaidyanathan, Multirate Systems and Filter Banks, low price ed.,
Prentice Hall, 1993.
[38] P. P. Vaidyanathan, “Multirate Digital Filters, Filter Banks, Polyphase
Networks, and Applications: A Tutorial,”Proceedings of the IEEE, vol. 78,
no. 1, pp.56-93, January 1990.
[39] D. G. Dudley , Mathematical Foundationa for Electromagnetic Theory, 1st
ed., IEEE Press, 1994.
[40] W. Rudin, Principles of Mathematical Analysis, 3rd ed., McGraw-Hill,
1998.
[41] J. W. Brown and R. V. Churchill, Complex Variables and Applications, 8th
ed., McGraw-Hill, 2009.
[42] J. D. Glover, M. S. Sarma and T. J. Overbye, Power System Analysis and
Design, 4th ed., Thomson, 2008.
[43] U. H. Gerach, LINEAR MATHEMATICS IN INFINITE DIMENSIONS
Signals Boundary Value Problems and Special Functions, 2nd Beta ed.,
Textbook for Classroom at The Ohio State University.
[44] R. X. Gao and R. Yan, Wavelets: Theory and Applications for Manufac-
turing, 1st ed., Springer, 2011.
[45] I. Stakgold and M. Holst, Green’s Functions and Boundary Value problems,
3rd ed., John Wiley & Sons, Inc., Publication, 2011.
96 BIBLIOGRAPHY
[46] H. Kuo, Introduction to Stochastic Integration, 1st ed., Springer, 2006.

[47] O. Bousquet and F. Perez-Cruz, “Kernel Methods and their Applications
to Signal Processing,”
[48] B. Widrow and S. D. Stearns, Adaptive Signal Processing, 1st ed., Prentice
Hall, 1985.
[49] F. Farassat, “Introduction to Generalized Functions With Applications in
Aerodynamics and Aeroacoustics,” NASA Technical Paper 3428, 1996.
[50] B. Osgood, “The Fourier Transform and its Applications,” Lecture Note at
Stanford University.
[51] I. N. Sneddon, The Use of Integral Transforms, 1st ed., McGraw - Hill,
1972.
[52] J. M. Erdman, “Companion to Real Analysis,” Lecture Note at Portland
State University, 2012.
[53] L. Erdos, “Fourier Transform,” 2008.

[54] H. Weizsacker, “Basic Measure Theory,” Revised Translation, Fachbereich
Mathematik Technische Universitat Kaiserslautern, 2008.
[55] B. K. Driver, “Analysis Tools with Applications,” Preprints, Springer, 2003.
[56] J. Maly, “Fourier Transform,”

[57] N. Aronszajn, “Theory of Reproducing Kernels,” Transactions of the Amer-
ican Mathematical Society, vol. 68, no. 3, pp. 337-404, May 1950.
[58] A. Alexanderian,“A Brief Note on the Karhunen-Loeve Expansion,”
[59] D. Sejdinovic and A. Gretton,“What is an RKHS?,” 2012.

[60] N. Bianchi,“Electrical Machine Analysis using Finite Elements,” 1st ed.,
CRC Press, 2005.
[61] proofwiki.org
[62] A. C. R. Belton,“Functional Analysis,” Lecture Note at Lancaster Univer-

sity, 2006.
[63] T. B. Ward,“Functional Analysis lecture notes,” University of East Anglia.
[64] http://www.math.uiuc.edu/„mjunge/metric2.pdf.
[65] A. Alexanderian,“On Compact Operators,”

[66] http://www.uio.no/studier/emner/matnat/math/MAT2400/v11/Metric.pdf
[67] http://www.u.arizona.edu/„mwalker/econ519/Econ519LectureNotes/Sequences.pdf
BIBLIOGRAPHY 97
[68] https://en.wikipedia.org
[69] J. K. Hunter,“Applied Analysis,” Lecture Notes at University of California
at Davis.
[70] math.unice.fr/„frapetti/CorsoF/cours2.pdf
[71] math.unice.fr/„frapetti/CorsoF/cours3.pdf
[72] http://www.math.drexel.edu/„foucart/TeachingFiles/F12/M504Lect6.pdf
[73] people.reed.edu/„jerry/311/lec08.pdf
[74] M. Beck, G. Merchesi, D. Pixton and L. Sabalka,“A First Course in Com-
plex Analysis,” Version 1.52, 2016.
[75] C. Berg, “Complex Analysis,” Department of Mathematical Sciences, Uni-
versitetsparken 5, 2012.
[76] E. M. Stein and R. Sharkarchi, “Complex Analysis,” Princeton Lectures in
Analysis, Princeton University Press, 2007.
[77] C. Bennewitz, “Complex Analysis,” Lecture Notes at Lund University 2006.
[78] G. Cain, “Complex Analysis,” Lecture Notes at Georgia Institute of Tech-
nology, 1999.

Introduction To Functional Analysis PDF

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Introduction To Functional Analysis PDF

Caricato da

Copyright:

Formati disponibili

Introduction to Hilbert Space

Assoc. Prof. Waroth Kuhirun

February 20, 2017

Assoc. Prof. Waroth Kuhirun

3 Normed Spaces and Banach Spaces 15

4 Inner Product Spaces and Hilbert Spaces 25

5 Operators and Functionals 43

6 Eigenvalues, Eigenvectors, Eigenfunctions 69

C Holder, Cauchy-Schwarz and Minkowski Inequalities 85

1.1 Supremum, Maximum, Infemum and Mini-

2. For β P R, if β ă α, β is not an upper bound of S.

Therefore, α is called the supremum (least upper bound) of S and we write

Definition 2 (Maximum). Let a nonempty set S Ă R and S. If α P S and

Definition 3 (Infemum). Let S Ă R and S be bounded below. Suppose that the

2. For β P R, if β ą α, β is not a lower bound of S.

Therefore, α is called the infemum (greatest lower bound) of S and we write

Definition 4 (Minimum). Let a nonempty set S Ă R and S. If α P S and

By definition, if maximum exists, supremum equals maximum and if mini-

then maxtan u does not exist.

then mintan u does not exist.

Definition 7 (Onto Functions or Mappings1 ). Consider f : X Ñ Y . Suppose

Definition 8 (Bijective Functions or Mappings). A function or mapping f is

Definition 9 (Continuity of Functions or Mappings). Consider a function or

|x ´ x0 | ă δ, then |f pxq ´ f px0 q| ă 

Therefore, f is said to be continuous at x0

Definition 10 (Indicator Functions). A function IA : R Ñ R is said to be an

Definition 11 (Positive and Negative Parts of f ). Let f be a function defined

f ` pxq “ maxtf pxq, 0u and f ´ pxq “ maxt´f pxq, 0u.

Definition 12 (Simple Functions). A simple function f is a function of the

1.3 Countable and Uncountable Sets

Definition 14 (Countable and Uncountable Sets). Let Jn “ 1, 2, 3, . . . , n and

1. A set S is said to be finite or finitely countable iff, for some n P N, there

2. A set S is said to be infinitely countable iff, for some n P N, there is a

3. A set S is said to be countable iff S is finite or infinitely countable.

4. A set S is uncountable iff S is not countable.

Example 3. Show that S “ t0, 2, 4, . . .u is a countable set.

Solution Let f : J Ñ S be defined as

f pnq “ 2pn ´ 1q for n P J

1.4 Neighborhood, Points and Sets

Definition 16 (Limit Points). A point x0 is called a limit point of S Ă R iff

Definition 17 (Closed Sets). A set S P R is called a closed set iff every x P S

Definition 18 (Interior Points). A point x0 is called an interior point of S Ă R

Definition 19 (Open Sets). S is an open set iff every point x P S is an interior

Definition 20 (Denseness). S is dense in X iff every point in X is a limit

Definition 21 (Closure). Let S Ă R and S 1 be the set of all limit points of S

Definition 22 (Bounded Sets). A subset M ‰ H of R is said to be a bounded

1.5 Sequences and Limits

Definition 25 (Limit, Convergent and Divergent Sequences). A sequence txn , n P

Definition 26 (Fields). Field F is a nonempty set satisfying the following prop-

1. α ` β P F and α.β P F for α and β P F

2. α ` β “ β ` α and α.β “ β.α for α and β P F

3. pα ` βq ` γ “ α ` pβ ` γq and pα.βq.γ “ α.pβ.γq for α, β and γ P F

4. α.pβ ` γq “ α.β ` α.γ for α, β and γ P F

6. For all α P F there exists β P F such that α ` β “ 0

7. For all α P F provided that α ‰ 0 there exists γ P F such that α.γ “ 1

Example 4. Show that the set of all real numbers R is a field.

1. α ` β P R and α.β P R for α and β P R

2. α ` β “ β ` α and α.β “ β.α for α and β P R

3. pα ` βq ` γ “ α ` pβ ` γq and pα.βq.γ “ α.pβ.γq for α, β and γ P R

4. α.pβ ` γq “ α.β ` α.γ for α, β and γ P R

6. For all α P R there exists β P R such that α ` β “ 0

7. For all α P R provided that α ‰ 0 there exists γ P R such that α.γ “ 1

it follows that the set of all real numbers R is a field.

|x ´ x0 | ă δ, then |f pxq ´ f px0 q| ă

}xn } ď }x ´ xn } ` }x} ď }x} `

}xn | ď supt|x1 |, . . . , |xN |, |x| ` u