Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
MAT 415/515
Fall Semester 2013-14
Lecture 3 Notes
These notes correspond to Section 5.2 in the text.
Gram-Schmidt Orthogonalization
We have seen that it can be very convenient to have an orthonormal basis for a given vector
space, in order to compute expansions of arbitrary vectors within that space. Therefore, given a
non-orthonormal basis, it is desirable to have a process for obtaining an orthonormal basis from it.
Fortunately, we have such a process, known as Gram-Schmidt orthogonalization. Suppose that
we have a linearly independent, but not orthonormal, set of functions {
1
,
2
, . . .} that span a given
vector space V . To construct an orthonormal set {
1
,
2
, . . .} from this set, we proceed as follows.
First, to obtain
1
, we simply normalize
1
:
1
=
1
.
Next, to obtain
2
, we need to ensure that it is orthogonal to
1
, and then normalize it.
As an intermediate step, we seek a function
2
of the form
2
=
2
+c
12
1
such that
1
|
2
= 0. Then, we can set
2
=
2
/
2
. Taking the scalar product of both sides of
the above equation with
1
, we obtain
0 =
1
|
2
=
1
|
2
+c
12
1
|
1
.
Because the
j
are orthonormal, it follows that
c
12
=
1
|
2
.
We conclude that
2
can be obtained as follows:
2
=
2
1
|
2
2
=
2
.
We now have a set of two functions that is orthonormal.
Now, to obtain
3
, we must ensure that it is orthogonal to
1
and
2
, and then normalized.
To that end, we seek a function
3
of the form
3
=
3
+c
13
1
+c
23
2
such that
1
|
3
=
2
|
3
= 0. Then, we can set
3
=
3
/
3
. Taking the scalar product of
both sides of the above equation with
1
, and then separately,
2
, we obtain
0 =
1
|
3
=
1
|
3
+c
13
1
|
1
+c
23
1
|
2
0 =
2
|
3
=
2
|
3
+c
13
2
|
1
+c
23
2
|
2
.
1
Because the
j
are orthonormal, it follows that
c
13
=
1
|
3
, c
23
=
2
|
3
.
We conclude that
3
can be obtained as follows:
3
=
3
1
|
3
1
2
|
3
3
=
3
.
We now have a set of three functions that are orthonormal.
Continuing this process, we see that we can obtain each function
j
as follows:
j
=
j
j1
k=0
k
|
j
j
=
j
.
This yields a set of functions {
1
,
2
, . . .} that is an orthonormal basis of the space spanned by
{
1
,
2
, . . .}, with respect to the scalar product that is used.
Example We wish to obtain a set of orthonormal polynomials with respect to the scalar product
f|g =
_
1
1
f
(s)g(s) ds.
This will be accomplished by applying Gram-Schmidt orthogonalization to the set {1, x, x
2
, x
3
, . . .}.
Setting
j
(x) = x
j
for j = 0, 1, 2, . . . , our orthogonal set {
j
}, j = 0, 1, 2, . . . ,, is obtained as follows:
0
(x) =
0
(x)
= 1,
0
(x) =
0
(x)
=
1
1|1
1/2
=
1
_
_
1
1
1 ds
_
1/2
=
1
2
,
1
(x) =
1
(x)
0
|
1
0
(x)
= x
_
1
x
_
1
2
= x
1
2
_
1
1
s ds
= x
1
2
0
= x,
2
1
(x) =
1
(x)
=
x
x|x
1/2
=
x
_
_
1
1
s
2
ds
_
1/2
=
_
3
2
x,
2
(x) =
2
(x)
0
|
2
0
(x)
1
|
2
1
(x)
= x
2
_
1
x
2
_
1
_
_
3
2
x
x
2
_
_
3
2
x
= x
2
1
2
_
1
1
s
2
ds
3
2
x
_
1
1
s
3
ds
= x
2
1
2
2
3
3
2
0
= x
2
1
3
,
2
(x) =
2
(x)
=
x
2
1
3
x
2
1
3
x
2
1
3
_
1/2
=
x
2
1
3
_
_
1
1
_
x
1
3
_
2
ds
_
1/2
=
_
5
2
_
3
2
x
2
1
2
_
.
Continuing this process, we obtain
3
(x) =
_
7
2
_
5
2
x
3
3
2
x
_
.
In general,
n
(x) =
_
2n + 1
2
P
n
(x),
where P
n
(x) is the Legendre polynomial of nth degree. We will learn more about these orthogonal
(but not orthonormal) polynomials later in this course. 2
While Gram-Schmidt orthogonalization can be applied to the monomial bais {1, x, x
2
, x
3
, . . .}
to obtain an orthonormal sequence of polynomials, it can be quite cumbersome, as can be seen from
the preceding example. However, a modication of this procedure can yield a much more ecient
approach.
Suppose that we have already generated a sequence of n orthonormal polynomials
0
,
1
,
2
, . . . ,
n1
with respect to some scalar product
f|g =
_
b
a
f
(x)g(x)w(x) dx,
3
where
j
is of degree j for j = 0, 1, 2, . . . , n 1. Then, to obtain
n
, which of degree n, we
orthogonalize x
n1
(x), which is of degree n, against
0
,
1
, . . . ,
n1
using the same approach as
in Gram-Schmidt orthogonalization. That is, we compute
n
(x) = x
n1
(x)
n1
j=0
j
|x
n1
j
(x),
n
(x) =
n
(x)
.
Now, consider the scalar product
j
|x
n1
. Using the properties of the scalar product, we
have
j
|x
n1
= x
j
|
n1
.
However, because
0
,
1
, . . . ,
n1
are orthonormal, p|
n1
= 0 if p(x) is any polynomial of
degree less than n 1. Because x
j
(x) is of degree j + 1, it follows that
j
|x
n1
= 0 whenever
j + 1 < n 1, or j < n 2. Therefore, our orthogonalization procedure simplies to
n
(x) = x
n1
(x)
n2
|x
n1
n2
(x)
n1
|x
n1
n1
(x),
n
(x) =
n
(x)
.
That is, any family of orthogonal polynomials satises a three-term recurrence relation, in which
each polynomial depends on the previous two. Table lists several families of orthogonal polynomials
that can be generated from such a recurrence relation; we will see some of these families later in
the course.
Polynomials Scalar Product
Legendre
_
1
1
P
n
(x)P
m
(x) dx = 2
mn
/(2n + 1)
Shifted Legendre
_
1
0
P
n
(x)P
m
(x) dx =
mn
/(2n + 1)
Chebyshev, rst kind
_
1
1
T
n
(x)T
m
(x)(1 x
2
)
1/2
dx =
mn
/(2
n0
)
Shifted Chebyshev, rst kind
_
1
0
T
n
(x)T
m
(x)[x(1 x)]
1/2
dx =
mn
/(2
n0
)
Chebyshev, second kind
_
1
1
U
n
(x)U
m
(x)(1 x
2
)
1/2
dx =
mn
/2
Leguerre
_
0
L
n
(x)L
m
(x)e
x
dx =
mn
Associated Laguerre
_
0
L
k
n
(x)L
k
m
(x)e
x
dx =
mn
(n +k)!/n!
Hermite
_
H
n
(x)H
m
(x)e
x
2
dx = 2
n
mn
n!
As can be seen in the following example, Gram-Schmidt orthogonalization can be applied to
vectors in any inner product space, such as vectors in R
n
.
Example Given the vectors in R
3
,
|a
1
=
_
_
1
1
2
_
_
, |a
2
=
_
_
1
2
3
_
_
, |a
3
=
_
_
0
1
1
_
_
,
we will use Gram-Schmidt orthogonalization to obtain an orthonormal set of vectors, {|b
1
, |b
2
, |b
3
}.
We have
|b
1
=
|a
1
a
1
|a
1
1/2
4
=
1
6
1/2
|a
1
=
1
6
_
_
1
1
2
_
_
,
|b
2
= |a
2
b
1
|a
2
|b
1
=
_
_
1
2
3
_
_
6
1
6
_
_
1
1
2
_
_
=
_
_
1/2
1/2
0
_
_
,
|b
2
=
|b
2
|b
1/2
=
1
_
1
2
_
1/2
|b
=
1
2
_
_
1
1
0
_
_
,
|b
3
= |a
3
b
1
|a
3
|b
1
b
2
|a
3
|b
2
=
_
_
0
1
1
_
_
6
_
1
6
_
_
1
1
2
_
_
2
1
2
_
_
1
1
0
_
_
=
_
_
0
1
1
_
_
+
1
6
_
_
1
1
2
_
_
1
2
_
_
1
1
0
_
_
=
2
3
_
_
1
1
1
_
_
,
|b
3
=
|b
3
|b
1/2
=
1
_
4
3
_
1/2
|b
=
1
3
_
_
1
1
1
_
_
.
We conclude that our orthonormal set of vectors is
|b
1
=
1
6
_
_
1
1
2
_
_
, |b
2
=
1
2
_
_
1
1
0
_
_
, |b
3
=
1
3
_
_
1
1
1
_
_
.
2
5