MATH 304

Linear Algebra
Lecture 29:
Orthogonal sets.
The Gram-Schmidt process.

Orthogonal sets
Let V be an inner product space with an inner
product h, i and the induced norm k k.
Definition. A nonempty set S V of nonzero
vectors is called an orthogonal set if all vectors in
S are mutually orthogonal. That is, 0
/ S and
hx, yi = 0 for any x, y S, x 6= y.
An orthogonal set S V is called orthonormal if
kxk = 1 for any x S.
Remark. Vectors v1 , v2 , . . . , vk V form an
orthonormal set if and only if

1 if i = j
hvi , vj i =
0 if i 6= j

Examples. V = Rn , hx, yi = x y.
The standard basis e1 = (1, 0, 0, . . . , 0),
e2 = (0, 1, 0, . . . , 0), . . . , en = (0, 0, 0, . . . , 1).
It is an orthonormal set.
V = R3 , hx, yi = x y.
v1 = (3, 5, 4), v2 = (3, 5, 4), v3 = (4, 0, 3).
v1 v2 = 0, v1 v3 = 0, v2 v3 = 0,
v1 v1 = 50, v2 v2 = 50, v3 v3 = 25.
Thus the set {v1 , v2 , v3 } is orthogonal but not
orthonormal. An orthonormal set is formed by
normalized vectors w1 = kvv11 k , w2 = kvv22 k ,
w3 = kvv33 k .

 V = C [, ], hf , g i =

f (x)g (x) dx.

f1 (x) = sin x, f2 (x) = sin 2x, . . . , fn (x) = sin nx, . . .

hfm , fn i =

sin(mx) sin(nx) dx

1
cos(mx nx) cos(mx + nx) dx.
2

sin(kx) 
= 0 if k Z, k 6= 0.

k
x=
Z
cos(kx) dx =
dx = 2.

cos(kx) dx =

k = 0 =

Z

1 
cos(m n)x cos(m + n)x dx
hfm , fn i =
2

if m = n
=
0 if m 6= n

Thus the set {f1 , f2 , f3 , . . . } is orthogonal but not

orthonormal.
It is orthonormal with respect to a scaled inner
product
Z
1
hhf , g ii =
f (x)g (x) dx.

Orthogonality = linear independence

Theorem Suppose v1 , v2 , . . . , vk are nonzero
vectors that form an orthogonal set. Then
v1 , v2 , . . . , vk are linearly independent.
Proof: Suppose t1 v1 + t2 v2 + + tk vk = 0
for some t1 , t2 , . . . , tk R.
Then for any index 1 i k we have
ht1 v1 + t2 v2 + + tk vk , vi i = h0, vi i = 0.
= t1 hv1 , vi i + t2 hv2 , vi i + + tk hvk , vi i = 0
By orthogonality, ti hvi , vi i = 0 = ti = 0.

Orthonormal bases
Let v1 , v2 , . . . , vn be an orthonormal basis for an
inner product space V .
Theorem Let x = x1 v1 + x2 v2 + + xn vn and
y = y1 v1 + y2 v2 + + yn vn , where xi , yj R. Then
(i) hx, yi = x1 y1 + x2 y2 + + xn yn ,
p
(ii) kxk = x12 + x22 + + xn2 .
Proof: (ii) follows from (i) when y = x.
+
*
+
* n
n
n
n
X
X
X
X
yj vj
xi vi ,
hx, yi =
xi vi ,
yj vj =
i=1

j=1
n
n
XX

xi yj hvi , vj i =

i=1 j=1

j=1

i=1

n
X
i=1

xi yi .

Let v1 , v2 , . . . , vn be a basis for an inner product

space V .
Theorem If the basis v1 , v2 , . . . , vn is an
orthogonal set then for any x V
hx, v1 i
hx, v2 i
hx, vn i
x=
v1 +
v2 + +
vn .
hv1 , v1 i
hv2 , v2 i
hvn , vn i
If v1 , v2 , . . . , vn is an orthonormal set then
x = hx, v1 iv1 + hx, v2 iv2 + + hx, vn ivn .
Proof: We have that x = x1 v1 + + xn vn .
= hx, vi i = hx1 v1 + + xn vn , vi i, 1 i n.
= hx, vi i = x1 hv1 , vi i + + xn hvn , vi i
= hx, vi i = xi hvi , vi i.

Let V be a vector space with an inner product.

Suppose that v1 , . . . , vk V are nonzero vectors
that form an orthogonal set. Given x V , let
hx, vk i
hx, v1 i
v1 + +
vk , o = x p.
p=
hv1 , v1 i
hvk , vk i
Let W denote the span of v1 , . . . , vk .
Theorem (a) o w for all w W (denoted o W ).
(b) kok = kx pk = min kx wk.
wW

Thus p is the orthogonal projection of the vector

x on the subspace W . Also, p is closer to x than
any other vector in W , and kok = dist(x, p) is the
distance from x to W .

Orthogonalization
Let V be a vector space with an inner product.
Suppose x1 , x2 , . . . , xn is a basis for V . Let
v1 = x1 ,
hx2 , v1 i
v1 ,
hv1 , v1 i
hx3 , v1 i
hx3 , v2 i
v3 = x3
v1
v2 ,
hv1 , v1 i
hv2 , v2 i
.................................................
hxn , vn1 i
hxn , v1 i
v1
vn1 .
vn = xn
hv1 , v1 i
hvn1 , vn1 i
v2 = x2

Then v1 , v2 , . . . , vn is an orthogonal basis for V .

The orthogonalization of a basis as described above
is called the Gram-Schmidt process.

Normalization
Let V be a vector space with an inner product.
Suppose v1 , v2 , . . . , vn is an orthogonal basis for V .
v1
v2
vn
Let w1 =
, w2 =
,. . . , wn =
.
kv1 k
kv2 k
kvn k
Then w1 , w2 , . . . , wn is an orthonormal basis for V .
Theorem Any finite-dimensional vector space with
an inner product has an orthonormal basis.
Remark. An infinite-dimensional vector space with
an inner product may or may not have an
orthonormal basis.