DISCRETE MATHEMATICS AND ITS APPLICATIONS

Series Editor KENNETH H. ROSEN

FINITE-DIMENSIONAL
LINEAR ALGEBRA

Mark S. Gockenbach
Michigan Technological University

Houghton,

U.S.A.

CRC Press
Taylor &. Francis Croup
Boca Raton

London

New York

CRC Press is an

Taylor

&

Imprint of the
Francis Croup, an Informa business

A CHAPMAN

& HALL BOOK

Contents

Preface

xv

About the author

Some
1.1

problems posed

on

vector spaces

Linear equations

1.1.1

Systems of linear algebraic equations

1.1.2

Linear ordinary differential equations

1.1.3

Some

1.1.4
1.2

xxi

interpretation:
equation

The structure of the solution set

to a linear

Finite fields and applications in discrete mathematics

Best approximation

1.2.1

Overdetermined linear systems

1.2.2

Best

approximation by

8
11

polynomial

1.3

Diagonalization

13

1.4

Summary

17

Fields and vector spaces

2.1

2.2

19

Fields

19

2.1.1

Definition and

2.1.2

Basic properties of fields

19

examples

21

Vector spaces
2.2.1

29

Examples

of vector spaces

31

2.3

Subspaces

38

2.4

Linear combinations and spanning sets

43

2.5

Linear

independence

50

2.6

Basis and dimension

57

2.7

Properties of bases

66

2.8

Polynomial interpolation and the Lagrange

2.8.1

2.9

basis

Secret sharing

Continuous

piecewise

73
77

polynomial functions

82

2.9.1

Continuous piecewise linear functions

84

2.9.2

Continuous

87

2.9.3

Error in

piecewise quadratic

functions

polynomial interpolation

90

ix

Contents

Linear
3.1

operators

93

Linear operators

93

3.1.1
3.2

3.3

3.7

3.2.3
3.2.4

Derivative and differential operators

Representing spanning sets and bases using matrices

3.2.5

The transpose of

linear operator

on

Euclidean

The matrix of

101
103

matrix

103

104

linear operator

on

general

107
108
vector spaces

Linear operator equations

111
116

3.4.2

Homogeneous linear equations

Inhomogeneous linear equations

118

3.4.3

General solutions

120

Existence and

uniqueness

The kernel of

of solutions

117

124

operator and injectivity

3.5.2
The rank of a linear operator and surjectivity
Existence and uniqueness
3.5.3
The fundamental theorem; inverse operators
linear

124
126
128
131

3.6.1

The inverse of

linear operator

133

3.6.2

The inverse of

matrix

134

Gaussian elimination

142

3.7.1

Computing A'1

148

3.7.2

Fields other than R

149

Newton's method

3.9

Linear

153

3.9.1

ordinary differential equations

The dimension of ker(L)

3.9.2

Finding

basis for

158
158

ker(L)

3.9.2.1

The easy

3.9.2.2

The case of

3.9.2.3

The case of

case:

161

Distinct real roots

real roots

repeated
complex roots
for linear independence

3.9.3

The Wronskian test

3.9.4

The Vandermonde matrix

162
162
163
163
166

Graph theory

168

3.10.1 The incidence matrix of

graph

168

3.10.2 Walks and matrix multiplication

169

Graph isomorphisms
Coding theory

171

Generator matrices; encoding and decoding

3.11.2 Error correction

177

3.10.3
3.11

spaces

Isomorphic vector spaces

3.3.1
Injective and surjective functions; inverses

3.8

3.10

101

The matrix of

3.5.1

3.6

101

3.2.2

3.4.1

3.5

95

More properties of linear operators

Vector spaces of operators
3.2.1

3.3.2
3.4

Matrix operators

3.11.1

3.11.3

The

probability

of

errors

3.12 Linear programming

3.12.1

Specification of linear programming problems

175

179
181
183
184

Contents

xi

3.12.2 Basic theory

3.12.3 The simplex method

Finding

3.12.3.2

Unbounded LPs

199

3.12.3.3

Degeneracy and cycling

200

on

an

initial BPS

the standard LPs

Determinants and eigenvalues

4.1

4.1.1

Permutations

4.1.2

The

196

202
205

The determinant function

206
210
of the determinant

complete expansion

212

4.2

Further properties of the determinant function

217

4.3

Practical computation of det(A)

4.3.1
A recursive formula for det(A)

221

4.3.2

226

Cramer's rule

4.4

A note about

4.5

Eigenvalues

4.6

and the characteristic

4.7
4.8

4.9

Eigenvalues
Diagonalization

polynomial

of real matrix

251

Eigenvalues
operators
Systems of linear ODEs
4.8.1
Complex eigenvalues

257
259

the initial value

4.8.2

Solving

4.8.3

Linear systems in matrix form

problem

261
265

4.9.1

265

unimodular matrices

Totally
Transportation problems

Invariant
5.1.1
5.1.2

268
273

subspaces

Direct

273
276

sums

and

277

5.2.1

Eigenspaces
generalized eigenspaces
eigenspaces
Appendix: Beyond generalized eigenspaces

5.2.2

The

294

Generalized

Cayley-Hamilton

theorem

5.3

Nilpotent operators

5.4

The Jordan canonical form of

5.5

The matrix

5.6

260

Integer programming

The Jordan canonical form

5.2

232
235
241

of linear

4.9.2

5.1

224

230

polynomials

4.5.1

191

3.12.3.1

3.12.4 Variations
4

186

283
290
300

matrix

309

318

exponential

5.5.1

Definition of the matrix

319

5.5.2

Computing the

319

matrix

exponential
exponential

Graphs and eigenvalues

325

5.6.1

325

5.6.2

5.6.3
5.6.4

Cospectral graphs
Bipartite graphs and eigenvalues
Regular graphs
Distinct eigenvalues of a graph

326
328
330

xii
6

Contents

Orthogonality
6.1

The

adjoint

Orthogonal
6.3.1

6.4

The

6.8
6.9

342

linear operator

343

vectors and bases

350

bases

351
357

Overdetermined linear systems

361
368

Least-squares polynomial approximation

371

Orthogonal complements
6.6.1

6.7

337

linear operator

The Gram-Schmidt process

6.5.1

6.6

333

Orthogonal
projection theorem

6.4.1

6.5

of

The adjoint of

6.2.1
6.3

333

approximation
products
Examples of norms and inner products

6.1.1
6.2

and best

Norms and inner

377

The fundamental theorem of linear

algebra

revisited

Complex inner product spaces

6.7.1
Examples of complex inner product spaces
6.7.2
Orthogonality in complex inner product spaces
6.7.3
The adjoint of a linear operator
More on polynomial approximation
6.8.1 A weighted L2 inner product
The energy inner

386
388
....

Piecewise

6.9.2

Continuous piecewise quadratic functions

394

397
401
404
407

6.9.3
6.10

Higher degree finite element spaces

Gaussian quadrature
6.10.1 The trapezoidal rule and Simpson's

409
411
rule

412

6.10.2 Gaussian

quadrature
6.10.3 Orthogonal polynomials
6.10.4 Weighted Gaussian quadrature

413
415
419

6.11 The Helmholtz decomposition

6.11.1 The divergence theorem
6.11.2

420
421

Stokes's theorem

422

6.11.3 The Helmholtz decomposition

7

423

The

spectral theory of symmetric

7.1

The spectral theorem for symmetric matrices

7.2

matrices

425
425

7.1.1

Symmetric positive definite matrices

428

7.1.2

Hermitian matrices

430

The spectral theorem for normal matrices

7.2.1

7.3

389
390

product and Galerkin's method

polynomials

6.9.1

381

Outer

434

products and the spectral decomposition

Optimization and the Hessian

matrix

....

437
440

7.3.1

Background

440

7.3.2

Optimization of quadratic functions

441

7.3.3

Taylor's theorem

443

7.3.4

First- and second-order

7.3.5

Local

optimality

quadratic approximations

conditions

444
446

Contents
7.4
7.5

Lagrange multipliers
Spectral methods for differential equations

448

7.5.1

454

The

singular

8.1

Introduction to the SVD

8.1.1

8.3
8.4

8.5

value

463
463

singular

matrices

467

The SVD for

general matrices
Solving least-squares problems using the SVD
The SVD and linear inverse problems
8.4.1
Resolving inverse problems through regularization

470
476

483
.

489

8.4.2

The truncated SVD method

489

8.4.3

Tikhonov

490

regularization

The Smith normal form of

matrix

494

An algorithm to compute the Smith normal form

Applications of the Smith normal form

8.5.2

Matrix factorizations and numerical linear

algebra

507
507

Operation counts
Solving Ax = b using

9.1.2

495
501

The LU factorization
9.1.1

9.2

456

decomposition

The SVD for

8.5.1

9.1

453

Eigenpairs of the differential operator

Solving the BVP using eigenfunctions

7.5.2

8.2

xiii

512
the LU factorization

514

Partial pivoting

516

9.2.1

517

9.2.3

Finite-precision arithmetic
Examples of errors in Gaussian elimination
Partial pivoting

9.2.4

The PLU factorization

522

9.2.2

9.3

The

9.4

Matrix
9.4.1

Cholesky

519

factorization

524
530

norms

Examples
sensitivity of

9.5

The

9.6

Numerical

9.6.1

518

of induced matrix
linear

systems

534

norms

to errors

537

stability

Backward

542

error

analysis

543

9.6.2

545

9.7

Analysis of Gaussian elimination with partial pivoting

The sensitivity of the least-squares problem

9.8

The QR factorization

554

9.8.1

556

Solving

the

least-squares problem

9.8.2

9.9

Computing the QR factorization

9.8.3
Backward stability of the Householder QR algorithm
9.8.4
Solving a linear system
Eigenvalues and simultaneous iteration

548

556
.

561
562
564

9.9.1

Reduction to

9.9.2

The power method

566

9.9.3

Simultaneous iteration

567

9.10 The QR algorithm

triangular form

564

572

Contents

xiv

9.10.1

A practical QR
9.10.1.1

Reduction

9.10.1.2

The

9.10.1.3
LO

Analysis in vector
10.1 Analysis in R"

573

algorithm
to upper

form

Hessenberg
algorithm
algorithm

explicitly shifted QR
The implicitly shifted QR

574
576
579
581

spaces

581

Convergence and continuity in

10.1.2 Compactness
10.1.3 Completeness of R"
10.1.4 Equivalence of norms on R
10.1.1

Rn

582

584
586
586

10.2 Infinite-dimensional vector spaces

10.2.1 Banach and Hubert spaces

590

10.3 Functional analysis

596

10.3.1

The dual of

592

Hilbert space

600

605

10.4 Weak convergence

10.4.1

611

Convexity

A The Euclidean algorithm

617

A.0-1

Computing multiplicative

A.0.2

Related results

inverses in

Zp

618

619

Permutations

621

Polynomials
C.l Rings of polynomials
C.2 Polynomial functions

625

C. 2.1

Factorization of

625
630

polynomials

D Summary of analysis in R

632
633
633

D.0.3

Convergence
Completeness of R
Open and closed sets

D.0.4

Continuous functions

636

D. 0.1
D.0.2

634
635

Bibliography

637

Index

641