MAE 107 Numerical Methods Notes

Section 1: Linear Algebra

is a Linear Operator if it satisfies the following two conditions:

L ( x + y )=L ( x )+ L( y)
L ( cx )=c L(x )
Example 1:

f ( x )=ax

f ( x + y )=a ( x + y )=ax+ ay=f ( x ) + f ( y )

f ( cx ) =a ( cx )=c ( ax )=cf (x)
This example satisfies both conditions so it is a linear function
Example 2:

f ( x )=ax +b

f ( x + y )=a ( x + y ) +b=ax +ay +b ax +ay +2 b=f ( x )+ f ( y )

f ( cx ) =a ( cx ) +b c ( ax ) +cb=cf (x)
This example fails to satisfy either condition so it is not a linear function despite
your intuition.
A linear equation is homogeneous if it has the form
in the form

L ( x )=0

and inhomogeneous

L ( x )=b

To solve a linear equation we must find the inverse such that

x=L1 (b) . This idea

is key in solving systems of equations. We can rewrite a system of equations in

matrix form as shown below.

a11 x +a12 y=b1

a21 x + a22 y=b2

][ ] [ ]

a11 a12 x
b
= 1
a21 a22 y
b2

Ax=b
The coefficient matrix, A, is a linear operator that transforms the vector x, into the
vector b. Solving this system amounts to finding the inverse of A:

x= A1 b

What this is saying is if A changes the vector x into b then the inverse of A will
transform the vector b back into x.
Another interpretation of this solution is that it is the point of intersection of the two
straight line described by the system of equations. There are three possibilities, the

lines can be parallel in which case there is no solution, they could intersect at a
single point implying a single solution, or the lines could be identical, intersecting at
all points on the line, implying an infinite number of solutions.

Images taken from Linear Algebra and its Applications 3 rd Ed. David Lay
A third interpretation of this system can be seen if we rewrite the matrix form as a
vector equation.

[ ] [ ] []
a11
a
b
x + 12 y = 1
a21
a22
b2

x and y are the weights of the column vectors of A that are needed to obtain the
vector b. This can be seen graphically as shown. In the diagram, the column vectors
of A are labeled

v1

and

v2 .

This image shows, at least qualitatively,

that any vector b, can be made with the
appropriate combination of vectors

v1

and

v 2 . The combination of

these vectors is the solution vector x.

(Note: This only holds if the vector b is
in the span of column vectors. More on
this later)

Image taken from David Lay

These interpretations can be extended to any dimension
difficult to visualize beyond

R3 . A few graphs in

R3

but they may be

are given below.

One Solution
No Solutions
interpretation
Images taken from David Lay

Infinite Solutions

Vector

A set of vectors are linearly independent if only the trivial solution satisfies the
following equation:

c 1 v 1+ c 2 v 2 +c 3 v 3 ++ c n v n=0

[ v1 v2 v3 vn ]

[]

c1
c2
c 3 =0

cn

Otherwise, the set is said to be linearly dependent. If a set is linearly dependent, at

least one vector can be made from a combination of the other vectors. The Span of
a set of vectors is all linear combination of the vectors.
Example 3:

[ ]

1 2 3
2 4 7 =[ v 1 v2 v 3 ]
2 4 2

This set of vectors are linearly dependent because

v 2=2 v 1

and

span of these vectors can be described by the plane:

c 1 v 1+ c3 v3

. One consequence of linear dependence is

that a solution only exists if it lies in the plane. We can no longer

create any vector b, in

R3 .

the

(Image taken from David Lay)

A second consequence of linear dependence is that if b is in spanA then there are
infinitely many solutions to the system. This is because we have an extra vector in
the spanA. Going back to the previous example, once a single solution is obtained,
any number of solutions can be obtained by adding

c v 12 c v 2 . Another way to

look at it is:

c 1 v 1+ c 2 v 2 +c 3 v 3=b
~
c 1 v 1+ c3 v3 =bc 2 v 2=b
v2

If b and

are in the plane of

span (v 1 v 3 ) then an infinite number of vectors

~
b can be created by varying the constant

c 2 and all these new vectors must lie

in that same plane, implying an infinite number of solutions exists.

The minimum number of vectors that span a space forms a basis for that space.
Going back to the previous example, the basis can be formed with
.

v1 v3

or

v2 v3

v 1v 2 cannot form a basis because they are a scalar multiple of each other.

In general, rectangular coefficient matrices cause problems because there are

usually either no solutions or infinitely many solutions.
System 1
system 2

[ ][ ] [ ]
a11 a12
b
x = 1
a21 a22
b2
y
a31 a32
b3

a11 a12 a13 a14

a21 a22 a23 a24
a31 a32 a33 a34

System 1 will generally not have a solution

many solutions
unless b happens to be in the spanA
vector in A

][ ] [ ]
w
b
x = 1
b2
y
b3
z

System 2 will generally have infinitely

because there is an extra column

Ideally, we want a square coefficient matrix with independent columns. This will
represent n equations with n unknown and a solution is guaranteed to exist.
The coefficient matrix A is said to be in row echelon form if it has a form similar to:

[ ]
5
0
0
0

7
1
0
0

1
5
9
0

1
0
5
7

The important thing is that the main diagonal is nonzero and all entries below the
main diagonal are zero. A matrix can be transformed into echelon form by

performing a series of row operations to eliminate all entries below the diagonal. All
three types of row operations change the overall effect of A on x so to compensate,
we must perform the same operations on the vector b. To simplify this process the
row operations are performed on the augmented matrix

[ a1 a2 anb ]

1. Add a multiple of one row to another

2. Interchange rows
3. Multiple all entries of a row by a scalar
Once the matrix is in echelon form, the solution vector can be found by back
substitution. We start by solving for the last variable and work are way up.
Computationally, this method is dominated by the

2 3
n
3

operations required to get

the matrix in echelon form. The back substitution is computationally negligible.

Example 4: Solve the following system of equations

[ ][ ] [ ]
2 0 3 x
3
=
4 0 9 y
9
8 8 5 z
2

Write the system as an augmented matrix and transform it into echelon form

2 0 3 3
4 0 9 9
8 8 5 2

e 22 e1

2 0 3 3
0 0 3 3
8 8 5 2

e 34 e1

[
[

2 0
3 3
0 0
3 3
0 8 7 10

2 0
3 3
0 0
3 3
0 8 7 10

]
]