Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
M. Anthony, M. Harvey
MT2118, 2790118
2011
Undergraduate study in
Economics, Management,
Finance and the Social Sciences
This is an extract from a subject guide for an undergraduate course offered as part of the
University of London International Programmes in Economics, Management, Finance and
the Social Sciences. Materials for these programmes are developed by academics at the
London School of Economics and Political Science (LSE).
For more information, see: www.londoninternational.ac.uk
This guide was prepared for the University of London International Programmes by:
Professor M. Anthony, BSc, MA, PhD and Dr M. Harvey, BSc, MSc, PhD, Department of
Mathematics, The London School of Economics and Political Science.
This is one of a series of subject guides published by the University. We regret that due to
pressure of work the authors are unable to enter into any correspondence relating to, or arising from, the guide. If you have any comments on this subject guide, favourable or unfavourable, please use the form at the back of this guide.
Contents
Contents
Preface ..........................................................................................................v
Introduction ................................................................................................. 1
1.1 This course ........................................................................................................ 1
1.2 Reading............................................................................................................. 2
1.3 Online study resources ....................................................................................... 3
1.4 Using the guide ................................................................................................. 4
1.5 Examination ...................................................................................................... 4
1.6 The use of calculators ........................................................................................ 5
2 Matrices, vectors, and systems of linear equations ................................. 7
2.1 Introduction ...................................................................................................... 7
2.2 Vectors and matrices .......................................................................................... 7
2.2.1 Row operations....................................................................................... 10
2.2.2 Determinants. ......................................................................................... 11
2.2.3 Invertible matrices .................................................................................. 13
2.3 Rank, range, null space, and linear equations ................................................... 15
2.3.1 The rank of a matrix ................................................................................ 15
2.3.2 Rank and systems of linear equations...................................................... 16
2.3.3 General solution of a linear system in vector notation .............................. 19
2.3.4 Null space .............................................................................................. 20
2.3.5 Range .................................................................................................... 21
2.4 Learning outcomes ......................................................................................... .22
2.5 Sample examination questions ......................................................................... 23
2.6 Answers to or comments on selected activities ................................................. 24
2.7 Sketch answers to or comments on selected questions ..................................... 26
3 Vector spaces, basis, dimension .............................................................. 29
3.1 Introduction .................................................................................................... 29
3.2 Vector spaces .................................................................................................. 29
3.2.1 Defnition of a vector space...................................................................... 29
3.2.2 Examples ................................................................................................ 31
3.3 Subspaces........................................................................................................ 32
3.3.1 An alternative characterisation of a subspace .......................................... 35
3.4 Subspaces connected with matrices ................................................................. 35
3.4.1 Null space .............................................................................................. 35
3.4.2 Range .................................................................................................... 36
3.5 Linear span...................................................................................................... 36
3.5.1 Lines and Planes in 3 ............................................................................ 37
3.5.2 Row space and column space ................................................................. 38
3.6 Linear independence. ....................................................................................... 38
3.7 Testing linear independence in n .................................................................... 40
3.8 Bases and dimension ....................................................................................... 43
3.8.1 Coordinates ............................................................................................ 45
3.8.2 Dimensions............................................................................................. 46
3.8.3 Dimension and bases of subspaces ......................................................... 47
3.9 Finding a basis for a linear span in n .............................................................. 48
3.10 Basis and dimension of range and null space ................................................. 49
3.11 Learning outcomes ........................................................................................ 52
3.12 Sample examination questions ....................................................................... 52
3.13 Answers to or comments on selected activities ............................................... 54
3.14 Sketch answers to or comments on selected questions ................................... 58
ii
Contents
iii
iv
Preface
This subject guide is not a course text. It sets out the logical sequence in
which to study the topics in the course. Where coverage in the main texts
is weak, it provides some additional background material. Further reading
is essential.
We are grateful to Keith Martin and James Ward for their careful reading
of a draft of the guide, and for their many helpful comments.
Notes
vi
Chapter 1: Introduction
Chapter 1
Introduction
In this very brief introduction, we aim to give you an idea of the nature
of this course and to advise on how best to approach it. We give general
information about the contents and use of this subject guide, and on
recommended reading and how to use the textbooks.
Aims
This half course is designed to:
enable you to acquire further skills in the techniques of linear algebra, as
well as understanding of the principles underlying the subject
prepare you for further courses in mathematics and/or related
disciplines (e.g. economics, actuarial science).
Learning outcomes
At the end of this course, and having completed the Essential reading and
activities, you should have:
used the concepts, terminology, methods and conventions covered in this
course to solve the mathematical problems in this subject
the ability to demonstrate an understanding of the underlying principles
of the subject
Topics covered
Descriptions of topics to be covered appear in the relevant chapters.
However, it is useful to give a brief overview at this stage.
Diagonalisation of 2 by 2 matrices was covered in 05B Mathematics
2. Here, the techniques are extended to larger matrices, but more
importantly the theoretical underpinning is given.This involves the
important ideas of vector spaces, linear independence, basis, dimension
and linear transformations. Wethen explore eigen values, diagonalisation,
and the applications of these techniques to solving systems of difference
and differential equations (extending methods from 05B Mathematics
2),and tofinding powers of matrices.Then,we study inner products,
orthogonality, and quadratic forms. Next, we turn our attention to the
ideas of direct sums, projections and least squares, topics that have useful
applications in many areass uch as statistics and econometrics. Finally,
we look at complex matrices andc omplex vector spaces, extending and
generalising the theory developed in the previous chapters.
Throughout, the emphasis is on the theory as much as on the methods.
That is to say, the aim in this course is not only to provide you with useful
technique sand methods of linear algebra, but to enable you to understand
why these techniques (and those you encountered in your earlier
mathematical study) work.
Not all chapters of the guide are the same length. It should not be thought
that you should spend the same amount of time on each chapter. We will
not try to specify how much relative time should be spent on each: that will
vary from person to person and we do not want to be prescriptive.
As a very rough guide (bearing in mind that this must vary from individual
to individual), we would suggest that the percentages of time spent on each
chapter are something along the lines suggested in the following table.
(This should not be taken as any indication about the composition of the
examination.)
Chapter
10
17
10
10
10
13
13
17
1.2 Reading
There are many books that would be useful for this course, and the course
does not follow any particular one.
Most topics in this course are covered in great detail by a large number of
books. For that reason, we have resisted the temptation to specify Essential
reading in each chapter of the guide. What is true, however, is that
textbook reading is essential. Textbooks will provide more in-depth
explanations than you will find in this guide (which is explicitly not meant
Chapter 1: Introduction
The VLE
The VLE, which complements this subject guide, has been designed to
enhance your learning experience, providing additional support and a
sense of community. It forms an important part of your study experience
with the University of London and you should access it regularly.
The VLE provides a range of resources for EMFSS courses:
Self-testing activities: Doing these allows you to test your own
understanding of subject material.
Electronic study materials: The printed materials that you receive from
the University of London are available to download, including updated
reading lists and references.
Past examination papers and Examiners commentaries: These provide
advice on how each examination question might best be answered.
A student discussion forum: This is an open space for you to discuss
interests and experiences, seek support from your peers, work
collaboratively to solve problems and discuss subject material.
Videos: There are recorded academic introductions to the subject,
interviews and debates and, for some courses, audio-visual tutorials
and conclusions.
Recorded lectures: For some courses, where appropriate, the sessions
from previous years Study Weekends have been recorded and made
available.
Study skills: Expert advice on preparing for examinations and
developing your digital literacy skills.
Feedback forms.
Some of these resources are available for certain courses only, but we
are expanding our provision all the time and you should check the VLE
regularly for updates.
Finally, we often use the symbol to denote the end of a proof, where we
have finished explaining why a particular result is true. This is just to make
it clear where the proof ends and the following text begins.
Chapter 1: Introduction
1.5 Examination
Important: Please note that subject guides may be used for several years.
Because of this we strongly advise you to always check both the current
Regulations, for relevant information about the examination, and the VLE
where you should be advised of any forthcoming changes. You should also
carefully check the rubric/instructions on the paper you actually sit and
follow those instructions.
A Sample examination paper is given as an appendix to this guide. There
are no optional topics in this course: you should do them all.
Please do not imagine for a moment that the questions in a real
examination will necessarily be very similar to these sample questions.
An examination is designed (by definition) to test you. You will get
examination questions that are unlike any questions in this guide. The
whole point of examining is to see whether you can apply knowledge
in both familiar and unfamiliar settings. The Examiners (nice people
though they are) have an obligation to surprise you! For this reason, it
is important that you try as many examples as possible: from the guide
and from the textbooks. This is not so that you can cover any possible
type of question the Examiners can think of! Its so that you get used to
confronting unfamiliar questions, grappling with them, and finally coming
up with the solution.
Do not panic if you cannot completely solve an examination question.
There are many marks to be awarded for using the correct approach or
method.
Remember, it is important to check the VLE for:
up-to-date information on examination and assessment arrangements
for this course
where available, past examination papers and Examiners commentaries
for this course which give advice on how each question might best be
answered.
Notes
Chapter 2
Contents
2.1
2.2
2.3
2.4
2.5
Reading
2.6
2.7
Introduction .
Vectors and
matrices . . .
Rank, range,
null
space,
and
linear
equations . . .
Learning outcomes . . . .
Sample examination questions
Answers to or
comments on
selected activities . . . . .
Sketch
answers to or
comments
on
selected
questions . . .
7
7
15
22
23
24
26
2.1
Introduction
In this chapter we first very briefly revise some basics about vectors and
matrices, which will be familiar from 05B
Mathematics
2. 2.
We
then
Mathematics
We
thenexplore
explore
some new topics, in particular the rank of a matrix, and the null space
and range of a matrix.
2.2
or a column-vector
v1
v2
. .
..
vn
w1
v1
w1 + v1
w2 v2 w2 + v2
. + . =
.
..
.. ..
.
wn
vn
wn + vn
v1
v1
v2 v2
... = ... .
vn
vn
a11 a12
a21 a22
.
..
..
.
am1
am2
...
...
..
.
a1n
a2n
.
..
.
. . . amn
Recall that if A and B are two matrices of the same size then we
define A + B to be the matrix whose elements are the sums of the
corresponding elements in A and B. Formally, the (i, j)th entry of the
matrix A + B is aij + bij where aij and bij are the (i, j)th entries of A
and B, respectively. Also, if c is a number, we define cA to be the
matrix whose elements are c times those of A; that is, cA has (i, j)th
entry caij . If A and B are matrices such that the number (say n) of
columns of A is equal to the number of rows of B, we define the
product C = AB to be the matrix whose elements are
cij = ai1 b1j + ai2 b2j + + ain bnj .
The transpose of a matrix
a11
a21
A = (aij ) =
...
a12
a22
..
.
...
...
..
.
a1n
a2n
..
.
am1
am2
...
amn
a11
a
12
AT = (aji ) =
...
a21
a22
..
.
...
...
..
.
am1
am2
..
.
a1n
a2n
...
amn
is the matrix
AT =
1
2
3
4
1 2
3 4
and B = ( 1
3 ) , then
1
, BT = 5 .
3
(A + B)T = AT + B T ,
(AB)T = B T AT .
The first two properties follow immediately from the definition. The
last one can be stated as: The transpose of the product of two
matrices is the product of the transposes in the reverse order.
or
v = (x1 , x2 , , xn )T .
v=
... = ( x1 x2 xn ) ,
xn
2.2.1
Row operations
notgoing
going to be reiterated
Recall from 05B
Mathematics
2 (for
it itisisnot
reiteratedhere
hereinin
Mathematics
2 (for
all its gory detail!) that there are three types of elementary row
operation one can perform on a matrix. These are:
1 ...
0 0 1 ...
0 0 0 1 ... ,
. . . . .
.
.
.. .. .. ..
. . .. ..
0 0 0 0 ... 0 0
where the entries denote any numbers. The 1s which have been
indicated are called leading ones.
A matrix in echelon form has the following properties:
(1) every non-zero row begins with a leading one
(2) a leading one in a lower row is further to the right
(3) zero rows are at the bottom of the matrix.
Once the augmented matrix is in echelon form, you can either solve the
resulting equations using back substitution, or continue with row
operations to further reduce the matrix to reduced echelon form.
A matrix is in reduced echelon form if, in addition to echelon form, it
has the property that
(4) every column with a leading one has zeros elsewhere.
The example above has reduced
1 0
0 0 1
0 0 0
. . .
.. .. ..
0 0
10
echelon form,
0
0
1
..
.
0
...
...
...
..
.
...
..
.
,
..
.
0 0
where the entries denote any numbers and the indicated 1s are
leading ones.
To achieve this using row operations, reduce the augmented matrix to
echelon form. Then beginning with the last row add suitable multiples
to each row above to get zeros above the leading ones.
Example: Consider the system of equations
x1 + x2 + x3
= 3
2x1 + x2 + x3
= 4
x1 x2 + 2x3
= 5.
1 1 1 3
( A|b ) = 2 1 1 4
1 1 2 5
1 1
1
3
R2 2R1 0 1 1 2
R3 R1
0 2 1
2
1 1 1 3
(1)R2 0 1 1 2
0 2 1 2
1 1 1 3
0 1 1 2
R3 + 2R2 0 0 3 6
1 1 1 3
0 1 1 2.
( 31 )R3 0 0 1 2
This matrix is in echelon form, continue to reduced echelon form,
R1 R3 1 1 0 1
R2 R3 0 1 0 0
0 0 1 2
R1 R2 1 0 0 1
0 1 0 0.
0 0 1 2
The solution can now be read directly from the matrix. This system of
equations is equivalent to
x1 = 1,
x2 = 0,
x3 = 2.
2.2.2
Determinants
The determinant of a square matrix A is a particular number
associated with A, written det A or |A|. When A is a 2 2 matrix, the
11
a
a b
det
=
c d
c
For example,
det
1
2
2
3
b
= ad bc.
d
= 1 3 2 2 = 1.
1 2 3
A = 4 1 1.
1 3 0
1 2
1 2
(2+3)
= 5.
M23 =
= 5 and C23 = (1)
M23 =
1 3
1 3
There is a simple way to associate the cofactor Cij with the entry aij of
the matrix. Locate the entry aij and cross out the row and the column
containing aij . Then evaluate the determinant of the (n 1) (n 1)
' ' or
matrix which remains. This is the minor, Mij . Then give it a +
' ' sign according to the position of aij on the following pattern:
+ + ...
+ + ....
.. ..
..
.. . .
.
. .
.
.
If A is an n n matrix,
a11 a12 . . .
a21 a22 . . .
|A| = ..
..
..
.
.
.
an1 an2 . . .
a1n
a2n def
.. = a11 C11 + a12 C12 + . . . + a1n C1n .
.
ann
def
1 1
4 1
4 1
= 1
2
+ 3
3 0
1 0
1 3
=
12
For very large matrices this method can involve too many
computations, and it is preferable to use row operations to reduce the
05B Mathematics
matrix to an upper triangular matrix as shown in Mathematics
2. 2.
However, it turns out that the cofactor expansion is much more useful
than it first appears, due to the following theorem.2 2
|A|
This allows you to choose any row or any column of a matrix to find its
determinant using a cofactor expansion.
Example: In the example above, instead of using the cofactor
expansion by row one as shown, we can choose to evaluate the
determinant of the matrix A by row three or column three, which will
involve fewer calculations since a33 = 0. To check the result |A| = 34,
we will evaluate the determinant again using column three. Remember
the correct cofactor signs,
1 2 3
4 1
1 2
7 5 2 3
1 4 3 2
2 0 0 0
2 7 5 1
A=
B=
.
23 57 1 1
1
2
6 0
11 2 0 0
2 10 14 4
2.2.3
Invertible matrices
learned
two
methodstotocalculate
calculate the inverse
In 05B
Mathematics
2 you
learned
two
methods
inverseofof
Mathematics
2 you
a 3 3 matrix either using determinants or using row operations.
Both of these methods generalise to n n matrices. Let A be an
n n matrix.
13
..
|A|
.
.
.
C1n C2n . . . Cnn
Note that the (i, j) entry of adj(A) is the cofactor Cji .
Inverse using row operations: Form the matrix (A I) by placing the
n n identity matrix next to the matrix A. If (A I) can be reduced to
(I B) using row operations, then A is invertible and A1 = B. If A
cannot be reduced to the identity matrix, then A is not invertible.
You should also recall the fact that the system of equations Ax = b
has a unique solution for any b Rn if and only if A is invertible. In
particular, the system Ax = 0 has only the solution x = 0 (known as
the trivial solution) if and only if A is invertible. We collect these
results in the following theorem.
Theorem 2.2 If A is an n n matrix, then the following statements
are equivalent (meaning if any one of these statements is true for A,
then all the statements are true).
A1 exists.
Ax = b has a unique solution for any b Rn .
Ax = 0 has only the trivial solution, x = 0.
The reduced echelon form of A is I.
|A| 6= 0.
5
A = 2
3
4
4
1
2
3 ,
0
14
See Anton.
2.3
2.3.1
1
A = 2
3
2
2
4
1 1
0 2.
1 3
1 2 1 1
1
2
1 1
2 2 0 2 0 2 2 0
3 4 1 3
0 2 2 0
1
2
1 1
1 2 1 1
0
1
1 0 0 1 1 0.
0 0 0 0
0 2 2 0
This last matrix is in echelon form and has two non-zero rows (and two
leading ones), so the matrix A has rank 2.
3
A = 1
1
1
1
2
1 3
1 1
2 1
has rank 2.
15
2.3.2
= 1
2x1 + 2x2
= 2
3x1 + 4x2 + x3
= 2.
1 2 1 1
1 2
1
1
2 2 0 2 0 2 2 0
3 4 1 2
0 2 2 1
1 2
1
1
1 2 1 1
0 1
1
0 0 1 1 0 .
0 2 2 1
0 0 0 1
Thus the original system of equations is equivalent to the system
x1 + 2x2 + x3
= 1
x2 + x3
= 0
16
= 1.
1 3 2 0 2
0 0 1 2 0
0 0 0 0 0
0 0 0 0 0
0
1
.
5
0
1 3 0 4 2 0 28
0 0 1 2 0 0 14
.
0 0 0 0 0 1
5
0 0 0 0 0 0
0
17
= 28
x3 + 2x4
= 14
x6
= 5.
x3 = 14 2x4 ,
The form of these equations tells us that we can assign any values to
x2 , x4 and x5 , and then the leading variables will be determined.
Explicitly, if we give x2 , x4 , x5 the arbitrary values s, t, u, where s, t, u
represent any real numbers, the solution is given by
x1 = 283s4t2u, x2 = s, x3 = 142t, x4 = t, x5 = u, x6 = 5.
Observe that there are infinitely many solutions, because the so-called
free unknowns x2 , x4 , x5 can take any values s, t, u R.
Generally, we can describe what happens when the echelon form has
r < n non-zero rows (0 0 . . . 0 1 . . . ). If the leading 1 is in the
kth column it is the coefficient of the unknown xk . So if the rank is r
and the leading 1s occur in columns c1 , c2 , . . . , cr then the general
solution to the system can be expressed in a form where the unknowns
xc1 , xc2 , . . . , xcr (the leading variables) are given in terms of the other
n r unknowns (the non-leading variables), and those n r unknowns
are free to take any values. In the preceding example, we have n = 6
and r = 3, and the 3 unknowns x1 , x3 , x6 can be expressed in terms of
the 6 3 = 3 free unknowns x2 , x4 , x5 .
In the case r = n, where the number of non-zero rows r in the echelon
form is equal to the number of unknowns n, there is only one solution
to the system for there is a leading one in every column since the
leading 1s move one step to the right as we go down the rows. In this
case there is a unique solution obtained from the reduced echelon form.
In fact, this can be thought of as a special case of the more general one
discussed above: since r = n there are n r = 0 free unknowns, and
the solution is therefore unique.
We can now summarise our conclusions concerning a general linear
system.
If the echelon form has a row (0 0 . . . 0 a), with a 6= 0, the
original system is inconsistent; it has no solutions.
If the echelon form has no rows of the above type it is consistent,
and the general solution involves n r free unknowns, where r is
the rank. When r < n there are infinitely many solutions, but when
r = n there are no free unknowns and so there is a unique solution.
18
2.3.3
x1
x2
x
x = 3,
x4
x5
x6
then
28 3s 4t 2u
28
3s
4t
2u
s
0 s 0 0
14 2t
14 0 2t 0
x=
=
+
+
+
.
t
0 0 t 0
u
0
0
0
u
5
5
0
0
0
That is, the general solution is
x = v + su1 + tu2 + uu3 ,
s, t, u R,
where
28
3
4
2
0
1
0
0
14
0
2
0
v=
, u1 =
, u2 =
, u3 =
.
0
0
1
0
0
0
0
1
5
0
0
0
Applying the same method generally to a consistent system of rank r
with n unknowns, we can express the general solution of a consistent
system Ax = b in the form
x = v + s1 u1 + s2 u2 + + snr unr .
Note that, if we put all the si s equal to 0, we get a solution x = v,
which means that Av = b, so v is a particular solution of the system.
Putting s1 = 1 and the remaining si s equal to zero, we get a solution
x = v + u1 , which means that A(v + u1 ) = b. Thus
b = A(v + u1 ) = Av + Au1 = b + Au1 .
Comparing the first and last expressions, we see that Au1 is the zero
vector 0. Clearly, the same equation holds for u2 , . . . , unr . So we
have proved the following.
If A is an m n matrix of rank r, the general solution of Ax = b is
the sum of:
a particular solution v of the system Ax = b and
19
+ x4 x5
7.
x1 + x2
Show that your solution can be written in the form p + su1 + tu2
where Ap = b, Au1 = 0 and Au2 = 0.
2.3.4
Null space
It is clear from what we have just seen that the general solution to a
consistent linear system involves solutions to the system Ax = 0. This
set of solutions is given a special name: the null space or kernel of a
matrix A. This null space, denoted N (A), is the set of all solutions x
to Ax = 0, where 0 is the zero vector. That is,
Definition 2.5 (Null space) For an m n matrix A, the null space of
A is the subset
N (A) = {x Rn : Ax = 0}
of Rn , where 0 = (0, 0, . . . , 0)T is the zero vector of Rm .
Note that the linear system Ax = 0 is always consistent since, for
example, 0 is a solution. It follows from the discussion above that the
general solution to Ax = 0 involves n r free unknowns, where r is
the rank of A (and that, if r = n, then there is just one solution,
namely x = 0).
Example: Referring back to the example on page 16, consider the
system of equations Ax = b given by
x1 + 2x2 + x3
= 1
2x1 + 2x2
= 2
3x1 + 4x2 + x3
= 2.
20
1 2 1
1 2
1
1 2 1
1 0 1
A = 2 2 0 0 2 2 0 1 1 0 1 1
3 4 1
0 2 2
0 0 0
0 0 0
Setting the non-leading variable x3 = t, we find that the null space of
A consists of all vectors, x,
1
t R.
x = t 1
1
The system of equations Ax = 0 has infinitely many solutions.
We now formalise the connection between the solution set of a general
consistent linear system, and the null space of the matrix determining
the system.
Theorem 2.5 Suppose that A is an m n matrix, that b Rn , and
that the system Ax = b is consistent. Suppose that x0 is any solution
of Ax = b. Then the set of all solutions of Ax = b consists precisely
of the vectors x0 + z for z N (A); that is,
{x : Ax = b} = {x0 + z : z N (A)}.
Proof To show the two sets are equal, we show that each is a subset
of the other. Suppose, first, that x is any solution of Ax = b. Because
x0 is also a solution, we have
A(x x0 ) = Ax Ax0 = b b = 0,
so the vector z = x x0 is a solution of the system Az = 0; in other
words, z N (A). But then x = x0 + (x x0 ) = x0 + z where
z N (A). This shows that
{x : Ax = b} {x0 + z : z N (A)}.
Conversely, if z N (A) then
A(x0 + z) = Ax0 + Az = b + 0 = b,
so x0 + z {x : Ax = b}. This shows that
{x0 + z : z N (A)} {x : Ax = b}.
So the two sets are equal, as required.
2.3.5
Range
The range of a matrix A is defined as follows.
Definition 2.6 (Range of a matrix) Suppose that A is an m n
matrix. Then the range of A, denoted by R(A), is the subset
R(A) = {Ax : x Rn }
of Rm . That is, the range is the set of all vectors y Rm of the form
y = Ax for some x Rn .
21
So, R(A), as the set of all such products, is the set of all linear
combinations of the columns of A. For this reason R(A) is also called
the column space of A. (More on this in the next chapter.)
1 2
Example: Suppose that A = 1 3 . Then for x = (1 , 2 )T ,
2 1
1 2
1 + 22
1
Ax = 1 3
= 1 + 32 ,
2
2 1
21 + 2
so
1 + 22
R(A) = 1 + 32 : 1 , 2 R .
21 + 2
1
2
c1 = 1 , c2 = 3
2
1
2.4
Learning outcomes
At the end of this chapter and the relevant reading, you should be able
to:
solve systems of linear equations and calculate determinants (all of
this being revision)
compute the rank of a matrix and understand how the rank of a
matrix relates to the solution set of a corresponding system of
linear equations
22
2.5
= 3
x1 x2 x3
= 1
x1 + 2x2 + 2x3
= 1.
Question 2.2 Write down the augmented matrix for the following
system of equations, and use it to solve the system by reducing the
augmented matrix to reduced echelon form. Express your answer in
vector form.
x1 + x2 + x3 + x5
= 1
5.
3 2 1 3
C = 2 1 1 0.
6 2 4 6
(a) If C is the augmented matrix of a system of equations Ax = b,
C = (A|b), find all possible solutions.
(b) If C is the coefficient matrix of a system of homogeneous
equations, Cx = 0, find all possible solutions.
Question 2.4 Find the rank of
1
2
A=
1
0
the matrix
0
1
3
3
1
1
1
2
0 2
1 3
.
2 2
2 0
1 2
A = 5 1
1 1
x
x = y
z
2
b=7
23
2.6
7 5
2 0
|A| =
23 57
11 2
2 3
5 2 3
2
0 0
= 2 57 1 1 = 2(2)
1 1
1
2 0 0
0 0
3
= 20
1
2
1 4 3 2 1 4 3
1 1
2 7 5 1 0 1 1 3
|B| =
3
=
= 6
2
6 0 0 6
3 2
1
2 8
2 10 14 4
0 2 8
0
9 16
= 6 9 16 = 6(25) = 150
=
1 1
6 6
3 1
2 = 0
0 0
A1
3
= 9
10
2
6
7
4
11 .
12
You should already know that you have the correct answer, as you
should have checked that AA1 = I !
Activity 2.5 Using row operations, we reduce the matrix as
3 1
1 3
1 1 1 1
1 1
1 1 1 1 3 1
1 3 0 4
1 2
2 1
1 2
2 1
0 3
24
follows.
1 1
4 0
3 0
1
9
6
3
16
6
1
0
0
1
1
1
1 1
1
1 0 0
1 0
0
1
1
0
1 1
1 0.
0 0
Since there are two non-zero rows in this echelon form, rank(A) = 2.
Activity 2.6 The next row operation is R2 3R3 followed by
R1 + 2R2 .
Activity 2.7 We reduce the augmented matrix to reduced echelon
form:
1 1 1 1
1 1 1 1 2
4
R2 +R1
0 0 1 2
3 R
(A|b) = 1 1 0 1 1
3 R1
1 1 2 3 4
7
0 0 1 2
1 1 1 1 0
1 1 1 1 2
4
R2 R3
R3 R2
1 R1
0 0 1 2 0
0 0 1 2 1
2R
3 0 0 0 0 1
0 0 0 0 1
2
1 1 0 1 0
1
R1 R2
0 0 1 2 0
1 .
0 0 0 0 1
2
2
1
2
4
1
3
0
1
2
x4 = t,
x3 = 12t,
x2 = s,
x1 = 1+s+t
= x
x1
1+s+t
s
x2
x3 = 1 2t
x4
t
x5
2
1
1
1
0
1
0
1 + s 0 + t 2 = p + su1 + tu2 ,
0
0
1
2
0
0
1 1 1 1 2
4
0
1 1 0 1 1
1 = 3
1 1 2 3 4
0
7
2
1
1
1
1
1
1
1 1
1 0
1 2
1 1
1 0
1 2
1
1 2
0
1
1 1 0 = 0 ,
3 4
0
0
0
1 2
0
0
1 1 2 = 0 .
3 4
1
0
0
25
2.7
1 1 0 0 1
1
0 0 1 0 2
2
0 0 0 1 0
1
x2 and x5 are non-leading variables. Set x2 = r and x5 = s.
Using the equations, solve for the other variables in terms of these.
x1
1 r + s
1
1
1
r
x2
0 1 0
r, s R.
x3 = 2 2s = 2 +r 0 +s 2 ,
x4
1
1
0
0
x5
s
0
0
1
Question 2.3 The rank of C is 3. (a) If C is the augmented matrix
of a system Ax = b then the system is inconsistent. There is no
solution (but if there had been a solution x it would have been a vector
in R3 ). (b) The general solution of Cx = 0 is
x1
t
1
x t
1
x = 2 = = t
tR
,
x3
t
1
x4
0
0
These vectors are in R4 .
Question 2.4 Using row operations, the matrix A reduces to echelon
form
1 0 1
0
2
0 1 1 1 1
A . . .
.
0 0 1 1 3
0 0 0
0
0
There are three nonzero rows (three leading ones), so the rank of A is
3.
To find N (A) we need to solve Ax = 0, which is a system of 4
equations in 5 unknowns. Call them x1 , x2 , . . . , x5 . Continuing to
reduced echelon form,
1 0 0 1 1
2
0 1 0 0
A . . .
.
0 0 1 1 3
0 0 0 0
0
The leading variables are x1 , x2 , and x3 . Set the non-leading variables
x4 = s and x5 = t. Then the solution is
x1
s + t
1
1
x2 2t
0
2
s, t R.
x3 = s 3t = s 1 + t 3 ,
x4
s
1
0
x5
t
0
1
26
So the null space consists of all vectors of the form x = sv1 + tv2 ,
where v1 and v2 are the vectors displayed above. It is a subset of R5 .
The range of A can be described as the set of all linear combinations of
the columns of A,
R(A) = {1 c1 + 2 c2 + 3 c3 + 4 c4 + 5 c5 : i R}
where
0
1
0
2
1
2
1
1
1
3
c1 = , c2 = , c3 =
, c4 = , c5 = .
2
1
3
1
2
0
3
2
2
0
This is a subset of R4 . We will find a better way to describe this set
when we look at the column space of a matrix in the next chapter.
Question 2.5 |A| = 3 9. (a) If |A| 6= 0, that is, if 6= 3, then the
system will have a unique solution. In this case, using Cramers rule,4
z = (3 3)/( 3).
To answer (b) and (c), reduce the augmented
with = 3
1
1 2 0 2
(A|b) = 5 1 3 7 0
0
1 1 1
1
1 2 0
2
0 3 1
1 0
0
0 3 1 2
Mathematics
See the 05B
Mathematics
22
guide to review this.
0
3
1
0
1
0
2
3
2
2
1 .
1
4
2
1 2 0
2
1 0
3
3
(A|b) . . . 0 1 31 31 0 1 31 31 .
0 0 0
0 0 0
0
0
The solution can now be read from the matrix. Setting the non-leading
variable z = t,
2
4 2 4
3
x
3 3t
3
y = 1 + 1 t = 1 + t 1 = p + tv,
t R.
3
3
3
3
t
0
1
z
27
28
Chapter 3
Contents
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Reading
There is no specific essential reading for this chapter. It is essential
that you do some reading, but the topics discussed in this chapter are
adequately covered in so many texts on linear algebra that it would be
artificial and unnecessarily limiting to specify precise passages from
precise texts. The list below gives examples of relevant reading. (For
full publication details, see Chapter 1.)
3.9
3.10
3.11
3.12
3.13
3.14
3.1
3.8
Introduction .
Vector spaces
Subspaces . .
Subspaces
connected
with matrices
Linear span .
Linear independence . . .
Testing linear
independence
in Rn . . . . .
Bases and dimension . . .
Finding a basis for a linear
span in Rn . .
Basis
and
dimension of
range and null
space . . . . .
Learning outcomes . . . .
Sample examination questions
Answers to or
comments on
selected activities . . . . .
Sketch
answers to or
comments
on
selected
questions . . .
29
29
32
35
36
38
40
43
48
49
52
52
54
58
Introduction
In this chapter we study the important theoretical concept of a vector
space.
3.2
Vector spaces
3.2.1
29
for example,
(x + y) = x + y,
(x) = ()x,
x + y = y + x,
and so on.
But it is not only vectors in Rn that can be added and multiplied by
scalars. There are other sets of objects for which this is possible.
Consider the set V of all functions from R to R. Then any two of these
functions can be added: given f, g V we simply define the function
f + g by
(f + g)(x) = f (x) + g(x).
Also, for any R, the function f is given by
(f )(x) = (f (x)).
These operations of addition and scalar multiplication are sometimes
said to be pointwise addition and pointwise scalar multiplication. This
might seem a bit abstract, but think about what the functions x + x2
and 2x represent: the former is the function x plus the function x2 ,
and the latter is the function x multiplied by the scalar 2. So this is
just a different way of looking at something you are already familiar
with. It turns out that V and its rules for addition and multiplication
by a scalar satisfy the same key properties as does the set of vectors in
Rn with its addition and scalar multiplication. We refer to a set with
an addition and scalar multiplication which behave appropriately as a
vector space. We now give the formal definition of a vector space.
Definition 3.1 (Vector space) A (real) vector space V is a
non-empty set equipped with an addition operation and a scalar
multiplication operation such that for all , R and all u, v, w V ,
1. u + v V
2. u + v = v + u
3. u + (v + w) = (u + v) + w
7. (u + v) = u + v
(distributive law)
8. ( + )v = v + v
(distributive law)
9. (v) = ()v
10. 1v = v.
Other properties follow from those listed in the definition. For instance,
we can see that 0x = 0 for all x, as follows:
0x = (0 + 0)x = 0x + 0x,
30
Vector spaces
3.2.2
Examples
Example: The set Rn is a vector space with the usual way of adding
and scalar multiplying vectors.
Example: The set V of functions from R to R with pointwise addition
and scalar multiplication (described earlier in this section) is a vector
space. Note that the zero vector in this space is the function that maps
every real number to 0that is, the identically-zero function.
31
V = y : x, y R .
0
Then V is a vector space with the usual addition and scalar
multiplication. To verify this, we need only check that V is closed
under addition and scalar multiplication. The associative, commutative
and distributive laws (properties 2, 3, 7, 8, 9 ,10) will hold for vectors
in V because they hold for all vectors in R3 (and all linear
combinations of vectors in V are in V ). Furthermore, if we can show
that V is closed under scalar multiplication, then for any particular
v V , 0v = 0 V and (1)v = v V . So we simply need to
check that V 6= (V is non-empty), that if u, v V then u + v V ,
and if R and v V then v V . Each of these is easy to check.
3.3
Subspaces
The last example above is informative. Arguing as we did there, if V is
a vector space and W V is non-empty and closed under scalar
multiplication and addition, then W too is a vector space (and we do
not need to verify that all the other properties hold). The formal
definition of a subspace is as follows.
Definition 3.2 (Subspace) A subspace W of a vector space V is a
non-empty subset of V that is itself a vector space (under the same
operations of addition and scalar multiplication as V ).
The discussion given justifies the following important result.
Theorem 3.1 Suppose V is a vector space. Then a non-empty subset
W of V is a subspace if and only if:
for all u, v W , u + v W (W is closed under addition), and
for all v W and R, v W (W is closed under scalar
multiplication).
32
Subspaces
x
x
S=
: y = 2x, x R
U=
: y = 2x + 1, x R .
y
y
Each vector in one of the sets is the position vector of a point on that
line. We will show that the set S is a subspace of R2 , and that the set
U is not a subspace of R2 .
If v =
0
1
, these sets can equally well be expressed as,
and p =
1
2
S = {x : x = tv, t R}
U = {x : x = p + tv, t R}.
(since s + t R).
(since s R).
0
1
1
+
=
/U
1
3
4
since 4 6= 2(1) + 1.
3. U is not closed under scalar multiplication:
0
0
0
U, 2 R but 2
=
/U
1
1
2
33
If V is a vector space, the sets V and {0} are subspaces of V . The set
{0} is not empty, it contains one vector, namely the zero vector. It is a
subspace because 0 + 0 = 0 and 0 = 0 for any R.
Given any subset S of a vector space V , how do you decide if it is a
subspace? First check that 0 S. Then using some vectors in the
subset, see if adding them and scalar multiplying them will give you
another vector in S. To prove that S is a subspace, you will need to
verify that it is closed under addition and closed under scalar
multiplication for any vectors in S, so you will need to use letters to
represent general vectors, or components of general vectors, in the set.
That is, using letters show that the sum u + v and the scalar product
u of vectors in S also satisfy the definition of a vector in S.
To prove a set S is not a subspace you only need to find one
counterexample, one or two particular vectors (use numbers) for which
the sum or the scalar product does not satisfy the definition of S.
Note that if 0 is not in the set, it cannot be a subspace.
S1 = x2 : x R
S2 = 2x : x R
0
0
34
3.3.1
3.4
3.4.1
Null space
Suppose that A is an m n matrix. Then the null space N (A), the set
of solutions to the homogeneous linear system Ax = 0, is a subspace
of Rn .
Theorem 3.3 For any m n matrix A, N (A) is a subspace of Rn .
Proof To prove this we have to verify that N (A) 6= , and that if
u, v N (A) and R, then u + v N (A) and u N (A). Since
A0 = 0, 0 N (A) and hence N (A) 6= . Suppose u, v N (A).
Then to show u + v N (A) and u N (A), we must show that
u + v and u are solutions of Ax = 0. We have
A(u + v) = Au + Av = 0 + 0 = 0
and
A(u) = (Au) = 0 = 0,
so we have shown what we needed.
35
3.4.2
Range
Recall that the range of an m n matrix is
R(A) = {Ax : x Rn }.
Theorem 3.4 For any m n matrix A, R(A) is a subspace of Rm .
Proof The set R(A) is non-empty as A0 = 0 R(A). We need to
show that if u, v R(A) then u + v R(A) and, for any R,
v R(A). So suppose u, v R(A). Then for some y1 , y2 Rn ,
u = Ay1 , v = Ay2 . We need to show that u + v = Ay for some y.
Well,
u + v = Ay1 + Ay2 = A(y1 + y2 ),
so we may take y = y1 + y2 to see that, indeed, u + v R(A). Next,
v = (Ay1 ) = A(y1 ),
so v = Ay for some y (namely y = y1 ) and hence v R(A).
3.5
Linear span
Recall that by a linear combination of vectors v1 , v2 , . . . , vk we mean a
vector of the form
v = 1 v1 + 2 v2 + + k vk .
If we add together two vectors of this form, we get another linear
combination of the vectors v1 , v2 , . . . , vk . The same is true of any
scalar multiple of v.
36
Linear span
3.5.1
S = y : 3x 2y + z = 0 .
z
Then for x S,
x
x
x
0
1
0
= 0 + y = x 0 +y 1 .
x = y =
y
2y 3x
3x
2y
3
2
z
That is, x = xv1 + yv2 where x, y can be any real numbers and v1 , v2
are the vectors given above. Since S is the linear span of two vectors,
it is a subspace of R3 . Of course, you can show directly that S is a
subspace by showing it is closed under addition and scalar
multiplication.
If d 6= 0 then the plane is not a subspace. It is an affine set, a
translation of a linear space.
S = y : ax + by + cz = d
37
x
1
0
s, t R.
x = y = s 0 + t1
2
z
3
This yields three equations in the two unknowns, s and t. Eliminating s
and t from these equations yields a single Cartesian equation between
the variables x, y, z:
x=s
= z = 3x + 2y or 3x 2y + z = 0.
y=t
z = 3s + 2t
What is the set Lin{v} of a single non-zero vector v? We have already
seen that this defines a line through the origin in R3 . (See Learning
activity 3.6 in section 3.3.)
3.5.2
3.6
Linear independence
Linear independence is a central idea in the theory of vector spaces. If
{v1 , v2 , . . . , vk } is a set of vectors in a vector space V , then the vector
equation
1 v1 + 2 v2 + + r vk = 0
always has the trivial solution, 1 = 2 = = k = 0.
We say that vectors x1 , x2 , . . . , xk in Rn are linearly dependent (LD) if
there are numbers 1 , 2 , . . . , m , not all zero, such that
1 x1 + 2 x2 + + k xk = 0.
38
Linear independence
1 = 2 = = k = 0 :
2
i1
i+1
k
1
v1
v2
vi1
vi+1
vk ,
i
i
i
i
i
39
2
1
v1 = 2 , v2 = 1 ,
5
3
in the example above are linearly independent, since one is not a scalar
multiple of the other.
3.7
x=
... .
k
Recall (learning activity, page 22) that the matrix product Ax is
exactly the linear combination 1 v1 + v2 + + k vk .
Then the question of whether or not a set of vectors in Rn is linearly
independent can be answered by looking at the solutions of the
homogeneous system Ax = 0.
Theorem 3.7 The vectors v1 , v2 , . . . , vk are linearly dependent if and
only if the linear system Ax = 0 has a solution other than x = 0,
where A is the matrix A = (v1 v2 vk ). Equivalently, the vectors
are linearly independent precisely when the only solution to the system
is x = 0.
40
v2 =
1
1
v3 =
2
5
Now, we know from our experience of solving linear systems with row
operations that the system Ax = 0 will have precisely the one solution
x = 0 if and only if we obtain from the n k matrix A an echelon
matrix in which there are k leading ones. That is, if and only if
rank(A) = k. (Think about this!) Thus, we have the following result.
Theorem 3.8 Suppose that v1 , . . . , vk Rn . Then the set
{v1 , . . . , vk } is linearly independent if and only if the matrix
(v1 v2 vk ) has rank k.
But the rank is always at most the number of rows, so we certainly need
to have k n. Also, there is a set of n linearly independent vectors in
Rn . In fact, there are infinitely many such sets, but an obvious one is
{e1 , e2 , . . . , en } ,
where ei is the vector with every entry equal to 0 except for the ith
entry, which is 1. Thus, we have the following result.
Theorem 3.9 The maximum size of a linearly independent set of
vectors in Rn is n.
So any set of more than n vectors in Rn is linearly dependent. On the
other hand, it should not be imagined that any set of n or fewer is
linearly independent: that isnt true.
Example: In R4 , which of the following sets of vectors are linearly
independent?
1
1
2
0
2
0 2 1 0 5
L1 =
, , , , ,
9
3
1
9
1
0
2
1
0
1
1
1
0 2
L2 =
, ,
9
0
2
41
1
1
2
0 2 1
L3 =
, , ,
9
3
0
2
1
1
1
2
0
0 2 1 0
L4 =
, , , .
9
3
1
0
2
1
0
Try this yourself before reading the answers.
The set L1 is linearly dependent because it consists of five vectors in
R4 . The set L2 is linearly independent because neither vector is a
scalar multiple of the other. To see that the set L3 is linearly
dependent, write the vectors as the columns of a matrix A and reduce
A to echelon form to find that the rank of A is 2. This means that
there is a non-trivial linear combination of the vectors which is equal to
0, or equivalently, that one of the vectors is a linear combination of the
other two. The last set, L4 contains the set L3 and is therefore also
linearly dependent, since it is still true that one of the vectors is a linear
combination of the others.
c2 = c02 ,
...,
cm = c0m .
42
3.8
1
2
x = Az = (x1 x2 . . . xn )
... ,
n
where we have written z as (1 , 2 , . . . , n )T . Expanding this matrix
product, we have that any x Rn can be expressed as a linear
combination
x = 1 x1 + 2 x2 + + n xn ,
as required. This linear combination is unique since the vectors are
linearly independent (or, because there is a leading one in every column
of the echelon matrix, so there are no free variables).
2
It follows from this theorem that if we have n linearly independent
vectors in Rn , then any vector in Rn can be expressed in exactly one
way as a linear combination of the n vectors. We say that the n
vectors form a basis of Rn . The formal definition of a (finite) basis for
a vector space is as follows.
Definition 3.6 ((Finite) Basis) Let V be a vector space. Then the
subset B = {v1 , v2 , . . . , vn } of V is said to be a basis for (or of) V if:
43
x
W = y : x + y 3z = 0 .
z
Solution. If x = (x, y, z)T is any vector in W , then its components
must satisfy y = x + 3z, and we can express x as
x
x
1
0
x = y = x + 3z = x 1 +z 3 = xv+zw x, z R.
z
z
0
1
This shows that the set {v, w} spans W . The set is linearly
independent. Why? Because of the positions of the zeros and ones, if
v + w = 0 then necessarily = 0 and = 0.
Example: The set
1
1
S=
,
2
1
44
1
1
1
U = 0 ,2, 2 ,
5
3
1
1
1
1
W = 0 ,2,2 .
5
3
1
Show that one of these sets is a basis of R3 and that one of these sets
spans a plane in R3 . Find a basis for this plane. Then find a Cartesian
equation for the plane.
3.8.1
Coordinates
What is the importance of a basis? If S = {v1 , v2 , . . . , vn } is a basis
of Rn , then any vector v Rn can be expressed uniquely as
v = 1 v1 + 2 v2 + + n vn . The real numbers 1 , 2 , , n are
the coordinates of v with respect to the basis, S.
Definition 3.7 (Coordinates) If S = {v1 , v2 , . . . , vn } is a basis of a
vector space V and v = 1 v1 + 2 v2 + + n vn , then the real
numbers 1 , 2 , , n are the coordinates of v with respect to the
basis, S. We use the notation
1
2
[v]S =
...
n
1
0
1
1
B=
,
and
S=
,
0
1
2
1
are each a basis of R2 . The coordinates of the vector v = (2, 5)T in
each basis are given by the coordinate vectors,
2
1
[v]B =
and
[v]S =
.
5 B
3 S
In the standard basis, the coordinates of v are precisely the
components of the vector v. In the basis S, the components of v are
given by writing
1
1
2
v = 1
+3
=
.
2
1
5
45
3.8.2
Dimensions
A fundamental result is that if a vector space V has a finite basis, then
all bases of V are of the same size. 2
Theorem 3.13 Suppose that the vector space V has a finite basis
consisting of d vectors. Then any basis of V consists of exactly d
vectors.
This enables us, finally, to define exactly what we mean by the
dimension of a vector space V .
Definition 3.8 (Dimension) The number d of vectors in a finite basis
of a vector space V is the dimension of V , and is denoted dim(V ).
The vector space V = {0} is defined to have dimension 0.
A vector space which has a finite basis is said to be finite-dimensional.
Not all vector spaces are finite-dimensional. (For example, the vector
space of real functions with pointwise addition and scalar multiplication
has no finite basis. Such a vector space is said to be infinite
dimensional.)
Example: We already know Rn has a basis of size n. (For example,
the standard basis consists of n vectors.) So Rn has dimension n
(which is reassuring, since it is often referred to as n-dimensional
Euclidean space).
If we know the dimension of a vector space V , then we know how
many vectors we need for a basis. If we have the correct number of
vectors for a basis and we know either that the vectors span V , or that
they are linearly independent, then we can conclude that both must be
true and they form a basis, as is shown in the following theorem. That
is, we do not need to show both. 3
Theorem 3.14 Let V be a finite-dimensional vector space of
dimension d. Then:
d is the largest size of a linearly independent set of vectors in V .
Furthermore, any set of d linearly independent vectors is necessarily
a basis of V ;
46
W = y : x + y 3z = 0
z
has dimension 2, because we found a basis for it consisting of two
vectors. If we choose any set of 2 linearly independent vectors in W ,
then that set will be a basis of W . For example, the vectors
v1 = (1, 2, 1)T and v2 = (3, 0, 1)T are linearly independent (why?),
so by the theorem, S = {v1 , v2 } is a basis of W .
3.8.3
47
3.9
1
2
1
3
1
1
2
0
x1 = 2 , x2 = 2 , x3 = 4 , x4 = 0 .
1
2
1
3
1
2
1
3
Write the vectors as the columns of a matrix A and then take AT
(effectively writing each column vector as a row).
1
2
T
A =
1
3
1 2
1 2
2 4
0
0
1 1
2 2
.
1 1
3 3
1
2
1
3
1
0
0
0
1
1
2
0
2
2
4
0
1 1
1
2 2
0
1 1
0
3 3
0
1
3
1
3
2
6
2
6
1 1
0 0
0 0
0 0
1
1
1
1
2
2
2
2
1
0
0
0
1
1
0
0
0
0
0
0
1
1
0
0
2
2
0
0
1 1
0 0
.
0 0
0 0
The echelon matrix at the end of this tells us that a basis for
Lin{x1 , x2 , x3 , x4 } is formed from the first two rows, transposed, of
the echelon matrix, that is,
1
0
1 1
2 , 2 .
1
0
48
3.10
49
We have already seen that a matrix A and its reduced echelon form
have the same row space, and that the non-zero rows form a basis of
this row space. So the dimension of the row space of A, RS(A), and
the dimension of the column space of A, R(A), are each equal to the
number of leading ones in an echelon form of A, that is both are equal
to rank(A).
Example: Let A be the matrix,
1
A = 2
9
The reduced echelon form of the
1
E = 0
0
1 2 1
0 1 1.
1 3 4
matrix is (verify this!)
0 21 12
1 23 12 .
0 0 0
The leading ones in this echelon matrix are in the first and second
columns, so a basis for R(A) can be obtained by taking the first and
second columns of A. (Note: columns of A, not of the echelon
matrix!) Therefore a basis for R(A) is
1
1
2, 0 .
9
1
A basis of the row space of A consists of the two non-zero rows of the
reduced matrix, or the first two rows of the original matrix,
1
0
1
2
0 1
1 0
,
, .
or
1
3
2
1
2
21
1
1
1
2
2
Note that the column space is a 2-dimensional subspace of R3 (a
plane) and the row space is a 2-dimensional subspace of R4 . The
columns of A and E satisfy the same linear dependence relations,
which can be easily read from the reduced echelon form of the matrix,
c3 =
50
3
1
c1 + c2 ,
2
2
c4 =
1
1
c1 + c2 .
2
2
1 1 2
A = 2 0 1
9 1 3
1
1
4
from the previous example. Verify the rank-nullity theorem for this
matrix.
51
3.11
Learning outcomes
At the end of this chapter and the relevant reading, you should be able
to:
explain what is meant by a vector space and a subspace
prove that a given set is a vector space, or a subspace of a given
vector space
explain what is meant by the linear span of a set of vectors
explain what is meant by linear independence and linear
dependence
determine whether a given set of vectors is linearly independent or
linearly dependent, and in the latter case, nd a non-trivial linear
combination of the vectors which equals the zero vector
explain what is meant by a basis, and by the dimension of a
nite-dimensional vector space
explain what is meant by coordinates with respect to a basis and be able
to use this definition
3.12
x1
x2
: x2 = 3x1 .
52
Question 3.3 Show that the set S1 spans R3 , but any vector v R3
can be written as a linear combination of the vectors in S1 in infinitely
many ways. Show that S2 and S3 do not span R3 .
1
0
1
2
1
1
1
S1 = 2 , 0 , 1 , 1 , S2 = 0 , 1 , 2 ,
3
1
1
0
1
3
9
2
1
S3 = 1 , 0 .
1
1
Question 3.4 Show that the following three vectors are linearly
independent:
2
3
2
1 , 4, 3 .
2
6
1
4
Express the vector v = 7 as a linear combination of these three
3
vectors.
Question 3.5 Let
2
x1 = 3 ,
5
1
x2 = 1 ,
2
a
v = b.
c
1
0
4
9
2 1 11 2
,
,
,
.
1
3
5
1
2
4
1
3
Question 3.7 Let A be any matrix. Let v1 and v2 be two non-zero
vectors and suppose that Av1 = 2v1 and Av2 = 5v2 . Prove that
{v1 , v2 } is linearly independent.
(Hint: Assume 1 v1 + 2 v2 = 0; multiply this equation through by A
to get a second equation for v1 and v2 ; then solve the two equations
simultaneously.)
Can you generalize this result?
Question 3.8 Let B be the basis {v1 , v2 , v3 }, where v1 = (0, 1, 0)T ,
v2 = (4/5, 0, 3/5)T , v3 = (3/5, 0, 4/5)T , and let
x = (1, 1/5, 7/5)T . Find the coordinates of x with respect to B.
53
Question 3.9 For each of the sets Si of vectors given below, find a
basis of the vector space Lin(Si ) and state its dimension.
1
2
1
0
2
3
S1 =
,
S2 =
,
,
,
2
3
1
0
2
3
1
2
4
1
2
1
1
2
0
4
1
S3 = 0 , 1 , 2
S4 = ,
, ,
1
1
1
1
3
9
3
2
8
2
Question 3.10 Which of the following sets are a basis for R3 ? (State
reasons for your answers.)
2
4
7
1
1
1
S1 = 2 , 1 , 1 , 2
S2 = 0 , 1
3
0
0
1
1
1
1
3
1
2
2
1
S3 = 1 , 2 , 3
S4 = 1 , 1 , 3
1
1
0
1
0
3
For any set which is a basis of R3 , find the coordinates of the vector
w = (1, 2, 1) in that basis.
Question 3.11 Write down a basis for the yz plane in R3 .
Question 3.12 Find a basis for the null space of the matrix
1 1 1 0
.
2 1 0 1
Question 3.13 Let
1
1
A=
0
1
2
3
1
2
1
0
1
5
1
2
3
13
2
2
.
4
5
1 2 1 3 0
B = 0 1 1 1 1 .
1 3 2 0 1
Find the rank of B and verify the rank-nullity theorem.
Let b = c1 + c5 , the sum of the first and last column of the matrix B.
Without solving the system, use the information you have obtained to
write down a general solution of the system of equations Bx = b.
3.13
54
and
x + x0
u + v = y + y0 V
0
x
v = y V.
0
for any t R,
x
Activity 3.7 A general vector in S1 is of the form x2 , x R, and
0
1
one particular vector, taking x = 1, is 1 . To show S1 is not a
0
subspace you need to find one counterexample, one or two particular
vectors in S1 which do not satisfy the closure property, such as
1
1
1
2
1 S1 but 1 + 1 = 2
/ S1 .
0
0
0
0
55
x
A general vector in S2 is of the form 2x , x R, and one particular
0
1
vector, taking x = 1, is 2 . S2 is a subspace. We show it is closed
0
under addition and scalar multiplication using general vectors: if
u, v S2 , R,
a+b
b
a
a, b R a+b R,
u+v = 2a + 2b = 2(a + b) S2
0
0
0
a
u = 2(a) S2
a R.
0
Activity 3.8 Any two such vectors will be of the form
v = 1 v1 + 2 v2 + + k vk
and
v0 = 10 v1 + 20 v2 + + k0 vk
and we will have
v + v0 = (1 + 10 )v1 + (2 + 20 )v2 + + (k + k0 )vk ,
which is a linear combination of the vectors v1 , v2 , , vk . Also,
v = (1 v1 +2 v2 + +k vk ) = (1 )v1 +(2 )v2 + +(k )vk ,
is a linear combination of the vectors v1 , v2 , , vk .
Activity 3.9 Suppose d = 0. Let u, v S and R. Then
0
x
x
u = y , v = y0 ,
z
z0
where ax + by + cz = 0 and ax0 + by 0 + cz 0 = 0. Consider u + v. This
equals
X
x + x0
Y = y + y0
Z
z + z0
and we want to show this belongs to S. Now, this is the case, because
aX + bY + cZ = a(x + x0 ) + b(y + y 0 ) + c(z + z 0 )
= (ax + by + cz) + (ax0 + by 0 + cz 0 ) = 0 + 0 = 0,
and similarly it can be shown that, for any R, v S. So, in this
case, S is a subspace. You can see why this argument fails when d is
not 0; for, then aX + bY + cZ will equal 2d, which will not be the
same as d. So we will not have u + v S. (Similarly, we could see
that v will not be in S if 6= 1.)
Activity 3.10 The non-trivial linear combination 0v + 10 is 0.
56
1 1
2
Activity 3.11 Let A =
. Then, using row operations,
2 1 5
it can be seen that the general solution x to Ax = 0 is
x = (r, 3r, r)T for r R. In particular, taking r = 1, and
multiplying out the equation Ax = 0, we have that
1
1
2
0
+3
=
.
2
1
5
0
Activity 3.12 The general solution to the system is
3r/2
x
x = y = r/2 ,
r
z
for r R. Taking r = 1 and multiplying out the equation Ax = 0,
we see that
1
1
2
3 0 1 2 1
+ = 0,
3
2 1
2 9
0
2
1
and hence
2
1
1
1 3 0 1 2
=
+ .
3
2 1
2 9
1
0
2
1 1 1
1 1 1
B = 0 2 2 ,
A = 0 2 2.
1 3 5
1 3 5
|A| 6= 0, so W is a basis of R3 . |B| = 0, so U is not. Because the set
U is linearly dependent, one of the vectors is a linear combination of
the other two. Also, any two vectors of U are linearly independent
since no one of them is a scalar multiple of any other. Therefore any
two vectors from the set U are a basis of Lin(U ). Since Lin(U ) is a
two-dimensional subspace of R3 , it is plane.
Using the first two vectors in U for the basis, if x U ,
x
1
1
x = y = s 0 + t 2 , s, t R.
z
1
3
57
y
y
, s=tx= x
2
2
and
3
x + y,
2
2
so we have x 2y + z = 0. This is the Cartesian equation of the plane.
z = s + 3t =
Note that a Cartesian equation could equally well have been obtained
by writing the two basis vectors and the vector x as the columns of a
matrix M and using the fact that |M | = 0 if and only if the columns of
M are linearly dependent. That is,
1 1 x
0 2 y = 2x + 4y 2z = 0.
1 3 z
Activity 3.20 A general solution of the system of equations Ax = 0 is
1
1
2
2
3
1
2
2
x = s1
1 + s2 0 = s1 u1 + s2 u2 .
0
1
The set {u1 , u2 } is a basis of the null space of A, so dim(N (A)) = 2.
From the example, rank(A) = 2. The matrix A has n = 4 columns.
rank(A) + nullity(A) = 2 + 2 = 4 = n.
Note that the basis vectors of the null space give precisely the same
linear dependence relations between the column vectors as those given
in the example. Since Au1 = 0 and Au2 = 0,
1
3
Au1 = c1 c2 +c3 = 0
2
2
3.14
and
1
1
Au2 = c1 c2 +c4 = 0.
2
2
58
x
1
0
1
1
2 + 0 + 1 + 1 = y
0
z
1
1
3
has infinitely many solutions for any vector b R3 , and therefore
spans R3 , one can solve the system Ax = b:
x
1 1 0 1
x=
b = y .
A = 2 0 1 1
z
3 1 1 0
If the row echelon form of the coefficient matrix A has three leading
ones, then there will always be infinitely many solutions: a solution
will exist since a leading one in each row means an augmented system
cannot be inconsistent and there will be infinitely many solutions since
there is one free (non-leading) variable.
Row reducing
1 1
A = 2 0
3 1
1
0 1
1 1 0
0
1 0
shown),
1
2
4
0
1
1
1
1 1
1 0 1
3
0 0
0
12
1
1
1
2
The row echelon form of the augmented matrix (A|b) has three
leading ones.
Therefore, the set S1 spans R3 , and any vector b R3 can be written
as a linear combination of the vectors in S1 in infinitely many ways.
S2 does not span R3 . Since
1 2 1
1 2 1 0
1
0 1 2 0 0
1 3 9 1
0
0 1 ) cannot be expressed as a
in S2 as the system Bx = b has no
2
1
5
1 0
1 2
2 0 0 1
10 1
0 0
1
2
0
0
0
1
59
4
2
3
2
217
18
16
7 =
1
4 +
3 .
55
55
11
3
1
6
2
Question 3.5 There are many ways of solving this problem, and there
are infinitely many possible x3 s. We can solve it by trying to find a
vector x3 such that the matrix (x1 x2 x3 ) has rank 3.
Another approach is to answer the second part of the question first,
determine what vectors v form a linearly dependent set with x1 and x2
(find the condition on a, b, c as asked) and then write down any
vector whose components do not satisfy the condition.
Write the three vectors as the columns
1 2
A = 1 3
2 5
a
b .
c
Notice that you can choose to order x1 and x2 so that the row
reduction will be easier since it makes no difference in this question.
The vectors {x1 , x2 , v} will be linearly dependent if the row echelon
form of A has a row of zeros.
1 2
a
1 2
a
1 2 a
R2 R1
R
R
3
2
0 1 b a
0 1
ba
A = 1 3 b R3
2R1
0 1 c 2a
0 0 cab
2 5 c
The vectors will be linearly dependent if and only if the components of
v satisfy
a+bc=0
So, choose any vector for x3 which does not satisfy this equation, such
T
as x3 = ( 1 0 0 ) .
Note that this condition is the equation of a plane in R3 determined by
the vectors x1 and x2 . The set {x1 , x2 , v} is linearly dependent if and
only if v is the position vector of a point in this plane.
Question 3.6 Let
1
2
A=
1
2
0
1
3
4
4
11
5
1
9
2
,
1
3
the matrix with columns equal to the given vectors. If we only needed
to show that the vectors were linearly dependent, it would suffice to
show, using row operations, that rank(A) < 4. But were asked for
more: we have to find an explicit non-trivial linear combination that
equals the zero vector. So we need to find a non-trivial solution of
60
1
0
4
9
0
2
1 11 2 0
5 3
+
= .
1
3
5
1
0
2
4
1
3
0
Question 3.7 To prove that {v1 , v2 } is linearly independent, assume
that 1 and 2 are scalars such that
()
1 v1 + 2 v2 = 0.
Then
A(1 v1 + 2 v2 )
1 Av1 + 2 Av2
1 (2v1 ) + 2 (5v2 )
21 v1 + 52 v2
= 0
= 0
= 0
= 0.
1 2
=
Question 3.9 S1 spans R since
6 0. A basis of R2 is S2 or
2 3
{e1 , e2 }. R2 has dimension 2.
2
A basis of S2 is
1
1
1
multiples of it). The set spans the line x = t
with Cartesian
1
equation y = x. This is a one-dimensional subspace of R2 .
To find a basis of S3 or S4 , write the vectors as the columns of a
matrix. You can either row reduce AT , and the non-zero rows will be a
basis, or, you can row reduce A and the columns of A corresponding to
the columns with leading ones in the echelon form will be a basis.
Using the first method for S3 (row reduce AT ),
1 0 1
1 0 1
1 0 1
2 1 3 0 1 5 0 1 5 .
1 2 9
0 2 10
0 0 0
Then a basis is given by the top two vectors. The set S3 spans a plane
(through the origin) in R3 , Lin(S3 ) has dimension 2.
61
Using the second method for S4 , write the vectors as the columns of a
matrix A and reduce A to echelon form. Columns 1, 2 and 4 have
leading ones, so the first, second and fourth vector form a basis of
Lin(S4 ), which is a 3-dimensional subspace of R4 .
Question 3.10 S1 is not a basis. No set of four vectors in R3 can be
linearly independent.
S2 is not a basis. Two vectors cannot span R3 : there must be at least
three.
S3 is not a basis. Either notice that the third vector is the sum of the
first two, or reduce AT to echelon form and show it has a row of zeros,
or show |A| = 0. The set is not linearly independent.
S4 is a basis. You can reduce AT to echelon form or you can
compute |A| = 5 6= 0, which shows that Ax = b has a unique solution
for all b R3 .
1
To find the coordinates of w = 2 in the basis S4 you need to
1
find the unique solution of
1
1
2
1
1 + 1 + 3 = 2 .
1
0
3
1
Reduce the augmented matrix to reduced echelon form to find
= 2, = 3, = 1, so that
2
[w]S4 = 3 .
1 S
4
Question 3.11 The yz plane is the set of all vectors of the form
(0, y, z)T . So the set of vectors {e2 , e3 } is a basis of the yz plane.
Question
3.12 We omit thedetails. A basis for the null space is
(1, 2, 1, 0)T , (1, 1, 0, 1)T . There are many other possible answers.
Question 3.13 A basis for the column space is
1
2
2
1 3 2
,
,
.
1
4
1
2
5
Details of the calculations are omitted.
Question
1
B = 0
1
1 2
0 1
0 0
62
2 1 3 0
1 2 1 3
0
1
1 1 1 1 0 1 1 1 1 0
3 2 0 1
0 1 1 3 1
0
1
1
0
3
1
1
0
1
1 0
12
0
2 1
1 1
0 0
0
0
1
3
2
12
12
1 0
0 1
0 0
2
1
0
1
1
0
3
1
4
0
1
2
5
1 0
2
1 0 12 .
0 1 12
The leading ones are in the first, second and fourth columns. For the
solutions of Bx = 0, set the non-leading variables x3 = s and x5 = t.
1 ,
0
5
2
0
0
1 0
1 , 0 R5 .
0
1
12
21
2
3
1
0 , 1 , 1 R3 .
1
3
0
x1
s 52 t
1
5
1
x2 s + 2 t
1
1
x3 =
s
= 1 + 0 = x1 + x2 .
1
x4
0
1
2t
x5
t
0
2
The set {x1 , x2 } R5 is a basis of the null space.
Therefore rank(B) = 3 and nullity(B) = 2.
is n = 5, so that
rank(B) + nullity(B) = 3 + 2 = 5 = n.
T
If b = c1 + c5 , then p = ( 1 0 0 0 1 ) is a solution of Bx = b.
(Why?) Combining this with the solution of the system Bx = 0, a
general solution of Bx = b is
1
1
5
0
1
1
x = 0 + 1 + 0 = p + x1 + x2 .
0
0
1
1
0
2
63
64