Arab Open University

Faculty of Computer Studies

M132: Linear Algebra

 Linear Algebra

When you take a digital photo with your

phone or transform the image in
Photoshop, when you play a video game or
watch a movie with digital effects, when
you do a web search or make a phone call,
you are using technologies that build upon
linear algebra.
 Linear Algebra
Linear algebra provides concepts that are crucial to many
areas of information technology and computing, including:
• Graphics
• Image processing
• Cryptography
• Machine learning
• Computer vision
• Optimization
• Graph algorithms
• Quantum computation
• Computational biology
• Information retrieval and web search
 Linear Algebra

Linear algebra in turn is built on two basic elements,

MATRICES and VECTORS.

2
2 1 2 0  1 0 0 
0 9 0 1  7 0  
 4
3 0 9 0 0 1  
  4
7 0 8 0 4 0
8 
4×6 matrix
5- vector

1 0 1 1 1 1 1 0
8- vector
 Course Contents

 Systems of Linear Equations

 Matrices and Matrix Operations
 Vectors, Linear Combinations and Linear
Independence
 Vector Spaces, Subspaces, Span, Basis and
Dimensions
 Linear Transformations, Null spaces and Ranges
 Eigenvalues and Eigenvectors
 Systems of Linear Equations

• A set of equations is called a system of equations.

• A linear equation in n unknowns has the form
a1 x1  a2 x2    an xn  b where the variables are of
first-degree.
• If all equations in a system are linear, the system is a
system of linear equations, or a linear system.
• The solutions must satisfy each equation in the
system.
 Systems of Linear Equations

Example: Solve the linear system

3x  2 y  11 (1)
x  y  3 (2)
Solution: y  x3 Solve (2) for y
3 x  2( x  3)  11 Substitute y = x + 3 in (1)
3 x  2 x  6  11 Solve for x

5x  5
x 1
y  1 3 Substitute x = 1 in y = x + 3
y4 Solution set: {(1, 4)}
 Systems of Linear Equations

Matrices
• Information in science and mathematics is often
organized into rows and columns to form
rectangular arrays.
• Tables of numerical data that arise from physical
observations

5 1 3
2 1 
 4
2×3
 Systems of Linear Equations

Linear Systems ~ Matrices

Solution of a linear system is obtained by performing

appropriate operations on a certain matrix:

5x  y  3 5 1 3
2 1 4 
2x  y  4 
 Systems of Linear Equations
For any system of linear equations, we have 3 possibilities

Unique solution Infinitely many solutions No solution

x  3y  4 (1) x y 3 (1) x  y  3 (1)

2 x  4 y  1 (2) 2 x  2 y  6 (2) x  y  4 (2)

y
y y
(2) (1) (2)
(1) (1)
(2)
x x x
(1, −1)

x = 1, y = −1 Let y = t, x = 3 − t, t in R None
Independent System Dependent System Inconsistent System
 Systems of Linear Equations
Consider t he system of m equations in n unknowns :
a11 x1  a12 x2    a1n xn  b1
a21 x1  a22 x2    a2 n xn  b2

am1 x1  am 2 x2    amn xn  bm

The matrix form AX  B is The augmentedmatrix is

 a11 a12  a1n   x1   b1  a11 a12  a1n b1 
a  a2 n   x2   b2  a a  a 
 21 a22   21 22 2n b2 
       
      
am1 am 2  amn   xn  bm  
 m1 m 2
a a a mn bm
A is called the coefficient matrix .
 Systems of Linear Equations
 Consider t he system of 3 equations in 4 unknowns :
x1  3 x2  x3  x4  7
x1  2 x2  2 x4  9
3 x1  x2  4 x3  7

The matrix form is AX  B,

 x1 
1 3  1 1 x   7 
where A  1 2 0  2, X    , B   9 
2

 x3 
3 1 4 0     7 
 x4 
The augmented matrix is
 1 3 1 1  7
A B   1 2 0  2 9

 3 1 4 0 7 
 Systems of Linear Equations
 Consider t he homogenous system of linear equations :
x1  3 x2  x3  x4  0
x1  2 x2  2 x4  0
3x1  x2  4 x3 0
The matrix form is AX  O,

 x1 
1 3 1 1 x  0 
where A  1 2 0  2 , X   2  and O  0
   x3   
3 1 4 0    0
 x3 
1 3 1 1 0

The augmented matrix is A O   1 2 0  2 0
 
3 1 4 0 0

 Systems of Linear Equations

x  2 y  3z  9 x  2
2x  y  z  8 y  1
3x  z 3 z  3

1 2 3 9  1 0 0 2
   
2  1 1 8 0 1 0  1
   
 3 0  1 3  0 0 1 3
 Systems of Linear Equations
 Definition: An m×n matrix A is said to be in reduced row
echelon form if it satisfies the following conditions:
• All zero rows (consisting entirely of zeros), if any, are at the
bottom.
• The first nonzero entry from the left of a nonzero row is 1,
called the leading 1 for that row.
• Each leading 1 is to the right of all leading 1’s in the rows
above it.
• Each leading 1 is to the only nonzero entry in its column.

1 0 7 0 0 1 2 0  1 0
e.g. 1 0 0 2 0
0 1 0 4
0 1 4 0  0 0 1  7 0

  0 0 0 1 0 0 0 0 0 1
0 0 1 9    
0 0 0 0 0 0 0 0 0 0
 Systems of Linear Equations
1 3 0 5 1 0 0 2
0 1 0 1 0 1 0 1
   
0 0 1 9 0 0 1 9

1 0 7 0 1 0 7 0
0 1 4 0 0 1 4 0
 
0 1 4 1 0 0 0 1
   
0 0 0 0 0 0 0 0

1 3 2 0  1 0 1 3 2 0  1 0
0 0 0 1  7 0 0 0 0 1  7 0
 
0 0 0 0 0 0 0 0 0 0 0 1
   
0 0 0 0 0 1 0 0 0 0 0 0
 Systems of Linear Equations
 Definition: The elementary row operations on an m×n matrix
A are:
• Interchanging two rows.
• Multiplying one row by a nonzero number.
• Add a multiple of one row to a different row.

2 3 5 1 1 2  1 1 2
e.g. A  1 1 2 R1  R2 2 3 510 R2 20 30 50 
     
2 0 3 2 0 3  2 0 3 

1 1 2
R3  R3  2 R1 20 30 50   B.
 
 0  2  1

The matrix B is row equivalent to the matrices A.

 Systems of Linear Equations
Elementary Row Operations (Example)
R1 : x  y  2 z  9 1 1 2 9 
2 3 
R 2 : 2 x  4 y  3z  1 
4 1

R3 : 3x  6 y  5 z  0  3 6 5 0 

• R2 = R2 – 2R1
R1 : x  y  2 z  9  1 1 2 9
 0 7  17 
R 2 : 2 y  7 z  17 
2

 3 6 5 0 
R3 : 3x  6 y  5 z  0
• R3 = R3 – 3R1
R1 : x  y  2 z  9  1 1 2 9
R 2 : 2 y  7 z  17
 0 2  7  17 
 
R 3 : 3 y  11z  27  0 3  11  27 
 Systems of Linear Equations

Find a matrix B in reduced row echelon form that is row equivalent

to the matrix

0 - 2 4 4
1 - 1 3 1
A .
0 1 - 2 - 2 
 
 3 - 4 11 0 
 Systems of Linear Equations
0 - 2 4 4  1 - 1 3 1 1 - 1 3 1 
1 - 1 3 1 0 - 2 4 4  0 - 2 4 4 
A  R1  R2   R4  3R1  
0 1 - 2 - 2  0 1 - 2 - 2  0 1 - 2 - 2 
     
3 - 4 11 0 3 - 4 11 0  0 - 1 2 - 3 

1 - 1 3 1 R1  R2 1 0 1 - 1 1 0 1 - 1
  0 1 - 2 - 2 0 1 - 2 - 2
1 0 1 - 2 - 2    R3  R4  
 R2 R3  R2
2 0 1 - 2 - 2  0 0 0 0 0 0 0 - 5
  R R    
 0 - 1 2 - 3  4 2 0 0 0 - 5 0 0 0 0

1 0 1 - 1 1 0 1 0
1 0
 1 - 2 - 2 R1  R3 0 1 - 2 0
 R3     B.
5 0 0 0 1 R2  2 R3 0 0 0 1
   
0 0 0 0 0 0 0 0
 Systems of Linear Equations

Find a matrix in reduced row echelon form that is row equivalent

to the matrix
 1 - 2 -1 2 3
 1 - 2 -1 -1 0
 .
- 1 2 1 0  1
 
2 -4 -2 2 4
 Systems of Linear Equations

 1 - 2 - 1 2 3 R2  R1 1 - 2 - 1 2 3 
 1 - 2 - 1 - 1 0 0 0 0 - 3  3 
  R3  R1  
- 1 2 1 0  1 0 0 0 2 2 
  R4  2 R1 0 0 0 - 2  2
 2 - 4 - 2 2 4

1 - 2 - 1 2 3  R1  2 R2 1 - 2 - 1 0 1
  0 0 0 1
1 0 0 0 1 1  1
 R2 R3  2 R2  
3 0 0 0 2 2  0 0 0 0 0
  R4  2 R2 0 0 0 
0 0 0 - 2  2  0 0
 Systems of Linear Equations
 Let AX = B and CX = D be two systems of linear equations each of m
equations in n unknowns. If the augmented matrices [A | B ] and [C | D ] of
these systems are row equivalent, then both linear systems have exactly the
same solutions.
 To solve the system AX = B :
• Form the augmented matrix [A | B ].
• Find the matrix [C | D ] in reduced row echelon form that is row equivalent
to the matrix [A | B ] that by using elementary row operations.
• For the matrix [C | D ], there are 3 possibilities:
o Number of leading 1’s = number of unknowns (variables), then the
system has the unique solution X = D.
o Number of leading 1’s < number of unknowns, then the system has
infinitely many solutions. Here the non-leading variables (unknowns
corresponding to columns that do not contain leading 1) end up as
parameters and the leading variables (unknowns corresponding to
columns that contain leading 1) are given in terms of these parameters.
o The system is inconsistent (0 = 1 !!!), the system has no solution.
 Systems of Linear Equations
AX = B

[A | B ]

[C | D ]

1 0 0 *  1 0 * *  1 0 * 0 
     
0 1 0 *  0 1 * *  0 1 * 0 
0 0 1 *  0 0 0 0  0 0 0 1 
Unique solution Infinitely many solutions No solution
 Systems of Linear Equations

Example 1 :

Solve the system of linear equations

x  2 y  3z  9
2x  y  z 8
3x  z  3
 Systems of Linear Equations
1. The augmented matrix of the system is
1 2 3 9
 
2  1 1 8 .
3 0  1 3
2. The reduced row echelon matrix tha t is row equivalent to
the augmented matrix is
1 0 0 2
 
 0 1 0  1.
0 0 1 3 
3. Number of leading 1' s (3)  Number of unknowns (3)
 x   2
 Unique Solution X   y    1.
 z   3 
 Systems of Linear Equations

Example 2 :
Solve the system of linear equations
x  y  2z  5w  3
2x  5 y  z  9w  3
2x  y  z  3w   11
x  3y  2z  7w  5
 Systems of Linear Equations
1. The augmented matrix of the system is
1 1 2 5 3
 
 2 5  1  9  3 .
2 1 1 3  11
 
 1  3 2 7  5 
2. The reduced row echelon matrix tha t is row equivalent to the augmented matrix is
1 0 0 2 5 
 
 0 1 0  3 2 
0 0 1  2 3
 
 0 0 0 0 0 
3. Number of leading 1' s (3)  Number of unknowns (4)  Infinitely many solutions.
Let w  t , t  R  x  2t  5, y  3t  2, z  2t  3
 x  2t  5  2t  5 
 y  3t  2   
  3t  2  
 The solutions are  , t  R . The solution set is S   t  R .
 z  2 t  3  2t  3  
    
 w  t  t  
 Systems of Linear Equations

Example 3 :
Solve the system of linear equations
x  2 y  3z  4w  5
x  3 y  5z  7w  11 .
x  z  2w  6
 Systems of Linear Equations
1. The augmented matrix of the system is

1 2 3 4 5 
 
1 3 5 7 11 .
 
1 0  1  2  6 

2. The reduced row echelon matrix tha t is row equivalent to the augmented matrix is

1 0  1  2 0
 
0 1 2 3 0 
 
 0 0 0 0 1 

3. The last equation is 0 x  0 y  0 z  0w  1, 0  1 ! ! ! !

 The sysyem has no solution (Incosistent).
 Systems of Linear Equations
 For the system of linear equations AX = B (B ≠ O),
If X1 and X2 are two solutions, then rX1 + sX2 , r + s = 1, is
also a solution.

e.g. If X1 and X2 are two solutions to the system of linear

equations AX = B (B ≠ O), then 3X1 − 2X2 and 0.25X1 + 0.75X2
are also solutions:
X 1 and X 2 are solutions  AX 1  B and AX 2  B,
A(3 X 1  2 X 2 )  3 AX 1  2 AX 2  3B  2 B  B
 3 X 1  2 X 2 is also a solution.

e.g. If X1, X2 and X3 are solutions to the system of linear

equations AX = B (B ≠ O), then 3X1 + 2X2 − 4X3 is also a
solution.
 Systems of Linear Equations
 For the homogenous system of linear equations AX = O,
• If X1 and X2 are two solutions, then rX1 + sX2 is also a
solution.

e.g. If X1 and X2 are two solutions, then 3X1 + 2X2 and 10X1
− 5X2 are also solutions.

• The homogenous system is always consistent (has solution)

which is either of following: The unique solution ( X = O,
Zero solution), called the trivial solution, or an infinitely
many solutions (including the trivial solution), called the
nontrivial solution.
 Systems of Linear Equations

Example :
Find all values of a for which the system
x  y  z  2
x  2y  z  3
x  y  ( a 2  5 )z  a
(i) has no solution,
(ii) has infinitely many solutions,
(iii) has a unique solutions.
 Systems of Linear Equations
1 1 1 2 1 1 1 2
  R2  R1  
1 2 1 3 0 1 2 1 
   
1 1 a2  5 a  R3  R1 0 0 a2  4 a  2
   
(i) The system has no solution if

a 2  4  0 and a  2  0
 a   2 and a  2  a  2.
(ii) The system has infinitely many solutions if

a 2  4  0 and a  2  0
 a   2 and a  2  a  2.
(iii) The system has a unique solution if

a 2  4  0  a   2  a  R \ { 2}.
 Systems of Linear Equations

Example :

1 1 1 1
Let A  1 0 0 1 .
 
1 2 1 0

Solve the homogeneous linear system AX  O.

 Systems of Linear Equations

1 1 1 1 0 1 0 0 1 0 
   
Let A O   1 0 0 1 0  0 1 0  1 0  .
1 2 1 0 0 0 0 1 1 0 
Number of leading 1' s is less than the number of unknowns,
We have infinitely many solutions.
Let x4  t  x1  t , x2  t , x3  t ,
 t  
  
 t 
S  , t  R
 t  
 
 t  