Página |1

Diego Fernando Gómez Páez

Facultad de Ciencias Físico-Químicas

Escuela de Ingeniería de Petróleos

Julio de 2010

LINEAR EQUATIONS SYSTEMS

1. INTRODUCTION [1]

In mathematics, a system of linear equations (or linear system) is a collection of linear equations
involving the same set of variables. For example,

Is a system of three equations in the three variables . A solution to a linear system is an

assignment of numbers to the variables such that all the equations are simultaneously satisfied. A
solution to the system above is given by

Since it makes all three equations valid.

In mathematics, the theory of linear systems is a branch of linear algebra, a subject which is
fundamental to modern mathematics. Computational algorithms for finding the solutions are an
important part of numerical linear algebra, and such methods play a prominent role
in engineering, physics, chemistry, computer science, and economics. A system of non-linear
equations can often be approximated by a linear system (see linearization), a helpful technique
when making a mathematical model or computer simulation of a relatively complex system.

A general system of m linear equations with n unknowns can be written as

 Página |2

Here are the unknowns, are the coefficients of the

system, and are the constant terms.

Often the coefficients and unknowns are real or complex numbers, but integers and rational
numbers are also seen, as are polynomials and elements of an abstract algebraic structure.

Vector equation

One extremely helpful view is that each unknown is a weight for a column vector in a linear
combination.

This allows all the language and theory of vector spaces (or more generally, modules) to be
brought to bear. For example, the collection of all possible linear combinations of the vectors on
the left-hand side is called their span, and the equations have a solution just when the right-hand
vector is within that span. If every vector within that span has exactly one expression as a linear
combination of the given left-hand vectors, then any solution is unique. In any event, the span has
a basis of linearly independent vectors that do guarantee exactly one expression; and the number
of vectors in that basis (its dimension) cannot be larger than m or n, but it can be smaller. This is
important because if we have m independent vectors a solution is guaranteed regardless of the
right-hand side, and otherwise not guaranteed.

Matrix equation

The vector equation is equivalent to a matrix equation of the form

Where A is an m×n matrix, x is a column vector with n entries, and b is a column vector
with m entries.

The number of vectors in a basis for the span is now expressed as the rank of the matrix.
 Página |3

2. SOLUTION OF LINEAR EQUATIONS SYSTEMS

2.1. DIRECT METHODS
2.1.1. Cramer’s Rule[2]

Given a system of linear equations, Cramer's Rule is a handy way to solve for just one of
the variables without having to solve the whole system of equations. They don't usually
teach Cramer's Rule this way, but this is supposed to be the point of the Rule: instead of
solving the entire system of equations, you can use Cramer's to solve for just one single
variable.

Let's use the following system of equations:

2x + y + z = 3

x– y–z=0

x + 2y + z = 0

We have the left-hand side of the system with the variables (the "coefficient matrix") and
the right-hand side with the answer values. Let D be the determinant of the coefficient
matrix of the above system, and let Dx be the determinant formed by replacing the x-
column values with the answer-column values:

System of Coefficient matrix's Answer Dx:coefficient determinant

equations determinant column with answer-column values
in x-column

2x + 1y + 1z = 3
1x – 1y – 1z = 0
1x + 2y + 1z = 0

Similarly, Dy and Dz would then be:

Evaluating each determinant, we get:

 Página |4

Cramer's Rule says that x = Dx ÷ D, y = Dy ÷ D, and z = Dz ÷ D. That is:

x = 3/3 = 1, y = –6/3 = –2, and z = 9/3 = 3

Example: Given the following system of equations, find the value of z.

2x + y + z = 1

x – y + 4z = 0

x + 2y – 2z = 3

To solve only for z, I first find the coefficient determinant.

Then I form Dz by replacing the third column of values with the answer column:

Then I form the

quotient and simplify:

z=2

The point of Cramer's Rule is that you don't have to solve the whole system to get the one
value you need.
 Página |5

2.2.2 Inverse Matrix Method [3]

DEFINITION: Assuming we have a square matrix A, which is non-singular (i.e. det(A) does
not equal zero), then there exists an n×n matrix A-1 which is called the inverse of A, such
that this property holds:

AA-1 = A-1A = I, where I is the identity matrix.

The inverse of a general n×n matrix A can be found by using the following equation.

Where the adj(A) denotes the adjoint (or adjugate) of a matrix. It can be calculated by the
following method:

Given the n×n matrix A, define B = bij to be the matrix whose coefficients are found by
taking the determinant of the (n-1) × (n-1) matrix obtained by deleting the ith row and
jth column of A. The terms of B (i.e. B = bij) are known as the cofactors of A.

Define the matrix C, where cij = (−1)i+j bij.

The transpose of C (i.e. CT) is called the adjoint of matrix A.

Lastly to find the inverse of A divide the matrix CT by the determinant of A to give its
inverse.

Method

DEFINITION: The inverse matrix method uses the inverse of a matrix to help solve a system
of equations, such like the above Ax = b. By pre-multiplying both sides of this equation by
A-1 gives:

or alternatively

So by calculating the inverse of the matrix and multiplying this by the vector b we can find
the solution to the system of equations directly. And from earlier we found that the
inverse is given by
 Página |6

From the above it is clear that the existence of a solution depends on the value of the
determinant of A. There are three cases:

1. If the det(A) does not equal zero then solutions exist using

2. If the det(A) is zero and b=0 then the solution will be not be unique or does not exist.

3. If the det(A) is zero and b=0 then the solution can be x = 0 but as with 2. is not unique or
does not exist.

2.2.3 Gauss Method [4]

Gaussian elimination is a method for solving matrix equations of the form

To perform Gaussian elimination starting with the system of equations

Compose the "augmented matrix equation"

Here, the column vector in the variables is carried along for labeling the matrix rows.
Now, perform elementary row operations to put the augmented matrix into the upper
triangular form

Solve the equation of the th row for , then substitute back into the equation of the
st row to obtain a solution for , etc., according to the formula
 Página |7

A matrix that has undergone Gaussian elimination is said to be in echelon form.

Example: Consider the matrix equation

In augmented form, this becomes

Switching the first and third rows (without switching the elements in the right-hand
column vector) gives

Subtracting 9 times the first row from the third row gives

Subtracting 4 times the first row from the second row gives

Finally, adding times the second row to the third row gives

Restoring the transformed matrix equation gives

 Página |8

Which can be solved immediately to give , back-substituting to obtain

(which actually follows trivially in this example), and then again back-substituting to
find

2.2.4 Gauss-Jordan Method [5] [6]

The Gauss-Jordan Elimination method works with the augmented matrix in order to solve
the system of equations.

The goal of the Gauss-Jordan Elimination method is to convert the matrix into this form
(four dimensional matrix is used for demonstration purposes).

1 0 0 0 | r1

0 1 0 0 | r2

0 0 1 0 | r3

0 0 0 1 | r4

r1,r2,r3,r4 represent the results of each equation (constant terms).

Once you have the matrix in this form, your problem is solved.
The only thing you have to figure out is how to convert the matrix into this form.

Requirements for a unique solution to a system of linear equations using the Gauss-
Jordan elimination method.

The requirements for a unique solution to a system of linear equations using the Gauss-
Jordan Elimination Method are that the number of unknowns must equal the number of
equations.

When the number of equations and the number of unknowns are the same, you will
obtain an augmented matrix where the number of columns is equal to the number of rows
plus 1.

For example, if you have a system of 4 linear equations in 4 unknowns, then the number of
rows must be equal to 4 and the number of columns must be equal to 5 (4 columns for the
coefficients and 1 column for the results).
 Página |9

The vertical line between the coefficients part of the matrix and the results part of the
matrix (the constant terms are in the results part of the matrix) does not count as a
column. It's there for display purposes only if you have the capability to display it.

When you create your matrix, just make sure that the number of columns is equal to the
number of rows plus 1.

Note that it is possible to get a unique solution to a system of linear equations where the
number of equations is greater than the number of unknowns.

An example of this would be 5 lines in a plane all intersecting in the same point. There is a
unique solution for the x and y variables that makes all the equations in the system true.
This type of situation, however, is not conducive to solving using the Gauss-Jordan
Elimination Method since that method requires the number of equations and the number
of unknowns to be the same.

Note also that it is not possible to get a unique solution to a system of linear equations
where the number of equations is less than the number of unknowns.

If you use the Gauss-Jordan Elimination Method, just make sure that the number of
equations is equal to the number of unknowns and the method will work just fine.

Example: Write the augmented matrix for the system of linear equations.

Use elementary row operations on the augmented matrix [A|b] to transform A into
diagonal form.
 P á g i n a | 10

By dividing the diagonal element and the right-hand-side element in each row by the
diagonal element in that row, make each diagonal element equal to one.

Hence,

2.2.5 LU Decomposition [7]

The LU-decomposition method first "decomposes" matrix A into A = L.U,

where L and U are lower triangular and upper triangular matrices, respectively. More
precisely, if A is a n×n matrix, L and U are also n×n matrices with forms like the following:

The lower triangular matrix L has zeros in all entries above its diagonal and the upper
triangular matrix U has zeros in all entries below its diagonal. If the LU-decomposition
of A = L.U is found, the original equation becomes B = (L.U).X. This equation can be
rewritten as B = L.(U.X). Since L and B are known, solving for B = L.Y gives Y = U.X. Then,
since U and Y are known, solving for X from Y = U.X yields the desired result. In this way,
the original problem of solving for X from B = A.X is decomposed into two steps:
 P á g i n a | 11

1. Solving for Y from B = L.Y

2. Solving for X from Y = U.X

Forward Substitution

How easy are these two steps? It turns out to be very easy. Consider the first step.
Expanding B = L.Y gives

It is not difficult to verify that column j of matrix B is the product of matrix A and
column j of matrix Y. Therefore, we can solve one column of Y at a time. This is shown
below:

This equation is equivalent to the following:

From the above equations, we see that y1 = b1/l11. Once we have y1 available, the second
equation yields y2 = (b2-l21y1)/l22. Now we have y1 and y2, from equation 3, we have y3 =
(b3 - (l31y1 +l32y2)/l33. Thus, we compute y1 from the first equation and substitute it into the
second to compute y2. Once y1 and y2 are available, they are substituted into the third
equation to solve for y3. Repeating this process, when we reach equation i, we will
have y1, y2, ..., yi-1 available. Then, they are substituted into equation i to solve for yi using
the following formula:
 P á g i n a | 12

Because the values of the yi's are substituted to solve for the next value of y, this process is
referred to as forward substitution. We can repeat the forward substitution process for
each column of Y and its corresponding column of B. The result is the solution Y.

Backward Substitution

After Y becomes available, we can solve for X from Y = U.X. Expanding this equation and
only considering a particular column of Y and the corresponding column of X yields the
following:

This equation is equivalent to the following:

Now, xn is immediately available from equation n, because xn = yn/un,n. Once xn is available,

plugging it into equation n-1

and solving for xn-1 yields xn-1 = (yn-1- un-1,nxn)/ un-1,n-1. Now, we have xn and xn-1. Plugging
them into equation n-2

and solving for xn-2 yields xn-2 = [yn-2- (un-2,n-1xn-1 + un-2,nxn-)]/ un-2,n-2.
 P á g i n a | 13

From xn, xn-1 and xn-2, we can solve for xn-3 from equation n-3. In general, after xn, xn-1,
..., xi+1 become available, we can solve for xi from equation i using the following relation:

Repeat this process until x1 is computed. Then, all unknown x's are available and the
system of linear equations is solved.

2.2.6 Thomas Algorithm [8]

In numerical linear algebra, the tridiagonal matrix algorithm (TDMA), also known as
the Thomas algorithm (named after Llewellyn Thomas), is a simplified form of Gaussian
elimination that can be used to solve tridiagonal systems of equations.

This case is tridiagonal matrices A - that is A has non-zero entries only on the diagonal the
super-diagonal and the sub-diagonal. Such matrices arise frequently in the study of
numerical differential equations.

Because of the structure of A we expect L to have non-zero entries only on the diagonal
and sub-diagonal, and U to have non-zero entries only on the diagonal and super-diagonal.
In fact a little thought will tell you that the diagonal on either L or U could be chosen to be
1's; we will choose U to have this form. Thus

This yields:
 P á g i n a | 14

The solution is now easy consisting of forward and backward substitution.

2.2 ITERATIVE METHODS

2.2.1 Jacobi Method [9]

The Jacobi method is easily derived by examining each of the equations in the linear
system in isolation. If in the th equation

we solve for the value of while assuming the other entries of remain fixed, we obtain

This suggests an iterative method defined by

Which is the Jacobi method. Note that the order in which the equations are examined is
irrelevant, since the Jacobi method treats them independently. For this reason, the Jacobi
method is also known as the method of simultaneous displacements, since the updates
could in principle be done simultaneously.

2.2.2 Gauss-Seidel Method [10]

 P á g i n a | 15

The Gauss-Seidel method (called Seidel's method by Jeffreys and Jeffreys 1988, p. 305) is a
technique for solving the equations of the linear system of equations one at a
time in sequence, and uses previously computed results as soon as they are available,

There are two important characteristics of the Gauss-Seidel method should be noted.
Firstly, the computations appear to be serial. Since each component of the new iterate
depends upon all previously computed components, the updates cannot be done
simultaneously as in the Jacobi method. Secondly, the new iterate depends upon the
order in which the equations are examined. If this ordering is changed, the components of
the new iterates (and not just their order) will also change.

In terms of matrices, the definition of the Gauss-Seidel method can be expressed as

where the matrices , , and represent the diagonal, strictly lower triangular,
and strictly upper triangular parts of , respectively.

The Gauss-Seidel method is applicable to strictly diagonally dominant, or symmetric

positive definite matrices .

3. References

[1] http://en.wikipedia.org/wiki/System_of_linear_equations

[2] http://www.purplemath.com/modules/cramers.htm
 P á g i n a | 16

[3] http://www.maths.surrey.ac.uk/explore/emmaspages/option1.html

[4] http://mathworld.wolfram.com/GaussianElimination.html

[5] http://www.algebra.com/algebra/homework/Linear-equations/THEO-2010.lesson

[6] http://ceee.rice.edu/Books/CS/chapter2/linear44.html

[7] http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/INT-APP/CURVE-linear-system.html

[8] http://www.math.buffalo.edu/~pitman/courses/mth437/na2/node3.html

[9] http://www.netlib.org/linalg/html_templates/node12.html#eqnjacobipointwise

[10] http://mathworld.wolfram.com/Gauss-SeidelMethod.html