Inverse and Determinant

October-10-12 1:09 PM

Given L and V, can we solve the inverse problem? Is there always a solution to this problem? Yes, no shit.

Given L and V, find u where v = L(u) Let M be the matrix associated with L, so Mu = v Can we find N such that NM=I?
So we are looking for N (inverse of M) that satisfies NM = I

Mu= v NMu = Nv

Iu = Nv U = Nv
ae + fc = 1 eb + fd = 0 ag + hc = 0 gb + hd = 1 (given) (unknown) Therefore, after manipulations:

Example: 2x2 matrix M= N=

Multiply NM=

Therefore,

N=

Therefore, we do have a solution for N if ad-bc As such, some matrices do not have an inverse. example: For projection: not unique

review projections

BUT the determinant of this matrix = 0 Therefore, there is no solution to this.

projection on x-axis if given a projection, can we identify a vector uniquely that would have that projection? No.

Determinant
Anamjit Sivia Page 1

For the matrix M = Denoted as: (not absolute value)

the quantity ad-bc is called the determinant of Matrix M.

Anamjit Sivia Page 2

Unit 11: System of Linear Equations

October-10-12 1:45 PM

Arise frequently when modelling a wide range of engineering systems e.g. electric circuits Modelling: the art of writing a mathematical description of a system being analyzed Model consisting one equation and 3 unknowns:

Continued in next section.

Anamjit Sivia Page 3

Unit 12: Reduced Normal Form (RNF)

October-17-12 1:06 PM

RECALL: We can write the solution to AX=B by starting at a single solution (particular solution) and then moving along the homogeneous solution (plural). The key is that we have been able to express each leading variable in terms of the free variables. AX = B
If A = M=

M=

A matrix is in RNF if: a. b. c. d. Example: The first non-zero entry in each row is A1. The other entries in the columns containing these leading 1's are zero. The leading 1's move to the right as we move down the rows. Any zero rows are collected at the bottom.

(augmented matrix)

infinite number of solutions

So, if M is already in RNF, it is easy to simply write down the solution to AX = B. If not, then we are going to perform operations on M to put it in reduced normal form (M'), where M' corresponds to an equivalent linear system with the same solution as M.

Elementary Operations- Gaussian Elimination

1. Interchange two equations (i.e. two rows of the augmented matrix) 2. Multiply one equation (row) by a non-zero constant. 3. Add a multiple of one equation to another equation (row).

An electrical circuit can be described in terms of a mathematical model.

Anamjit Sivia Page 4

(augmented matrix)

1. 2. 3. 4. 5. 6. 7.

Housekeeping: Multiply row 1 by -1 Subtract 1 from 2. Subtract 1 from 6 Multiply 2 by -1 Row 3 - row 2 Row 5 - row 2 And so on

sivia at 18/10/2012 12:10 PM One unique solution:

Summary:
There are three possible outcomes for a linear system: 1. Consistent Systems: a. Unique solution (occurs when every variable in RNF is a leading variable i.e. free variable) b. Infinite solutions (occurs when there is at least one free variable) 2. Inconsistent Systems: No solution (occurs when a contradiction is seen in the matrix)

Matrix Ranks
It turns out that the RNF of a matrix is a unique.
Anamjit Sivia Page 5

 It turns out that the RNF of a matrix is a unique. The number of leading 1's in the RNF of the matrix A is called the RANK of A. r = rank(A) In terms of solutions to AX = B with m equations and n variables, if the rank is r, the number of free variables is (n -r)
r = number of leading variables (n-r) = number of free variables n = total number of variables

Anamjit Sivia Page 6

System of Linear Equations

October-11-12 12:18 PM

Can you solve: 2x1 + 3x2 - 4x3 = 1 We are looking for all values

Example: x + y + 3w = 2 z - 2w = 1 two equations with four unknowns Are there values of x, y, z and w that satisfy both equations?

Declare x1 to be a basic variable and x2 and x3 as free variables 2x1 = 1-3x2 + 4x3

x and z are leading (basic) w and y are free variables. (can be freely assigned) x = 2 - y - 3w z = 1 + 2w

+ x2

+x3

x y z = w

2 0 1 0

-1 1 y[ 0 ] 0

-3 0 z 2 1

Not a unique solution.

Anamjit Sivia Page 7

Unit 13: Homogeneous Systems

October-18-12 12:50 PM

Balance the following equation:

Not correct

Converting to RNF using Gaussian elimination:

There exists one free variable (

There are an infinite number of solution to this system.

In general, homogeneous systems (AX=0): - x=0 is always a solution to homogeneous systems and is called the trivial solution. - Therefore, only two possibilities exist for homogeneous systems: Unique solution Infinite solutions (including trivial solution)

Anamjit Sivia Page 8

Unit 14: Matrix Inverses

October-24-12 1:32 PM

- Limit to square systems - AX = B as a starting point. The inverse of A will be denoted as a unique matrix.

. It will be shown that this is

- Assume that AX = B, where A is a square matrix and has a unique solution. - This means the RNF of [A|B] must be of the form [I|C] - So we are looking for a matrix (D) such that

Formal Definition of a Matrix Inverse: - D is called an inverse of A if we can find D such that AD = I and DA = I. - Can A have more than one inverse? Let's say D and E are both inverses of A Therefore, DA = I = AE (= AD = EA) D = DI = D(AE) = (DA)E = IE = E Hence, there is only one inverse to a matrix. It is denoted as

- Going back to use of elementary operations to obtain a matrix in RNF Elementary matrices: We can associate matrices with the three elementary operations. Interchange two rows:

Referred to as a permutation matrix

Multiply one row by a non zero scalar

Referred to as a diagonal matrix. Add a multiple of one row to another row

Referred to as a shear matrix. Therefore, we can conclude that there is a sequence of elementary matrices such that:

Algorithm to Find A:
Anamjit Sivia Page 9

Algorithm to Find A:

Note: this is an augmented matrix where A is a matrix and I is the identity matrix in the same dimension. For example, if A is 4x4, I is also 4x4 - Perform Gaussian Elimination on M to bring it to RNF.

Changes to: Therefore: Every elementary matrix is itself invertible and is also an elementary matrix corresponding to the inverse of the row operation that produced .

REVIEW: Every elementary matrix E is invertible and is also an elementary matrix corresponding to the inverse of the row operation that produced E. sivia at 31/10/2012 1:24 PM Two Important properties:

- Property 1: If A and B are both invertible matrices and of the same size, so is AB invertible and the inverse of We will make use of the formal definition of an inverse to prove this property. Proof: Take as a candidate inverse for

Therefore,

is the inverse of

- Property 2: If A is invertible, so is and Proof: Take A as a candidate inverse of

(from the inversion of A, then A is the inverse of Using property 2:

Using property 1:

Therefore, every invertible matrix can be expressed as a product of elementary matrices.

Anamjit Sivia Page 10

Therefore, every invertible matrix can be expressed as a product of elementary matrices. This is a significant result, don't know why.

Anamjit Sivia Page 11

Unit 16: Curve Fitting

November-07-12 1:51 PM

Curve Fitting: Take a set of points and fit a function to those points.

Interpolation: Takes a curve fit and constructs new, related data points on the curve
Extrapolation: Take a curve, extend the graph to generate new data points i.e. new data point outside the original data interval

Anamjit Sivia Page 12

Unit 17: The Least Squares Problem

November-08-12 12:34 PM

We are dealing with the class of AX = B problems where there is no solution. We begin by defining an error vector. (where Sizes: A m x n, X n x 1, B m x 1 There is no solution to this problem. Let's extend the notion of magnitude of an

or

vector to

If we choose X to minimize ||E||, this is the same X that minimizes the

Another way to write matrix multiplication

This gives us a geometric way of thinking about the consistency of AX = B, namely AX=B is consistent if and only if B can be expressed as a linear combination of the column vectors of A. (or B is in the column space of A) Let's define as the column space of A. Therefore, as varies over will vary over the entire column space The least squares problem arisen when vector B does not lie in the column space N. Solving this problem amounts to finding such that is the closest vector in W to B. We can think of this as solving for: It then follows that vector of A. is orthogonal to Therefore, is orthogonal to every column

These equations are referred to as the normal equations. Define

This is called the Least Squares Solution (Assuming that Numerical Example: We have (independent variables) and Y 1 0 1 1

exists)

is a dependent variable

0
1

1
-1

2
-1

0
0
Anamjit Sivia Page 13

1 2

-1 0

-1 0

0 1

Find a solution to AX = B where Augmented Matrix:

Therefore, no solution. But, we will not give up. We are EngScis. Why don't we try a least squares solution? Normal System:

Now:

Therefore,

i.e. Least Squares Solution

Find AX that is closest to B. Find the distance from AX to B.

AX closest to B:

B= Distance from AX to B

AX - B =

Anamjit Sivia Page 14

Anamjit Sivia Page 15

Unit 18: Numerical Integration

October-31-12 1:37 PM

This is the application section of ESC103. fucking awesome.

The Fundamental Theorem of Calculus:

This theorem sates that:

Where f(x) is a continuous function and F is a function such that This cannot be used when f(x) is not defined or cannot be found. We will use numerical integration in such cases. We will approximate:

By partitioning the interval [a,b] into n evenly spaced subintervals. 1 1 1 1 1

Step size 3 RECTANGULAR APPROXIMATIONS:

Left hand limit, Right hand limit, average TRAPEZOIDAL APPROXIMATION:

Anamjit Sivia Page 16

Unit 19: Simpson's Rule

November-01-12 12:51 PM

Another approach rather than using simple rectangles to approximate area: Consider fitting a polynomial P(x) through the discrete values such that P(x) is approximately f(x) Where We can use the 2-point Newton Cotes rule to fit a polynomial of degree 1 (line) but this is just a trapezoid. Now, lets use the 3- point Newton Cotes rule to fit a polynomial of degree 2 to each triplet of points:

Find [a,b,c] in P(x) = Such that:

Because 'area' is unaffected by horizontal position, we will use instead solve the simpler problem

This is a linear system of equations. AX = B

Therefore, and so on B=

Anamjit Sivia Page 17

Therefore, the estimate of the total area:

Where n is an even number: As such: This is known as Simpson's Rule

Anamjit Sivia Page 18

Unit 20: Errors in Numerical Integration

November-07-12 1:07 PM

Errors pervade all areas of Science. Therefore, it is important to understand, quantify and control them.

Derivation of Error Bounds

Recall, that if we use Simpson's rule to get a quadratic for three collinear points parallel to the x-axis, b and c would be zero. Similarly, if it is any straight line, we would have c=0. We use the Taylor Series expansion to derive these. Trapezoid Rule:

The slope of the function does not affect the function much; rather, it is the second derivative that gives us an estimate of the error we can expect. (e.g. slope may change faster and yield a greater error in a given function) Example: Given:

Evaluate:

Since 1

then

Therefore, K = 2. Take n =5. Therefore,

This is an estimate on the upper bound of the error. In order to adjust this, we can increase n. Therefore, will get smaller. Actual Error: Calculate

Anamjit Sivia Page 19

Simpson's Rule:

The order of the integration method is the first integer m such that the method does not compute exactly. For example, in the Trapezoid Rule: Second Order Simpson's Rule: Fourth Order

Curve Fitting
Sometimes we need to fit data where the data points do not follow a linear/ quadratic/ perfect solution

Use gaussian elimination to obtain a =2, b = -3, c = 1 What if we had tried to fit a straight line rather than a quadratic?

We obtain no solution. Through gaussian elimination.

Transpose If A is an m x n matrix, the transpose ( is the n x m matrix, where the rows of a transpose are the columns of A written in the same order. Example: and

Properties: 1. 2. 3. 4. 5. If A is invertible, so is its transpose and Proof of property 5: exists by assumption. Similarly, for the other case for non-commutative multiplication.

Anamjit Sivia Page 20

Unit 21: Numerically Solving Differential Equations

November-15-12 12:24 PM

We haven't yet done any fucking differential equations but we still do this shit. Circuit Example: (Ohm's Law) A change in resistance represents an instant change in voltage. This is a static system. Spring Example: (Newton's Law) A mass is attached to a spring, on a frictionless surface. The spring does not instantaneously change its position when it is stretched or compressed; it oscillates.. This is a dynamic system. In terms of derivatives:

We are trying to solve for

, a function.

Initial Value Problems

Example: A system is described by Find . Propose that where A is a constant. (t) only becomes unique with an initial value. Say, Therefore,

What if ? In this case, numerical methods may have to be used as it is not intuitive to guess a function that fulfills the conditions.

Euler's Method
We are going to make use of the fact the derivative is always known. Start at Use of slope at this point

. Lets take a step,

Algorithm (Euler's Method)

Where

gives appropriate values to

S is our slope estimate(L- left and R- right). Consider

Anamjit Sivia Page 21

Unit 22: Euler's Method

November-21-12 1:07 PM

The Euler's Method is a numerical approach to solving first order IVP's that makes use of derivative information. Given: Start at Algorithm: ----> slope Where are approximate values of and move forward in time by using the 'slope' to estimate the change in y.

Sources of Error: 1. The formula for going from to , is not exact. 2. The information going into the formula is not exact. How may we improve on the formula? Consider : slope estimate at : slope estimate at

Q: What do we use for A: Replace by the estimate obtained using Euler's Method

Summary of Algorithm:

Anamjit Sivia Page 22

Unit 23: Higher Order Systems

November-21-12 1:28 PM

(a second order system)

Introducing vector notation: Therefore, F transforms to

Example: Given that Solve for . is a solution to this problem since

and

We will solve for both y and y'.

The system equation enters here

Euler's Method:

OR

These types of equations are called difference equations. These equations allow us to solve for y at different times. For example, if we choose

Improved Euler's Method:

Choose

Anamjit Sivia Page 23

Recap: Numerical solutions to differential equations (Initial Value Problems) - Euler's Method, improved Euler's method - In a class of predictor-corrector methods (Runge-Kutta) A general approach to representing higher order systems. For example, consider two springs attached together.

Let be the compression of the spring with Equations of motion describing this system:

and similarly

Let's introduce two new variables:

This representation is referred to as the STATE-SPACE DESCRIPTION.

Anamjit Sivia Page 24

Unit 24: Boundary Value Problems (BVP)

November-22-12 12:28 PM

Suppose we want to solve an equation of the following form: on the interval [0,1], with the boundary conditions

This is distinct from the previous chapter as we are given an endpoint, and two points on y but nothing about the derivative of y. Looking at the beam example:

x=0 The function Represents the vertical displacement of the beam

X=L

Young's Modulus and I = the second moment of area = linear load density If the beam is hinged at 0 and free at x = L, the Boundary Conditions are:

As with numerical integration, we are going to partition the interval into 1 A= [a,b] We will simultaneously estimate the solution at the grid points where 1 1 1 1

evenly spaced sub-intervals.

are approximate values of

We will approximate the derivatives in the differential equation using finite differences keeping in mind that:

Different differences may be used: Forward difference denotes the 'forward difference' Backward difference

Central Difference:

Second derivatives can be difined:

Anamjit Sivia Page 25

for

and

for

This is now an algebraic equation in terms of the unknown values of y. This can be further simplified:

This equation can be easily solved. LOL Next, approximate the derivatives using finite differences. For this example:

Next, substitute finite differences into the differential equation:

Let's choose n = 5 We want to now simultaneously estimate Substitute at the grid points using the given boundary conditions.

Now, let's write this equation at each interior grid point.

Keep in mind boundary conditions.

Anamjit Sivia Page 26

Use Gaussian elimination to solve this system.

For this problem, there is an exact solution. Actual Values:

We can change the grid point (n) to increase the accuracy which increases the accuracy of finite differences. These results are approximate to two decimal places. However, doing so would involve solving a larger matrix.

Anamjit Sivia Page 27