ORDINARY DIFFERENTIAL EQUATIONS

ENGR 351
Numerical Methods for Engineers
Southern Illinois University Carbondale
College of Engineering
Dr. L.R. Chevalier
Dr. B.A. DeVantier

Ordinary Differential
Equationswhere to use them
The dissolution (solubilization) of a contaminant into
groundwater is governed by the equation:



where k
l
is a lumped mass transfer coefficient and C
s

is the maximum solubility of the contaminant into the
water (a constant). Given C(0)=2 mg/L, C
s
= 500
mg/L and k
l
= 0.1 day
-1
, estimate C(0.5) and C(1.0)
using a numerical method for ODEs.

( ) C C k
dt
dC
s l
=
A mass balance for a chemical in a completely mixed
reactor can be written as:



where V is the volume (10 m
3
), c is concentration (g/m
3
), F
is the feed rate (200 g/min), Q is the flow rate (1 m
3
/min),
and k is reaction rate (0.1 m
3
/g/min). If c(0)=0, solve the
ODE for c(0.5) and c(1.0)
2
kVc Qc F
dt
dc
V =
Ordinary Differential
Equationswhere to use them
Before coming to an exam Friday afternoon, Mr.
Bringer forgot to place 24 cans of a refreshing
beverage in the refrigerator. His guest are arriving in
5 minutes. So, of course he puts the beverage in the
refrigerator immediately. The cans are initially at
75, and the refrigerator is at a constant temperature
of 40.

Ordinary Differential
Equationswhere to use them
The rate of cooling is proportional to the difference in
the temperature between the beverage and the
surrounding air, as expressed by the following equation
with k = 0.1/min.



Use a numerical method to determine the temperature
of the beverage after 5 minutes and 10 minutes.

( )
air
T T k
dt
dT
=
Ordinary Differential
Equationswhere to use them
Ordinary Differential Equations
A differential equation defines a relationship
between an unknown function and one or
more of its derivatives
Physical problems using differential equations
electrical circuits
heat transfer
motion
Ordinary Differential Equations
The derivatives are of the dependent
variable with respect to the
independent variable
First order differential equation with y
as the dependent variable and x as the
independent variable would be:
dy/dx = f(x,y)
Ordinary Differential Equations
A second order differential equation
would have the form:
d y
dx
f x y
dy
dx
2
2
=
|
\

|
.
| , ,
}

does not necessarily have to include
all of these variables
Ordinary Differential Equations
An ordinary differential equation is one
with a single independent variable.
Thus, the previous two equations are
ordinary differential equations
The following is not:
( )
dy
dx
f x x y
1
1 2
= , ,
Ordinary Differential Equations
The analytical solution of ordinary
differential equation as well as partial
differential equations is called the
closed form solution
This solution requires that the constants
of integration be evaluated using
prescribed values of the independent
variable(s).
Ordinary Differential Equations
An ordinary differential equation of order n
requires that n conditions be specified.
Boundary conditions
Initial conditions
Ordinary Differential Equations
An ordinary differential equation of order n
requires that n conditions be specified.
Boundary conditions
Initial conditions
consider this beam where the
deflection is zero at the boundaries
x= 0 and x = L
These are boundary conditions
consider this beam where the
deflection is zero at the boundaries
x= 0 and x = L
These are boundary conditions
a
y
o
P
In some cases, the specific behavior of a system(s)
is known at a particular time. Consider how the deflection
of a beam at x = a is shown at time t =0 to be equal to y
o
.
Being interested in the response for t > 0, this is called
the initial condition.
Ordinary Differential Equations
At best, only a few differential
equations can be solved analytically in a
closed form.
Solutions of most practical engineering
problems involving differential
equations require the use of numerical
methods.
Scope of Lectures on ODE
One Step Methods
Eulers Method
Heuns Method
Improved Polygon
Runge Kutta
Systems of ODE
Adaptive step size control
Boundary value problems
Case studies
Scope of Lectures on ODE
Specific Study Objectives
Understand the visual representation of
Eulers, Heuns and the improved polygon
methods.
Understand the difference between local and
global truncation errors
Know the general form of the Runge-Kutta
methods.
Understand the derivation of the second-
order RK method and how it relates to the
Taylor series expansion.

Specific Study Objectives
Realize that there are an infinite number of
possible versions for second- and higher-
order RK methods
Know how to apply any of the RK methods to
systems of equations
Understand the difference between initial
value and boundary value problems
Review of Analytical Solution
dy
dx
x
dy x dx
y
x
C
=
=
= +
} }
4
4
4
3
2
2
3
At this point lets consider
initial conditions.

y(0)=1
and
y(0)=2
( )
( )
( )
( )
y
x
C
for y
C
then C
for y
C
and C
= +
=
= +
=
=
= +
=
4
3
0 1
1
4 0
3
1
0 2
2
4 0
3
2
3
3
3
What we see are different
values of C for the two
different initial conditions.

The resulting equations
are:


y
x
y
x
= +
= +
4
3
1
4
3
2
3
3
0
4
8
1 2
1 6
0 0. 5 1 1 . 5 2 2. 5
x
y
y (0)=1
y (0)=2
y (0)=3
y (0)=4
One Step Methods
Focus is on solving ODE in the form
( )
dy
dx
f x y
y y h
i i
=
= +
+
,
1
|
y
x
slope = |
y
i
y
i+1
h
This is the same as saying:
new value = old value + slope x step size
Eulers Method
The first derivative provides a direct
estimate of the slope at x
i
The equation is applied iteratively, or
one step at a time, over small distance
in order to reduce the error
Hence this is often referred to as Eulers
One-Step Method
Example
2
4x
dx
dy
=
For the initial condition y(1)=1, determine y
for h = 0.1 analytically and using Eulers
method given:
Error Analysis of Eulers
Method
Truncation error - caused by the nature of
the techniques employed to approximate
values of y
local truncation error (from Taylor Series)
propagated truncation error
sum of the two = global truncation error
Round off error - caused by the limited
number of significant digits that can be
retained by a computer or calculator
....end of example
Example
0
2
4
6
8
1 0
1 2
0 0. 5 1 1 . 5 2 2. 5
x
y
Analy t ical
Solut ion
Numer ical
Solut ion
Higher Order Taylor Series
Methods
This is simple enough to implement with
polynomials
Not so trivial with more complicated ODE
In particular, ODE that are functions of both
dependent and independent variables require
chain-rule differentiation
Alternative one-step methods are needed
( )
( )
y y f x y h
f x y
h
i i i i
i i
+
= + +
1
2
2
,
' ,
Modification of Eulers
Methods
A fundamental error in Eulers method
is that the derivative at the beginning of
the interval is assumed to apply across
the entire interval
Two simple modifications will be
demonstrated
These modification actually belong to a
larger class of solution techniques called
Runge-Kutta which we will explore
later.
Heuns Method
Determine the derivative for the interval
the initial point
end point
Use the average to obtain an improved
estimate of the slope for the entire
interval
y
x
i
x
i+1
Use this average slope
to predict y
i+1
( ) ( )
h
y x f y x f
y y
i i i i
i i
2
, ,
1 1
1
+ +
+
+
+ =
{

y
x
i
x
i+1
y
x
x
i
x
i+1
( ) ( )
h
y x f y x f
y y
i i i i
i i
2
, ,
1 1
1
+ +
+
+
+ =
y
x
x
i
x
i+1
( ) ( )
h
y x f y x f
y y
i i i i
i i
2
, ,
1 1
1
+ +
+
+
+ =
h y y
i i
| + =
+1
Improved Polygon Method
Another modification of Eulers Method
Uses Eulers to predict a value of y at
the midpoint of the interval
This predicted value is used to estimate
the slope at the midpoint
We then assume that this slope represents a
valid approximation of the average slope for
the entire interval
Use this slope to extrapolate linearly from x
i

to x
i+1
using Eulers algorithm



Improved Polygon Method
Both Heuns and the Improved Polygon
Method have been introduced graphically.
However, the algorithms used are not as
straight forward as they can be.

Lets review the Runge-Kutta Methods.
Choices in values of variable will give us
these methods and more. It is recommend
that you use this algorithm on your homework
and/or programming assignments.
Runge-Kutta Methods
Runge-Kutta Methods
RK methods achieve the accuracy of a Taylor
series approach without requiring the
calculation of a higher derivative
Many variations exist but all can be cast in
the generalized form:
( )
y y x y h h
i i i i +
= +
1
| , ,
{

| is called the incremental function
|, Incremental Function

can be interpreted as a representative
slope over the interval
( )
( )
( )
( )
| = + + +
=
= + +
= + + +
= + + + + +

a k a k a k
where the a s are constant and the k s are
k f x y
k f x p h y q k h
k f x p h y q k h q k h
k f x p h y q k h q k h q k h
n n
i i
i i
i i
n i n i n n n n n
1 1 2 2
1
2 1 11 1
3 2 21 1 22 2
1 1 1 1 2 2 1 1 1

' ' :
,
,
,
,
, , ,
( )
( )
( )
( )
| = + + +
=
= + +
= + + +
= + + + + +

a k a k a k
where the a s are constant and the k s are
k f x y
k f x p h y q k h
k f x p h y q k h q k h
k f x p h y q k h q k h q k h
n n
i i
i i
i i
n i n i n n n n n
1 1 2 2
1
2 1 11 1
3 2 21 1 22 2
1 1 1 1 2 2 1 1 1

' ' :
,
,
,
,
, , ,
NOTE:
ks are recurrence relationships,
that is k
1
appears in the equation for k
2

which appears in the equation for k
3
This recurrence makes RK methods efficient for
computer calculations
Second Order RK Methods
( )
( )
( )
y y a k a k h
where
k f x y
k f x p h y q k h
i i
i i
i i
+
= + +
=
= + +
1 1 1 2 2
1
2 1 11 1
,
,
( )
( )
( )
( )
| = + + +
=
= + +
= + + +
= + + + + +

a k a k a k
where the a s are constant and the k s are
k f x y
k f x p h y q k h
k f x p h y q k h q k h
k f x p h y q k h q k h q k h
n n
i i
i i
i i
n i n i n n n n n
1 1 2 2
1
2 1 11 1
3 2 21 1 22 2
1 1 1 1 2 2 1 1 1

' ' :
,
,
,
,
, , ,
Second Order RK Methods
We have to determine values for the
constants a
1
, a
2
, p
1
and q
11
To do this consider the Taylor series in terms
of y
i+1
and f(x
i
,y
i
)
( )
( ) ( )
y y a k a k h
y y f x y h f x y
h
i i
i i i i i i
+
+
= + +
= + +
1 1 1 2 2
1
2
2
, ' ,
( )
( )
f x y
f
x
f
y
dy
dx
substituting
y y f x y h
f
x
f
y
dy
dx
h
i i
i i i i
' ,
,
= +
= + + +
|
\

|
.
|
+
c
c
c
c
c
c
c
c
1
2
2
Now, f(x
i
, y
i
) must be determined by the
chain rule for differentiation
The basic strategy underlying Runge-Kutta methods
is to use algebraic manipulations to solve for values
of a
1
, a
2
, p
1
and q
11

( )
( )
y y a k a k h
y y f x y h
f
x
f
y
dy
dx
h
i i
i i i i
+
+
= + +
= + + +
|
\

|
.
|
1 1 1 2 2
1
2
2
,
c
c
c
c
By setting these two equations equal to each other and
recalling:
( )
( )
k f x y
k f x p h y q k h
i i
i i
1
2 1 11 1
=
= + +
,
,
we derive three equations to evaluate the four unknown
constants
a a
a p
a q
1 2
2 1
2 11
1
1
2
1
2
+ =
=
=
Because we have three equations with four unknowns,
we must assume a value of one of the unknowns.

Suppose we specify a value for a
2
.

What would the equations be?
a a
p q
a
1 2
1 11
2
1
1
2
=
= =
Because we can choose an infinite number of values
for a
2
there are an infinite number of second order
RK methods.

Every solution would yield exactly the same result
if the solution to the ODE were quadratic, linear or a
constant.

Lets review three of the most commonly used and
preferred versions.
( )
( )
( )
y y a k a k h
where
k f x y
k f x p h y q k h
a a
a p
a q
i i
i i
i i
+
= + +
=
= + +
+ =
=
=
1 1 1 2 2
1
2 1 11 1
1 2
2 1
2 11
1
1
2
1
2
,
,
Consider the following:

Case 1: a
2
= 1/2

Case 2: a
2
= 1

These two methods
have been previously
studied.

What are they?
( )
( )
a a
a p
a q
p q
a
y y k k h
where
k f x y
k f x h y k h
i i
i i
i i
1 2
2 1
2 11
1 11
2
1 1 2
1
2 1
1 1 1 2 1 2
1
2
1
2
1
2
1
1
2
1
2
= = =
=
=
= = =
= + +
|
\

|
.
|
=
= + +
+
/ /
,
,
Case 1: a
2
= 1/2

This is Heuns Method with
a single corrector.

Note that k
1
is the slope at
the beginning of the interval
and k
2
is the slope at the
end of the interval.
( )
( )
( )
y y a k a k h
where
k f x y
k f x p h y q k h
i i
i i
i i
+
= + +
=
= + +
1 1 1 2 2
1
2 1 11 1
,
,
( )
a a
a p
a q
p q
a
y y k h
where
k f x y
k f x h y k h
i i
i i
i i
1 2
2 1
2 11
1 11
2
1 2
1
2 1
1 1 1 0
1
2
1
2
1
2
1
2
1
2
1
2
= = =
=
=
= = =
= +
=
= + +
|
\

|
.
|
+
,
,
( )
( )
( )
y y a k a k h
where
k f x y
k f x p h y q k h
i i
i i
i i
+
= + +
=
= + +
1 1 1 2 2
1
2 1 11 1
,
,
Case 2: a
2
= 1

This is the Improved
Polygon Method.
Ralstons Method

Ralston (1962) and Ralston and Rabinowitiz (1978)
determined that choosing a
2
= 2/3 provides a minimum
bound on the truncation error for the second order RK
algorithms.

This results in a
1
= 1/3 and p
1
= q
11
= 3/4
( )
y y k k h
where
k f x y
k f x h y k h
i i
i i
i i
+
= + +
|
\

|
.
|
=
= + +
|
\

|
.
|
1 1 2
1
2 1
1
3
2
3
3
4
3
4
,
,
Example
( ) ( )
1 . 0
1 1 . . 1 1 . .
4
2
=
= = =
=
h size step
y e i x at y C I
y x
dx
dy
Evaluate the following
ODE using Heuns
Methods
Third Order Runge-Kutta Methods
Derivation is similar to the one for the second-order
Results in six equations and eight unknowns.
One common version results in the following
( )
( )
( )
y y k k k h
where
k f x y
k f x h y k h
k f x h y hk hk
i i
i i
i i
i i
+
= + + +

(
=
= + +
|
\

|
.
|
= + +
1 1 2 3
1
2 1
3 1 2
1
6
4
1
2
1
2
2
,
,
,
Note the third term
NOTE: if the derivative is a function of x only, this reduces to Simpsons 1/3 Rule
Fourth Order Runge Kutta
The most popular
The following is sometimes called the
classical fourth-order RK method
( )
( )
( )
y y k k k k h
where
k f x y
k f x h y k h
k f x h y hk
k f x h y hk
i i
i i
i i
i i
i i
+
= + + + +

(
=
= + +
|
\

|
.
|
= + +
|
\

|
.
|
= + +
1 1 2 3 4
1
2 1
3 2
4 3
1
6
2 2
1
2
1
2
1
2
1
2
,
,
,
,
Note that for ODE that are a function of x alone that
this is also the equivalent of Simpsons 1/3 Rule
( )
( )
( )
3 4
2 3
1 2
1
4 3 2 1 1
,
2
1
,
2
1
2
1
,
2
1
,
2 2
6
1
hk y h x f k
hk y h x f k
h k y h x f k
y x f k
where
h k k k k y y
i i
i i
i i
i i
i i
+ + =
|
.
|

\
|
+ + =
|
.
|

\
|
+ + =
=
(

+ + + + =
+
Example
Use 4th Order RK to solve the following differential equation:
( )
dy
dx
xy
x
I C y =
+
=
1
1 1
2
. .
using an interval of h = 0.1
Higher Order RK Methods
When more accurate results are
required, Buchers (1964) fifth order RK
method is recommended
There is a similarity to Booles Rule
The gain in accuracy is offset by added
computational effort and complexity
Systems of Equations
Many practical problems in engineering and
science require the solution of a system of
simultaneous differential equations
( )
( )
( )
dy
dx
f x y y y
dy
dx
f x y y y
dy
dx
f x y y y
n
n
n
n n
1
1 1 2
2
2 1 2
1 2
=
=
=
, , , ,
, , , ,
, , , ,

Solution requires n initial conditions

All the methods for single equations can be
used
The procedure involves applying the one-step
technique for every equation at each step
before proceeding to the next step
( )
( )
( )
dy
dx
f x y y y
dy
dx
f x y y y
dy
dx
f x y y y
n
n
n
n n
1
1 1 2
2
2 1 2
1 2
=
=
=
, , , ,
, , , ,
, , , ,

Boundary Value Problems

Recall that the solution to an nth order ODE
requires n conditions
If all the conditions are specified at the same
value of the independent variable, then we
are dealing with an initial value problem
Problems so far have been devoted to this
type of problem
Boundary Value Problems
In contrast, we may also have conditions a
different value of the independent variable.
These are often specified at the extreme
point or boundaries of as system and
customarily referred to as boundary value
problems
To approaches to the solution
shooting method
finite difference approach
General Methods for Boundary
Value Problems
The conservation of heat can be used to develop a heat
balance for a long, thin rod. If the rod is not insulated
along its length and the system is at steady state. The
equation that results is:
( )
d T
dx
h T T
a
2
2
0 + = '
T
1
T
2
T
a
T
a
T
1
T
2
T
a
T
a
( )
d T
dx
h T T
a
2
2
0 + = '
Clearly this second order
ODE needs 2 conditions.
This can be satisfied by
knowing the temperature
at the boundaries,
i.e. T
1
and T
2
T(0) = T
1


T(L) = T
2
( )
d T
dx
h T T
a
2
2
0 + = '
T(0) = T
1


T(L) = T
2
Use these conditions to solve
the equation analytically.

For a 10 m rod with
T
a
= 20
T(0) = 40
T(10) = 200
h = 0.01


T e e
x x
= + 7345 5345 20
0 1 0 1
. .
. .
Now that we have an analytical solution, lets evaluate our
two proposed numerical methods.
Shooting Method
( )
( )
d T
dx
h T T
dT
dx
z
dz
dx
h T T
a
a
2
2
0 + =
=
=
'
'
Given:
We need an initial value
of z.

For the shooting method, guess
an initial value.

Guessing z(0) = 10
( )
dz
dx
h T T
a
= '
Using a fourth-order RK method with a step size
of 2, T(10) = 168.38

This differs from the BC T(10) = 200

Making another guess, z(0) = 20

T(10) = 285.90

Because the original ODE is linear, the estimates
of z(0) are linearly related.
Guessing z(0) = 10
Using a linear interpolation formula between the values
of z(0), determine a new value of z(0)

Recall:

first estimate z(0) = 10 T(20) = 168.38
second estimate z(0)=20 T(20) = 285.90

What is z(0) that would give us T(20)=200?

150
200
250
300
0 5 10 15 20 25
z(0)
T
(
2
0
)
( ) ( ) z 0 10
20 10
28590 168 38
200 168 38 12 69 = +

=
. .
. .
150
200
250
300
0 5 10 15 20 25
z(0)
T
(
2
0
)
( )
( )
d T
dx
h T T
dT
dx
z
dz
dx
h T T
a
a
2
2
0 + =
=
=
'
'
We can now use
this to solve the first
order ODE
For nonlinear boundary value problems, linear
interpolation will not necessarily result in an accurate
estimation. One alternative is to apply three
applications of the shooting method and use quadratic
interpolation..
0
5 0
1 00
1 5 0
200
25 0
0 5 1 0
dist ance (m)
T
Analy t ical
Solut ion
Shoot ing
Met hod
Finite Difference Methods
The finite divided difference approximation for
the 2nd derivative can be substituted into the
governing equation.
( )
( )
d T
dx
T T T
x
d T
dx
h T T
T T T
x
h T T
i i i
a
i i i
a i
2
2
1 1
2
2
2
1 1
2
2
0
2
0
~
+
+ =
+
+ =
+
+
A
A
'
'
( )
( )
T T T
x
h T T
T h x T T h x T
i i i
a i
i i i a
+
+
+
+ =
+ + =
1 1
2
1
2
1
2
2
0
2
A
A A
'
' '
Collect terms
We can now apply this equation to each interior node
on the rod.

Divide the rod into a grid, and consider a node to be
at each division. i.e.. A x = 2m
T
1
T
2
L = 10 m
A x = 2 m
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

L = 10 m
A x = 2 m
Consider the previous problem:
L = 10 m
T
a
= 20
T(0) = 40
T(10) = 200
h = 0.01
We need to solve for the
temperature at the interior
nodes (4 unknowns).
Apply the governing
equation at these nodes (4
equations).
What is the matrix?
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
Notice the labeling for numbering Ax and i
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
40 200
Note also that the dependent values are
known at the boundaries (hence the term
boundary value problem)
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
40 200
Apply the governing equation at node 1
( )
( )
8 . 40 04 . 2
8 . 0 04 . 2 40
' ' 2
2 1
2 1
2
2 1
2
0
=
= +
A = A + +
T T
T T
T x h T T x h T
a
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
40 200
Apply the equation at node 2
( )
8 . 0 04 . 2
' ' 2
3 2 1
2
3 2
2
1
= +
A = A + +
T T T
T x h T T x h T
a
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
40 200
We get a similar equation at node 3
( )
8 . 0 04 . 2
' ' 2
4 3 2
2
4 3
2
2
= +
A = A + +
T T T
T x h T T x h T
a
( )
a i i i
T x h T T x h T
2
1
2
1
' ' 2 A = A + +
+
T(0)

T(10)

x=0 2 4 6 8 10
i=0 1 2 3 4 5
40 200
( )
8 . 200 04 . 2
8 . 0 200 04 . 2
' ' 2
4 3
4 3
2
5 4
2
3
= +
= +
A = A + +
T T
T T
T x h T T x h T
a
At node 4, we consider the
boundary at the right.
For the four interior nodes, we get the following
4 x 4 matrix
{ } | | 48 . 159 54 . 124 78 . 93 97 . 65
8 . 200
8 . 0
8 . 0
8 . 40
04 . 2 1 0 0
1 04 . 2 1 0
0 1 04 . 2 1
0 0 1 04 . 2
4
3
2
1
=

(
(
(
(

T
T
T
T
T
T
0
5 0
1 00
1 5 0
200
25 0
0 5 1 0
dist ance (m)
T
Analy t ical
Solut ion
Shoot ing
Met hod
Finit e
Differ ence
Example
Consider the previous example, but
with Ax=1. What is the matrix?
Specific Study Objectives
Understand the visual representation of
Eulers, Heuns and the improved polygon
methods.
Understand the difference between local and
global truncation errors
Know the general form of the Runge-Kutta
methods.
Understand the derivation of the second-
order RK method and how it relates to the
Taylor series expansion.

Specific Study Objectives
Realize that there are an infinite number of
possible versions for second- and higher-
order RK methods
Know how to apply any of the RK methods to
systems of equations
Understand the difference between initial
value and boundary value problems
end of lecture on ODE