Sei sulla pagina 1di 78

Ordinary Differential Equations

Jyun-Ming Chen
Fall 2015 2
Contents
• Review • Systems of ODE
• Euler’s method • Stability Issue
• 2nd order methods • Implicit Methods
– Midpoint • Adaptive Stepsize
– Heun’s
• Runge-Kutta Method

Fall 2015 3
Review
• DE (Differential Equation)
– An equation specifying the relations among the
rate change (derivatives) of variables
• ODE (Ordinary DE) vs. PDE (Partial DE)
– The number of independent variables involved

d 2x f f
x x(t )  1 f ( x, y )
dt 2
x y
Fall 2015 4
Review (cont)
• Solution of DE vs. Solution of Equation
• Solution of an • Geometrically,
equation:

f ( x)  x 2  1  0 f(x)
Solution : x  1  f (1)  0

Fall 2015 5
Need additional
conditions to
Review (cont) specify a solution

• Solution of an • Geometrically:
differential equation: x
dx
 x
dt
Solution :

t
x  ce  ce  t
  c  e t

That is, 
 t t 
 x  e , x  2e ,  t
 are all valid solutions 
 
Fall 2015 6
Concept of Vector Field

Fall 2015 7
Review (cont)
• Order of an ODE
– The highest derivative in the equation
• nth order ODE requires n conditions to
specify the solution
– IVP (initial value problem): All conditions
specified at the same (initial) point
– BVP (boundary value problem): otherwise

Fall 2015 8
IVP VS. BVP
y Revisit Shooting Problem
y  f (t )

g y   g (let g  10)


d2y
2
 10
dt
t dy
 10t  c1 (1)
dt
y  5t 2  c1t  c2 (2)

Fall 2015 9
IVP vs. BVP
IVP : BVP : Physical
y (t  0)  0 y (t  0)  0 meaning
y (t  0)  5 y (t  10)  100

c2  0 y (0)  c2  0
c1  5 y (10)  500  10c1  100  c1  60
 y (t )  5t  5t
2  y  5t 2  60t

Fall 2015 10
Ode2: solves 1st and 2nd order ODE
Ic1, ic2, bc: setting conditions
‘ do not evaluate

Maxima on ODE
dy d 2 x dx
  y , y ( 0)  2 2
  x  sin t , x(0)  1, x (0)  0
dx dt dt

Fall 2015 11
Linear ODE
• Linearity:
– Involves no product nor nonlinear functions of
y and its derivatives
• nth order linear ODE
dny d n 1 y dy
an ( x) n  an 1 ( x) n 1    a1 ( x)  a0 ( x) y  b( x)
dx dx dx

Fall 2015 12
Focus of This Chapter
• Solve IVP of nth order ODE numerically
• e.g.,
d2y  y (0)  1
  x, initial conditions  dy
 dx (0)  0
2
dx
Solve y ( x), x  0

Fall 2015 13
ODE (IVP)
• First order ODE (canonical form)
dy
 f ( x, y ), y ( x  0)  y0
dx

• Every nth order ODE can be converted to n


first order ODEs in the following method:

Fall 2015 14
n n 1
d y d y dy
an ( x) n  an 1 ( x) n 1    a1 ( x)  a0 ( x) y  b( x)
dx dx dx

dy1
y1 ( x)  y ( x)  y2
dx
dy1 dy
y2 ( x)   dy2
dx dx  y3
dx
dy2 d 2 y
y3 ( x )   
dx dx 2
 dyn 1
 yn
dyn 1 d n 1 y
dx
yn ( x)   n 1 dyn
dx dx an ( x )  an 1 ( x) yn    a1 ( x) y2  a0 ( x) y1  b( x)
dx

Fall 2015 15
Example
d3y d2y 2 dy
3
2 2 x  5x
dx dx dx
 dy3
 dx  2 y  x 2
y2  5 x
 y1  y 3

 
Let  y2  dydx1  dy  dy2
dx   y3
 y  dy2  d 2 y  dx
 3 dx dx2  dy1
 dx  y2

 y1   y2 
       
Let y   y2 , y  f ( y )   y3 
y   x 2 y  2 y  5x 
Fall 2015  3  2 3  16
Equation of Motion哪裡來?
Free Body Diagram (FBD)

Newton’s 2nd Law


F  cx  kx  mx
N  mg

Fall 2015 17
FBD: Pendulum

Newton’s 2nd Law


 mg sin   mL
Ts  mg cos 

Fall 2015 18
FBD can be difficult…(ref)

Fall 2015 19
Euler-Lagrange Equation (ref)
Examples

T
1 2
2
mx
1
 
T  m l
2
 2

V  kx
1 2 L  T V V  mgl cos 

 
2
d L L 1  2
L  T  V  m l  mgl cos 
L  T V  2
L dt q q L
  kx   mgl sin 
x 
L L
 mx  ml 2
x 
mx  kx  0 l  g sin   0
Pendulum on a Moveable Support (ref)

Fall 2015 22
x1  x
x2  x  x1   x2 
x3    x   x4 
x  
 3   
4

M  m x2  mlx4 cos x3  mlx42 sin x3  0


 x2   
   
x4 
cos x3 g
x2  sin x3  0  x4   
l l
 M  m ml cos x3   x  mlx42 sin x3 
 cos x3  2    g 
 1   x4    sin x3 
 l   l 
 M  m ml cos x3 
K   cos x3 
 1 
 l 
 x 2   mlx 2
4 sin x3 
1  
 x   K g
 4   sin x3 
Fall 2015  l  23
[Homework: Double Pendulum]

Fall 2015 24
End of Review
The Canonical Problem
dy
 f ( x, y ), y ( x0 )  y0
dx

y dy y dy
  f ( x, y ) x  0, 
x dx x dx
y  f ( x, y )x
y ( x  x)  y ( x)  y  f ( x, y )x This is
Euler’s
 y ( x  x)  y ( x)  f ( x, y )x method
Fall 2015 26
Example
dy
  y, y (0)  1
dx
Compare with exact sol:
f ( x, y )   y, f (0,1)  1
Choose x  0.1
y (0.1)  y (0)  (1)  0.1  1  0.1  0.9
y (0.2)  y (0.1)  (0.9)  0.1  0.9  0.09  0.81
y (0.3)  y (0.2)  (0.81)  0.1  0.81  0.081  0.729

Fall 2015 27
y
Example (cont)
1 Exact Solution :
dy
 dx
 y
ln y   x  c
 y  ce  x y (0)  c  1
 y=e–x
 y  ex

0.1 x

Fall 2015 28
Error Analysis
(Geometric Interpretation)
dy
 f ( x, y ), y ( x0 )  y0
dx
y dy
  f ( x, y )
x dx
y  f ( x, y )x
y1  y0  y  f ( x0 , y0 )x

 y1  y0  f ( x0 , y0 )x

Think in
terms of
Taylor’s If the true solution were a straight line, then
Fallexpansion
2015 Euler is exact 29
Error Analysis
(From Taylor’s Expansion)

y ( x  x)  y ( x)  y( x)x  y( x) x 2


2  y( x) x 3
6 
 y ( x)  f ( x, y )x  d
dx f ( x, y ) x 2
2 
Euler’s
Euler’s truncation error
O(x2) per step

1st order method

Fall 2015 30
y
Cumulative Error



x
Remark:
x Error 
x=0 x=T
But computation time

Number of steps = T/x


Cumulative Err. = (T/x)  O(x2) = O(x)
Fall 2015 31
Example (Euler’s)
dy x
  y , y0  1 exact : y ( x)  e
dx

f ( x, y )   y ( x )
y ( x  x)  y ( x)  y ( x)x
Let x  0.1
y ( x  x)  y ( x)(1  x)  0.9 y ( x)

y (0.1)  0.9, y (0.2)  0.81, y (0.3)  0.729,...


Fall 2015 32
Methods Improving Euler

Motivated by
Geometric Interpretation
Midpoint Method

y( x  2x )  y( x)  f ( x, y( x)) 2x


y( x  x)  y( x)  f ( x  2x , y( x  2x )) x
Fall 2015 34
Example (Midpoint)
dy x
  y , y0  1 exact : y ( x)  e
dx
f ( x, y )   y
y ( x  2x )  y ( x)  f ( x, y ) 2x
 y ( x)  y ( x) 2x  y ( x)1  2x 
y ( x  x)  y ( x)  f ( x  2x , y ( x  2x )) x
 y ( x)   y( x)1  2x x
y ( x  x)  y ( x)   y ( x)1  2x x
Let x  0.1
y ( x  0.1)  y ( x)  y ( x)(1  0.05)(0.1)  0.905 y ( x)
Fall 2015 y (0.1)  0.905, y (0.2)  0.819, y (0.3)  0.741,... 35
Heun’s Method

y 0 ( x  x)  y( x)  f ( x, y( x)) x (Predictor )


 
y( x  x)  y( x)  12 f ( x, y)  f ( x  x, y 0 ( x  x)) x
Fall 2015 36
Example (Heun’s)
dy
  y , y0  1 exact : y ( x)  e  x
dx
f ( x, y )   y
y 0 ( x  x)  y ( x)  f ( x, y )x
 y ( x)  y ( x)x  y ( x)1  x 
 
y ( x  x)  y ( x)  12 f ( x, y )  f ( x  x, y 0 ( x  x)) x
 y ( x)  12  y( x)  y ( x  x)x
0

 y ( x)  12  y ( x)  y ( x)1  x x Note the result


is the same as
Let x  0.1 Midpoint!?
y ( x  0.1)  y ( x)1  12  1  0.9 0.1  0.905 y ( x)

y (0.1)  0.905, y (0.2)  0.819, y (0.3)  0.741,...


Fall 2015 37
Remark
• Comparison of Euler, Euler Midpoint
Exact
Heun, midpoint (error) (error)

– 1st order: Euler 0.9 0.905


y(0.1) 0.905
(0.005)
– 2nd order: Heun, (0)

midpoint y(0.2) 0.819


0.81 0.819
(0.009) (0)
• “order”:
0.729 0.741
• All are special cases of y(0.3) 0.741
(0.012) (0)
RK (Runge-Kutta)
methods
Fall 2015 38
RK Methods

yi 1  yi  ( xi , yi , h)h
where
yi  y ( xi ), yi 1  y ( xi  h)
( xi , yi , h) : incrementa l function
(estimate slope from i to i  1)

Fall 2015 39
RK Methods (cont)
  a1k1  a2 k 2    an k n
a ' s : constant
k1  f ( xi , yi )
k 2  f ( xi  p1h, yi  q11k1h)
k3  f ( xi  p2 h, yi  q21k1h  q22k 2 h)

k n  f ( xi  pi h, yi  qn 1,1k1n    qn 1,n 1k n 1n)
pi , qij : constants
 Constants (ai , pi , qij ) are chosen to match Taylor' s expansion
Fall 2015 40
Taylor’s Expansion

y ( xi  h)  y ( xi )  y( xi )h  d
dx y( xi ) h2
2  O(h ) 3

f f
 y ( xi )  f ( xi , yi )h  [ x i  y i fi ] h2
2  O(h 3 )
f f
 yi  f i h  [ x i  y i fi ] h2
2  O(h 3 )

d f dx f dy f f
f ( x, y )     y'
dx x dx y dx x y
f f
  f
x y
Fall 2015 41
y( xi  h)  yi  f i h  [ fx  fy f i ] h2  O(h3 )
2

i i

RK 1st Order
n 1
  a1k1  a1 f ( xi , yi )
yi 1  yi  a1 f ( xi , yi )h

Compare with Taylor' s expantion


a1  1
 yi 1  yi  f ( xi , yi )h Euler' s method

Fall 2015 42
y( xi  h)  yi  f i h  [ fx  fy f i ] h2  O(h3 )
2

i i

RK 2nd Order
n  2   a1k1  a2 k 2
k1  f ( x1 , yi )
f i h : a1  a2  1
k 2  f ( xi  p1h, yi  q11k1h) f
x h 2
: a2 p1  1 2
yi 1  yi  (a1k1  a2 k 2 )h f
y f i h : a2 q11  2
2 1

Expand k 2 :
f f
k 2  f ( xi , yi )  ( p1h)  (q11k1h)  O(h 2 )
x i y i
yi 1  yi  a1k1h  a2 [ f i  fx p1h  fy q11k1h]h  O(h 3 )
f f
Fall 2015
 yi  a1 f i h  a2 f i h  a2 x p1h 2  a2 y q11 f i h 2  O(h433 )
RK 2 nd Order (cont)
f i h : a1  a2  1 
f  3 eqns,
x h :
2
a2 p1  2 
1

f  4 unknowns
y f i h : a2 q11  2 
2 1

a2  1  a1  0, p1  12 , q11  12
yi 1  yi  (a1k1  a2 k 2 )h
 yi  k 2 h  yi  f ( xi  p1h, yi  q11k1h)h
yi 1  yi  f ( xi  12 h, yi  12 f ( xi , yi )) h

yi 1  yi  (a1k1  a2 k2 )h  yi  12 (k1  k2 )h
k1  f ( xi , yi ), k2  f ( xi  h, yi  f i h)
Fall 2015 44
RK 4th Order
• Mostly commonly used one
• Higher order … more evaluation, but less
gain on accuracy
Classical
yn 1  yn  (k1  2k 2  2k3  k 4 )h
1
6
4th order
where RK
k1  f ( xn ,yn )
k 2  f ( xn  12 h,yn  12 k1h)
k3  f ( xn  12 h,yn  12 k 2 h)
k 4  f ( xn  h,yn  k3h)
Fall 2015 45
Classical 4 th order RK

Fall 2015 46
System of ODE
• Convert higher order ODE to 1st order
ODEs
• All methods equally apply, in vector form

Fall 2015 47
Example (Mass-Spring-Damper System)

• Governing Equation
mx  cx  kx  0
• After setting the initial conditions x(0) and x’(0),
compute the position and velocity of the mass for
any t > 0
Initial Condition
c

m
k
Fall 2015
x 48
Example (cont)
x1  x, x2  x
then
 x1  x2

mx2  cx2  kx1  0   1
 x2  m ( kx1  cx2 )
 x1 (0)  1
IC : 
 x2 (0)  0
n  Dim Euler
     
y ( x  x)  y ( x)  f ( x, y )x, y (0)  y0
Fall 2015 49
Example (cont)
Assume m=1,c=1, k=1
  x     x    x2 
x    f ( x , t )   1  (for ease of computation)
1 2
f   
 x2   m (kx1  cx2 )    x1  x2 

set t=0.1
     1  0   1 
x (0.1)  x (0.0)  f ( x (0.0),0.0)  0.1      0.1   
 0  1  0   0.1
     1    0.1   0.99 
x (0.2)  x (0.1)  f ( x (0.1),0.1)  0.1      0.1   
  0.1    1  ( 0.1 )    0.19 
     0.99    0.19   0.971 
x (0.3)  x (0.2)  f ( x (0.2),0.2)  0.1      0.1   
  0.19    0.99  (0.19)    0.27 

Fall 2015 50
Stability: Symptom

dy
 10 y, y(0)  1
dx
exact solution : y( x)  e 10 x

Fall 2015 51
Symptom: Unstable Spring System

Become unstable instantly …

Start with this …


Cause by stiff (k=4000) springs

Fall 2015 52
Stability (cont)
• Example Problem: dy
 y, y(0)  1,  0
Euler : dx
yi 1  yi  yi h  (1  h) yi exact solution : y( x)  e x
stable if 1  h  1
unstable if 1  h  1 
 
That is,   stability limit h*  2
 1  h  1, or 1  h  1 
 
 h  0 h  2  Conditionally stable
exact solution : y ( x)  e x
Fall 2015 53
Discussion
• Different algorithm different stability limit
– Check Midpoint Method
• Different problem different stability limit
– use the previous problem as benchmark

Fall 2015 54
Review: Numerical Differentiation
Taylor’s expansion:

Forward difference

Backward difference

Fall 2015 55
Numerical Difference (cont)
Central difference

Fall 2015 56
Implicit Method (Backward Euler)
dy
dx  f ( x, y )
Euler : (i  1, i ) Forward difference
yi 1  yi
 O(h)  f ( xi , yi )
h
 yi 1  yi  f ( xi , yi )h  O(h 2 )

Backward Euler : (i, i  1) Backward difference


yi  yi 1
 O(h)  f ( xi , yi )
h
 yi  f ( xi , yi )h  yi 1  O(h 2 )

i yi 1  f ( xi 1 , yi 1 )h  yi  O(h 2 )
Fall 2015 57
Example
f ( x, y )  y
 yi 1  f ( xi 1 , yi 1 )h  yi 1  yi 1h  yi  O(h 2 )
 (1  h) yi 1  yi  O(h 2 )
yi 1  11h yi

• Remark:
– Always stable (for this problem)
– Truncation error the same as Euler (only
improve the stability)
Fall 2015 58
Stiff Set of ODE

Use the change of variable u   2 y   z  v   y   z 


u  v  y  u  v   y
u  2v  z  1000u  2v   1000 z
Get the following solution: y  c1e  x z  c2 e 1000x
u (0)  1  2c1  c2  1
v(0)  1  c1  c2  0  c1  c2  1

Stability limit A stiff equation is a differential equation for which


certain numerical methods for solving the equation
are numerically unstable, unless the step size is
Fall 2015 taken to be extremely small 59
Fall 2015 60
Linear System of ODE with
Constant Coefficients
y  Cy

Explicit Euler Implicit Euler


yi 1  yi  hCyi  O(h )2 yi  yi 1  hCyi  O(h 2 )
yi 1  ( I  hC ) yi  O(h )
2 I  hC yi  yi 1  O(h 2 )
yi 1  I  hC  yi yi 1  I  hC  yi
1

Do not really use (..)-1.


Solve linear system
Fall 2015 instead 61
Analysis
Explicit and implicit Euler both in the form: y  Ay
x (i 1)  Ax (i )
x ( 0 )  c1 x1  c2 x2 represent x(0) in eigen basis
x (1)  Ax ( 0 )  Ac1 x1  c2 x2   c1 Ax1  c2 Ax2  c11 x1  c2 2 x2
x ( 2 )  Ax (1)  Ac11 x1  c2 2 x2   c11 Ax1  c2 2 Ax2  c112 x1  c2 22 x2
x ( n )  c11n x1  c2 n2 x2

x will converge if |i|  1

Fall 2015 62
Analysis (cont)
Explicit Euler Implicit Euler
yi 1  I  hC  yi yi 1  I  hC  yi
1

i : e - values of C
hi : e - values of hC
1  hi : e - values of I  hC 1  hi 1 : e - values of I  hC 1

• Stiff equations have large eigenvalues


• Explicit Euler requires small h to converge
• Implicit Euler always converges (in this problem)

Fall 2015 63
Example
 u  998u  1998v 
 u   998  1998 u 
         
v  999u  1999v  v   999 1999  v 

Explicit Implicit
stable if 1  1000h  1 1
stable if 1
 1  1  1000h  1 1  1000h
1000h  2  h  0.002
Fall 2015 64
Solving implicit methods by linearization is called
a “semi-implicit” method

Semi-Implicit Euler
• Not guaranteed to be stable, but usually is
Jacobian

f f
yn 1  yn  hf  yn   h yn 1  h yn
y y
 f   f 
 I  h  y n 1   I  h  yn  hf  yn 
 y   y 

Fall 2015 65
About Jacobian
Taylor’s expansion:
f 1  x1  h, x2  h   f 1  x1 , x2   xf11 h  xf12 h  O(h 2 )
f 2  x1  h, x2  h   f 2  x1 , x2   fx21 h  xf22 h  O (h 2 )
f 3  x1  h, x2  h   f 3  x1 , x2   xf31 h  xf32 h  O (h 2 )

 f 1  x1  h, x2  h   f 1  x1 , x2   x11 x12 
f f

     f 2 f 2  h 
f
 2 1  x  h , x 2  h   f  x , x  
  2 1 2   x1 x2  h   O ( h 2
)
 f 3  x1  h, x2  h   f 3  x1 , x2   xf3 xf3   
 1 2 

    f1 , f 2 , f 3  
 
  
f x  h  f x   h Jacobian
  x1 , x2 
Fall 2015 66
damping force
 cx
Initial Condition
c

m
k
x

Equation of Motion :
 x1  x  x1  x2
f mx  cx  kx  mx 
 2
x  
x

 
x 2   1

m kx1  cx2

mx  cx  kx  0
 0  1
y    k c 
y
m m 
Implicit Euler 1
1 h 
y  Cy yn 1   kh hc 
yn
yi 1  I  hC  yi
1  m 1 m 
Fall 2015 67
fake damping force
 cx 2
Initial Condition
c

m
k
x

Equation of Motion : Let m  c  k  1

 f  m 
x   cx 2
 kx  mx  x1  x  x1  x2
 
mx  cx  kx  0
2  2
x  
x 
 2
x   x1  x 2
2

y  f ( y )
f  0 1 

y  1  2 x2 
1
1  h   x2 
Semi-Implicit Euler yn 1  yn  h    x  682
Fall 2015  h 1  2 x 2  
h 1 x 2
Ex: Semi-Implicit Euler
 y1  y2
y   g , y (0)  0, y (0)  3   , y1 (0)  0, y2 (0)  3
 y 2   g
f   f1 , f 2  0 1
J  
y   y1 , y2  0 0
1
1  h 1 h
yn 1  yn  h   f n  yn  h   fn Compare Euler
0 1  0 1  yn 1  yn  hf n
 y1 (t  h)   y1 (t )  h h 2  y2 (t )   y1 (t  h)   y1 (t )   y2 (t ) 
       h
          y2 (t  h)   y2 (t )    g 
 y2 (t  h)   y2 (t )  0 h   g 

Fall 2015 69
Amazingly, this translates to…
 y1 (t  h)   y1 (t )  h h 2  y2 (t ) 
        
 y2 (t  h)   y2 (t )  0 h   g 
y (t  t )  y (t )  v(t )t  g t 
2

v(t  t )  v(t )  gt

Very similar to Verlet


integration formula… no
wonder Verlet is pretty stable

Fall 2015 70
GSL
x¢¢ + m x¢ ( x -1) + x = 0, x(0) =1, x¢(0) = 0
2

find x (100)
x¢ = y
y¢ = -x + m y (1- x 2 )

Fall 2015 71
Initialization

Fall 2015 72
Iteration

Fall 2015 73
Extra

Fall 2015 74
Adaptive Stepsize
• Solving ODE
numerically … tracing
the integral curve y(x)
• what’s wrong with
uniform step size
– Uniformly small: waste
effort
– Uniformly large: might
miss details

Fall 2015 75
Step Doubling
• Idea:
– Estimate the truncation error by taking each
step twice: one full step, two half steps
– control the step size such that the estimated
error is not too big.

1(2h1)

2(h1)
Desired h0
Fall 2015 76
Ex: RK 2nd Order
y ( x  2h)  y1  c(2h) 3  O(h 4 )
y ( x  2h)  y2  2c(h) 3  O(h 4 )
y1  c(2h) 3  O(h 4 )  y2  2c(h) 3  O(h 4 )
  y2  y1  6ch 3    h 3 (the truncatio n error)
3 1

 0  h0   0  3

    h0    h1 Overhead:
1  h1   1  # of f(x,y) evaluations
1 : the estimated error by step h1 24–2 = 6
h0 : the stepsize to produce the desired error  0

Fall 2015 77
Adaptive Step with RK4 (NR)

Fall 2015 78