S OLVING O PTIMAL C ONTROL P ROBLEMS WITH ACADO

TOOLKIT
Boris Houska, Moritz Diehl

Optimization in Engineering Center (OPTEC)

K. U. Leuven

Trondheim, June 07, 2010

Solving Optimal Control Problems with ACADO Toolkit – p. 1/19

Overview

• Scope of ACADO Toolkit

• An Optimal Control Tutorial Example
(with Software Demo)
• Algorithms and Modules in ACADO
• Outlook

Solving Optimal Control Problems with ACADO Toolkit – p. 2/19

 Motivation: Optimal Control and En-
gineering Applications

Optimal Control Applications in OPTEC:

• Optimal Robot Control, Kite Control, Solar Power
Plants, Bio-chemical reactions...

Solving Optimal Control Problems with ACADO Toolkit – p. 3/19

 Motivation: Optimal Control and En-
gineering Applications

Optimal Control Applications in OPTEC:

•Optimal Robot Control, Kite Control, Solar Power
Plants, Bio-chemical reactions...
Need to solve optimal control problems:
RT
minimize 0
L(τ, y(τ ), u(τ ), p) dτ + M (y(T ), p)
y(·),u(·),p,T

subject to:

∀t ∈ [0, T ] : 0 = f (t, ẏ(t), y(t), u(t), p)

0 = r(y(0), y(T ), p)

∀t ∈ [0, T ] : 0 ≥ s(t, y(t), u(t), p)

Solving Optimal Control Problems with ACADO Toolkit – p. 3/19

 What is ACADO Toolkit?

ACADO Toolkit:
• Automatic Control And Dynamic Optimization
• great variety of numerical optimization algorithms
• open framework for users and developers

Solving Optimal Control Problems with ACADO Toolkit – p. 4/19

 What is ACADO Toolkit?

ACADO Toolkit:
• Automatic Control And Dynamic Optimization
• great variety of numerical optimization algorithms
• open framework for users and developers
Key Properties of ACADO Toolkit
• Open Source (LGPL) www.acadotoolkit.org
• user interfaces close to mathematical syntax
• Code extensibility: use C++ capabilities
• Self-containedness: only need C++ compiler

Solving Optimal Control Problems with ACADO Toolkit – p. 4/19

Implemented Problem Classes in
ACADO Toolkit

• Optimal control of dynamic systems

(ODE, DAE, hybrid models)

Solving Optimal Control Problems with ACADO Toolkit – p. 5/19

Implemented Problem Classes in
ACADO Toolkit

• Optimal control of dynamic systems

(ODE, DAE, hybrid models)
• Multi-objective optimization
(joint work with Filip Logist)

Solving Optimal Control Problems with ACADO Toolkit – p. 5/19

Implemented Problem Classes in
ACADO Toolkit

• Optimal control of dynamic systems

(ODE, DAE, hybrid models)
• Multi-objective optimization
(joint work with Filip Logist)
• State and parameter estimation
(Niels Haverbeke, Ioanna Stamati, Boris Houska)

Solving Optimal Control Problems with ACADO Toolkit – p. 5/19

Implemented Problem Classes in
ACADO Toolkit

• Optimal control of dynamic systems

(ODE, DAE, hybrid models)
• Multi-objective optimization
(joint work with Filip Logist)
• State and parameter estimation
(Niels Haverbeke, Ioanna Stamati, Boris Houska)
• Feedback control based on real time
optimization
(with Hans Joachim Ferreau)

Solving Optimal Control Problems with ACADO Toolkit – p. 5/19

Implemented Problem Classes in
ACADO Toolkit

• Optimal control of dynamic systems

(ODE, DAE, hybrid models)
• Multi-objective optimization
(joint work with Filip Logist)
• State and parameter estimation
(Niels Haverbeke, Ioanna Stamati, Boris Houska)
• Feedback control based on real time
optimization
(with Hans Joachim Ferreau)
• Robust optimal control
(my PhD. thesis)

Solving Optimal Control Problems with ACADO Toolkit – p. 5/19

Overview

• Scope of ACADO Toolkit

• An Optimal Control Tutorial Example
(with Software Demo)
• Algorithms and Modules in ACADO
• Outlook

Solving Optimal Control Problems with ACADO Toolkit – p. 6/19

Tutorial Example: Time Optimal Con-
trol of a Rocket

Mathematical Formulation:

minimize T
s(·),v(·),m(·),u(·),T

subject to

ṡ(t) = v(t)
u(t)−0.2 v(t)2
v̇(t) = m(t)
2
ṁ(t) = −0.01 u(t)

s(0) = 0 s(T ) = 10
v(0) = 0 v(T ) = 0
m(0) = 1

−0.1 ≤ v(t) ≤ 1.7

−1.1 ≤ u(t) ≤ 1.1
5 ≤ T ≤ 15

Solving Optimal Control Problems with ACADO Toolkit – p. 7/19

Tutorial Example: Time Optimal Con-
trol of a Rocket

DifferentialState s,v,m;
Mathematical Formulation:
Control u;
Parameter T;
minimize T DifferentialEquation f( 0.0, T );
s(·),v(·),m(·),u(·),T OCP ocp( 0.0, T );
ocp.minimizeMayerTerm( T );
subject to
f « dot(s) == v;
ṡ(t) = v(t) f « dot(v) == (u-0.2*v*v)/m;
u(t)−0.2 v(t)2 f « dot(m) == -0.01*u*u;
v̇(t) = m(t) ocp.subjectTo( f );
2
ṁ(t) = −0.01 u(t)
ocp.subjectTo( AT_START, s == 0.0 );
ocp.subjectTo( AT_START, v == 0.0 );
s(0) = 0 s(T ) = 10 ocp.subjectTo( AT_START, m == 1.0 );
ocp.subjectTo( AT_END , s == 10.0 );
v(0) = 0 v(T ) = 0 ocp.subjectTo( AT_END , v == 0.0 );
m(0) = 1
ocp.subjectTo( -0.1 <= v <= 1.7 );
−0.1 ≤ v(t) ≤ 1.7 ocp.subjectTo( -1.1 <= u <= 1.1 );
ocp.subjectTo( 5.0 <= T <= 15.0 );
−1.1 ≤ u(t) ≤ 1.1 OptimizationAlgorithm algorithm(ocp);
algorithm.solve();
5 ≤ T ≤ 15

Solving Optimal Control Problems with ACADO Toolkit – p. 7/19

 Tutorial Example: Optimal Control of
a Jojo

Jojo Model
• Position/velocity of the hand: x, v
• Position/velocity of the hand: y , w
• Control the acceleration u of the hand

Solving Optimal Control Problems with ACADO Toolkit – p. 8/19

 Tutorial Example: Optimal Control of
a Jojo

Jojo Model
• Position/velocity of the hand: x, v
• Position/velocity of the hand: y , w
• Control the acceleration u of the hand
0 1 0 1
ẋ(t) v(t)
B C B C
B v̇(t) C B u(t) C
C = B
B C B C
B C
B ẏ(t) C B w(t) C
@ A @ A
ẇ(t) −µg + ku(t) + a(w(t) − v(t))

• damping ratio: k = mrJ2 +J , effective mass ratio:

µ = 1 − k , roll friction coefficient a

Solving Optimal Control Problems with ACADO Toolkit – p. 8/19

 Tutorial Example: Optimal Control of
a Jojo

Interesting aspect of the jojo model:

• At the time t for which x(t ) − y(t ) = L:
1 1 1
p
w(t1 ) = lim kv(t) + k2 v(t)2 + k(w(t)2 − 2v(t)w(t))
t→t−
1

Solving Optimal Control Problems with ACADO Toolkit – p. 9/19

 Tutorial Example: Optimal Control of
a Jojo

Interesting aspect of the jojo model:

• At the time t for which x(t ) − y(t ) = L:
1 1 1
p
w(t1 ) = lim kv(t) + k2 v(t)2 + k(w(t)2 − 2v(t)w(t))
t→t−
1

Our aim:
R t2
• Minimize the control action Φ := 0 u2 (t)dt
• Satisfy all model equations and constraints:
x(0) = y(0) = 0 m v(0) = w(0) = 0 ms
x(t1 ) − y(t1 ) = L
x(t2 ) = y(t2 ) = −0.1 m v(t2 ) = w(t2 ) = 0 ms

Solving Optimal Control Problems with ACADO Toolkit – p. 9/19

 Tutorial Example: Optimal Control of
a Jojo

Formulation of multi-stage optimal control problems

minimize Φ(x(·), u(·), p, T )

x(·),u(·),p,T

subject to:

∀t ∈ Ti : 0 = Fi (t, x(t), ẋ(t), u(t), p)

0 = lim Jk (x(t), x(tk ), p)

t→t−
k

∀t ∈ Ti : 0 ≤ hi (t, x(t), u(t), p)

0 = r(x(t1 ), . . . , x(tn+1 ), p)

for all i ∈ I, k ∈ K

Solving Optimal Control Problems with ACADO Toolkit – p. 10/19

Tutorial Example: Optimal Control of
a Jojo

Solving Optimal Control Problems with ACADO Toolkit – p. 11/19

Overview

• Scope of ACADO Toolkit

• An Optimal Control Tutorial Example
(with Software Demo)
• Algorithms and Modules in ACADO
• Outlook

Solving Optimal Control Problems with ACADO Toolkit – p. 12/19

The Power of Symbolic Functions

Symbolic Functions allow:

• Dependency/Sparsity
Detection
• Automatic Differentiation
• Symbolic Differentiation
• Convexity Detection
• Code Optimization
• C-code Generation

Solving Optimal Control Problems with ACADO Toolkit – p. 13/19

The Power of Symbolic Functions

Symbolic Functions allow:

Example 1:
• Dependency/Sparsity DifferentialState x;
Detection IntermediateState z;
TIME t;
• Automatic Differentiation Function f;

• Symbolic Differentiation z = 0.5*x + 1.0 ;

f « exp(x) + t ;
• Convexity Detection f « exp(z+exp(z)) ;

• Code Optimization if( f.isConvex() == YES )

printf("f is convex. ");
• C-code Generation

Solving Optimal Control Problems with ACADO Toolkit – p. 13/19

 Symbolic Functions

Example 2 (code optimization):

• Matrix A(3,3);
Vector b(3);
DifferentialState x(3);
Function f;
A.setZero();
A(0,0) = 1.0; A(1,1) = 2.0; A(2,2) = 3.0;
b(0) = 1.0; b(1) = 1.0; b(2) = 1.0;
f « A*x + b;
• expect 9 multiplications 12 additions to evaluate f.
• ACADO needs 3 multiplications and 6 additions.

Solving Optimal Control Problems with ACADO Toolkit – p. 14/19

 Integration Algorithms

DAE simulation and sensitivity generation:

• several Runge Kutta and a BDF integrator.
• first and second order automatic differentiation.

Solving Optimal Control Problems with ACADO Toolkit – p. 15/19

 Integration Algorithms

DAE simulation and sensitivity generation:

• several Runge Kutta and a BDF integrator.
• first and second order automatic differentiation.
• The BDF routine solves fully implicit index 1 DAE’s:
∀t ∈ [0, T ] : F (t, ẏ(t), y(t), u(t), p) = 0 .

Solving Optimal Control Problems with ACADO Toolkit – p. 15/19

 Integration Algorithms

DAE simulation and sensitivity generation:

• several Runge Kutta and a BDF integrator.
• first and second order automatic differentiation.
• The BDF routine solves fully implicit index 1 DAE’s:
∀t ∈ [0, T ] : F (t, ẏ(t), y(t), u(t), p) = 0 .
• Continous output of trajectories and sensitivities.
• Integrators are also available as stand alone
package.
• Sparse LA solvers can be linked.

Solving Optimal Control Problems with ACADO Toolkit – p. 15/19

 Nonlinear Optimization Algorithms

Nonlinear Optimal Control Problem

• ACADO solves problem of the general form:

RT
minimize 0
L(τ, y(τ ), u(τ ), p) dτ + M (y(T ), p)
y(·),u(·),p,T

subject to:

∀t ∈ [0, T ] : 0 = f (t, ẏ(t), y(t), u(t), p)

0 = r(y(0), y(T ), p)

∀t ∈ [0, T ] : 0 ≥ s(t, y(t), u(t), p)

Solving Optimal Control Problems with ACADO Toolkit – p. 16/19

 Nonlinear Optimization Algorithms

Implemented Solution Methods

• Discretization: Single- or Multiple Shooting
• NLP solution: several SQP type methods e.g. with
• BFGS Hessian approximations or

• Gauss-Newton Hessian approximations

• Globalization: based on line search

• QP solution: active set methods (qpOASES)

Solving Optimal Control Problems with ACADO Toolkit – p. 17/19

 Nonlinear Optimization Algorithms

Implemented Solution Methods

• Discretization: Single- or Multiple Shooting
• NLP solution: several SQP type methods e.g. with
• BFGS Hessian approximations or

• Gauss-Newton Hessian approximations

• Globalization: based on line search

• QP solution: active set methods (qpOASES)
Currently under development
• Collocation methods, Interior point methods
• SCP methods, Lifted Newton methods, ...

Solving Optimal Control Problems with ACADO Toolkit – p. 17/19

 Summary

Highlights of ACADO
• Self contained C++ code
• Automatic differentiation, integration routines
• Single- and multiple shooting
• SQP methods (exact Hessian, GN, BFGS,...)
• Easy setup of optimal control problems

Solving Optimal Control Problems with ACADO Toolkit – p. 18/19

 Summary

Highlights of ACADO
• Self contained C++ code
• Automatic differentiation, integration routines
• Single- and multiple shooting
• SQP methods (exact Hessian, GN, BFGS,...)
• Easy setup of optimal control problems
In Exercises:
• Simulation, Optimal Control, MPC, ...
• Please install from www.acadotoolkit.org.

Solving Optimal Control Problems with ACADO Toolkit – p. 18/19

Thank you for your attention!

Solving Optimal Control Problems with ACADO Toolkit – p. 19/19