Particle Swarm Optimization (PSO) Algorithm

and I ts Application in Engineering Design

Optimization
School of Information Technology
Indian Institute of Technology Kharagpur
September 9, 2005
Sushanta Kumar Mandal
Research Scholar
By
Outline
Introduction to Optimization
Optimization Procedure
Different Optimization Algorithms
Different Global Optimization Algorithms
Particle Swarm Optimization (PSO) Algorithm
Application of PSO in Design Optimization
Problems
Optimization
As ageless
as time
Calculus
Maximum and minimum of a smooth
function is reached at a stationary
point where its gradient vanishes.
Optimization is Everywhere
The more we know about something,
the more we see where optimization
can be applied.
Some personal decision making
- Finding fastest route home or class
- Optimal allocation of time for home work
- Optimal budgeting
Goal of Optimization
Find values of the variables that
minimize or maximize the objective
function while satisfying the
constraints.
Component of Optimization
Problem
Objective Function: An objective function
which we want to minimize or maximize.
For example, in a manufacturing process, we might
want to maximize the profit or minimize the cost.
In fitting experimental data to a user-defined
model, we might minimize the total deviation of
observed data from predictions based on the model.
In designing an inductor, we might want to
maximize the Quality Factor and minimize the
area.
Component of Optimization
Problem
Design Variables: A set of unknowns or
variables which affect the value of the objective
function.
In the manufacturing problem, the variables might
include the amounts of different resources used or the
time spent on each activity.
In fitting-the-data problem, the unknowns are the
parameters that define the model.
In the inductor design problem, the variables used
define the layout geometry of the panel.
Component of Optimization
Problem
Constraints: A set of constraints that allow the
unknowns to take on certain values but exclude
others.
For the manufacturing problem, it does not make
sense to spend a negative amount of time on any
activity, so we constrain all the "time" variables to be
non-negative.
In the inductor design problem, we would probably
want to limit the upper and lower value of layout
parameters and to target an inductance value within
the tolerance level.
Are All these ingredients
necessary?
Almost all optimization problems have objective
function.
No objective function. In some cases (for example,
design of integrated circuit layouts), the goal is to
find a set of variables that satisfies the constraints of
the model. The user does not particularly want to
optimize anything so there is no reason to define an
objective function. This type of problems is usually
called a feasibility problem.
Are All these ingredients
necessary?
Variables are essential. If there are no variables, we
cannot define the objective function and the problem
constraints.
Constraints are not essential. In fact, the field of
unconstrained optimization is a large and important
one for which a lot of algorithms and software are
available. It's been argued that almost all problems
really do have constraints.
What We Need for Optimization
Models: Modeling is the process of identifying
objective function, variables and constraints. The goal
of models is insight not the numbers. A good
mathematical model of the optimization problem is
needed.
Algorithms: Typically, an interesting model is too
complicated to be able to solve in with paper and
pencil. An effective and reliable numerical algorithm is
needed to solve the problem. There is no universal
optimization algorithm. Algorithm should have
robustness (good performance for a wide class of
problems), efficiency (not too much computer time)
and accuracy (can identify the error)
Flowchart of Optimal Design
Procedure
Need for optimization
Choose design variables
Formulate constraints
Formulate objective function
Set up variable bounds
Select an optimization algorithm
Obtain solution(s)
Mathematical Formulation of
Optimization Problems
( )
( )
1 2
minimize theobjectivefunction
min ( ), , ,.......,
subject toconstraints
( ) 0
0
n
i
i
f x x x x x
c x
c x
=
>
=
( ) ( )
2 2
1 2
2 2
1 2
1 2
Example
min 2 1
subject : 0
2
x x
x x
x x
(
+

s
+ s
Constraints
Inequality constraints: x
1
2
x
2
2
s0
Equality constraints: x
1
= 2
Variable Bounds
Maximum and minimum bounds on each
design variable.
Without variable bounds the constraints
completely surround the feasible region.
Variable bounds are used to confine the
search algorithm within these bounds.
Ex:

( ) ( ) L U
i i i
x x x s s
Classification of Optimization
Methods
Single variable
Multi-variable
Constrained
Non-constrained
Single objective
Multi-objective
Linear
Non-linear

Classifications of Optimization
Methods
Local and Global Optimizers
A local minimizer, x
B
*
, of the region B, is defined so
that: f(x
B
*
)sf(x), xeB.
Ex: Gradient based search methods, Newton-Rapson
algorithms, Steepest Decent, Conjugate-Gradient
algorithms, Levenberg-Marquardt algorithm etc.
Shortcomings: 1) One requires an initial guess to start
with. 2) Convergence to an optimal solution depends on
the chosen initial guess. 3) Most algorithms tend to get
stuck to a sub-optimal solution. 4) An algorithm
efficient in solving one optimization problem may not
be efficient in solving another one. 5) These are useful
over a relatively narrow range.
Local and Global Optimizers
The global optimizer, x
*
, is defined so that f(x
*
)sf(x),
xeS where S is the search space.
Ex. Simulated Annealing algorithm, Genetic
Algorithm, Ant Colony, Geometric Programming,
Particle Swarm Optimization etc.
Local and Global Optimizers
Particle Swarm Optimization
Evolutionary computational technique based on
the movement and intelligence of swarms
looking for the most fertile feeding location
It was developed in 1995 by James Kennedy and
Russell Eberhart
Simple algorithm, easy to implement and few
parameters to adjust mainly the velocity
A swarm is an apparently disorganized
collection (population) of moving individuals
that tend to cluster together while each individual
seems to be moving in a random direction
Continued
It uses a number of agents (particles) that
constitute a swarm moving around in the search
space looking for the best solution.
Each particle is treated as a point in a D-
dimensional space which adjusts its flying
according to its own flying experience as well as
the flying experience of other particles
Each particle keeps track of its coordinates in the
problem space which are associated with the best
solution (fitness) that has achieved so far. This
value is called pbest.
Continued
Another best value that is tracked by the PSO is
the best value obtained so far by any particle in
the neighbors of the particle. This value is called
gbest.

The PSO concept consists of changing the
velocity(or accelerating) of each particle toward
its pbest and the gbest position at each time step.

Continued
Each particle tries to modify its current position and velocity
according to the distance between its current position and pbest, and
the distance between its current position and gbest.
2
1( ) *( CurrentPosition ) 2( ) *( CurrentPosition
n n
1 1 best,n best,n
) rand v v c rand p c g
n
n
= + +
+
CurrentPosition[n+1] = CurrentPosition[n] + v[n+1]
current position[n+1]: position of particle at n+1th
iteration
current position[n]: position of particle at nth
iteration
v[n+1] : particle velocity at n+1th iteration
v
n+1
: Velocity of particle at n+1 th iteration
V
n
: Velocity of particle at nth iteration
c
1
: acceleration factor related to gbest
c
2
: acceleration factor related to lbest
rand1( ): random number between 0 and 1
rand2( ): random number between 0 and 1
gbest: gbest position of swarm
pbest: pbest position of particle

PSO Algorithm
For each particle
Initialize particle with feasible random number
END
Do
For each particle
Calculate the fitness value
If the fitness value is better than the best fitness value (pbest) in history
Set current value as the new pbest
End
Choose the particle with the best fitness value of all the particles as the gbest
For each particle
Calculate particle velocity according to velocity update equation
Update particle position according to position update equation
End
While maximum iterations or minimum error criteria is not attained
gbest and lbest
global version:
vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] presentx[ ][ ])
+2*rand( )*(pbestx[ ][gbest] presentx[ ][ ])


local version:
vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] presebtx[ ][ ]) +
2*rand( )*(pbestx[ ][lbest] presentx[ ][ ])
Particle Swarm Optimization:
Swarm Topology
In PSO, there have been two basic topologies used in
the literature
Ring Topology (neighborhood of 3)
Star Topology (global neighborhood)
I
4
I
0
I
1
I
2
I
3
I
4
I
0
I
1
I
2
I
3
PSO Parameters:
Velocity
An important parameter in PSO; typically the only
one adjusted
Calmps particles velocities on each dimenson
Determines fineness with which regions are
searched
If too high, can fly past optimal solutions
If too low, can get stuck in local minima
Flow Chart for Extraction Procedure using PSO
Input
No. of Particle
No. of iteration count
Error = 0.05
Initialize Randomly
1. Position of the particle
2. Velocity of the Particle
For each Particle
Simulate Model Parameters
Y11,Y12,Y21,Y22
Evaluate Fitness Function
Is
fitness(Current Pos.)
>
fitness(gbest)
?
Measurement
Data
Update
gbest = current pos.
Yes
No
Is
fitness(Current Pos)
>
fitness(lbest)
?
Update
lbest = current pos.
Yes
No
Is
particle > Max. no.
?
No
go for next
particle
Evaluate Fitness Function E
based on gbest
Is
fitness error < defined error
Extracted
Parameter
Yes
Is
iteration count > Max. count
?
No
Yes
No
Update
vlocity
Limit velocity
[Vmin, Vmax]
Update
position
Limit position
[Pmin, Pmax]
Next Iteration
Comparison of Genetic
Algorithm and PSO
0 2000 4000 6000 8000 10000
0
5
10
20
30
40
50
60
70
Number of Iteration
T
a
r
g
e
t

v
a
l
u
e

o
f

E
r
r
o
r

m
i
n
i
m
i
z
a
t
i
o
n
GA
PSO
Tested in a MATLAB Program, P4 1.7GHz CPU, 256M RAM.
No. of particle/ population size = 100
No. of simulation runs: 10000
Crossover Prob.= 0.9
Mutation Prob. = .01
Model Fitting
Model Fitting
I nductor Optimization
( )
( ) ( )
( ) ( )
( )
, , ,
1 , , ,
arg
, , , 1
arg
max
min
max
min
min min
max
min
2 2
maximize Q n d w s
subject to tol L L n d w s
t et
L n d w s tol L
t et
n n n
d d d
w w d
s s s
d n w s s
s
s +
s s
s s
s s
s s
> +
Constrained PSO Optimization
Thank You