Nonlinear Programming

Elimination Method
Fibonacci Method
Method, Golden Section Method
Chapters: 5.7 & 5.8
(E i
(Engineering
i Optimization
O i i i by
b S S Rao)
R )

P K Sahoo, BITS-Hyderabad Campus

Fibonacci Method
This method is used to find the minimum of a
function of one variable.
variable
Method: The method uses the Fibonacci
sequence {Fn} i.e.
F0=1=F1, Fn=Fn-1+Fn-2 , n>1
which yields the sequence
1 1 2 3 5 8 13 21
1,1,2,3,5,8,13,21..

P K Sahoo, BITS-Hyderabad Campus

Limitations of the method:

1. The initial interval of uncertainty,
y, in which the optimum
p
lies, has to be known.
2. The function being optimized has to be unimodal
(Unimodal function is one that has only one peak
(maximum) or valley (minimum) in a given interval) in
the initial interval of uncertainty.
uncertainty
3. The exact optimum cannot be located in this method.
Only an interval known as the final interval of
uncertainty will be known. The final interval of
uncertainty can be made as small as desired by using
more computations.
4. The number of function evaluations to be used in the
searchh or the
th resolution
l ti
required
i d has
h to
t be
b specified
ifi d
beforehand.
P K Sahoo, BITS-Hyderabad Campus

Procedure: Let L0 be the initial interval of

uncertainty
i
d fi d by
defined
b axb andd n be
b the
h
number of experiments to be conducted.
Define
F
L =
L(1)
*
2

n2

Fn

and place the first two experiments at points

x1 and x2, which are located at distance of
f
eachh endd off L0.
L *2 from
This gives
P K Sahoo, BITS-Hyderabad Campus

x1 = a + L*2 = a +

Fn 2
L0
Fn

x 2 = b L*2 = b

Fn 2
L0
Fn

=b

Fn Fn 1
L0
Fn

= b (b a ) +
=a+

Fn 1
L0
Fn

Fn 1
L0
Fn

....(2)

(This shows that

bb-xx2=xx1-a)
a)

L0
a x1

x2

b
P K Sahoo, BITS-Hyderabad Campus

Discard part of the interval by using unimodality

assumption.
assumption
Interval of uncertainty depending on f(x1) < or > f(x2)
after two experiment is
L2 = Length of [ a, x2 ] or [ x1 , b]
= x2 a or b - x1 ,(b x2 = x1 a b x1 = x2 a)
Fn1
L0
= x2 a =
Fn

Fn2
Fn1
*
L2 = L0 L2 = L0 1

= L0
Fn
Fn

and with one experiment left in it.

P K Sahoo, BITS-Hyderabad Campus

This experiment will be at a distance of

Fn2
Fn2 Fn1
Fn2
L =
L0 =
L0 =
L2
Fn
Fn1 Fn
Fn1
*
2

from one end and

Fn2 Fn1 Fn2
Fn3
L2 L = L2 1 =
L2 = L2 from the other end.
Fn1
Fn1
Fn1
*
2

Now place the third experiment in L2 so that the

current two experiments are located at a distance of
Fn3
Fn3 Fn1
Fn3 from each end of the
*
L3 =
L0 =
L0 =
L2
Fn
Fn1 Fn
Fn1 interval L .
2
P K Sahoo, BITS-Hyderabad Campus

Again the unimodality property will allow us to

reduce the interval of uncertainty to L3 given by
Fn3
Fn1 Fn3
L3 = L2 L = L2
L2 =
L2
Fn1
Fn1
*
3

Fn2
Fn2
=
L2 =
L0
Fn1
Fn

This process of discarding a certain interval and

placing
l i
a new experiment
i
t in
i the
th interval
i t
l can be
b
continued, so that the location of the jth experiment
andd the
th interval
i t
l off uncertainty
t i t att the
th end
d off jth
experiments are, given by
P K Sahoo, BITS-Hyderabad Campus

L =
*
j

Lj =

Fn j
Fn ( j 2 )
F n ( j 1)
Fn

L j 1
L0
Ln F1 1
= =
L0 Fn Fn

and for j=n, we have

Th ratio,
The
ti Ln/L0 will
ill permit
it us to
t determine
d t
i n,
the required number of experiments, to achieve
any desired accuracy in locating the optimum
p
point.

P K Sahoo, BITS-Hyderabad Campus

Position of the final experiment:

L*n F0 1
= = n
Ln1 F2 2

Fn j
Lj1
Lj =

2)
n
j

Thus after concluding n-1

n 1 experiments and
discarding the appropriate interval in each step, the
remaining interval will contain one experiment
namely the nth experiment, is also to be placed at the
centre of the present interval of uncertainty.
uncertainty That is,
is
the position of the nth experiment will be same as
that of (n-1)th one,
one and this is true for whatever value
we choose for n.

P K Sahoo, BITS-Hyderabad Campus

10

Since no new information can be gained by

placing the nth experiment exactly at the same
location as that of (n-1)th experiment, we
place
l
the
h nthh experiment
i
very close
l
to the
h
remaining valid experiment. This enables us to
obtain the final interval of uncertainty to
1
within 2 L .
n 1
1

P K Sahoo, BITS-Hyderabad Campus

11

Q. Find the minimum of f (x) = x2 2x, 0 x 1.5 with n = 4

L*2 =

Fn2
F
2
L0 = 2 L0 = 1.5 = 0.6
Fn
F4
5

L0 is
i the
h original
i i l interval
i
l of uncertainty.
i

Thus the positions of the first two experiments are

x1 = a + L*2 = 0.6
x2 = b L = 0.9
09
*
2

f ( x1 ) = 0.84, f ( x2 ) = 0.99

x1 x2
0

15
1.5

-.84 -.99

Since f(x1) > f(x2) we reject [0,x1]

p
is placed
p
at
The third experiment
x3 = b (x2 x1) =1.5 (.9 .6) =1.2
with f (x3) =0.96
w
.9

x1 x2 x3 b
.6
.9 1.2 1.5
-.99 -.96

P K Sahoo, BITS-Hyderabad Campus

12

Since f(x3) > f(x2) we reject [1.2,1.5]

Now the interval of uncertainty is [0.6,1.2]
Note that the point 0.9
0 9 is the middle point i.e.
ie
it is equal distance from both end points. Take
x4 =0.95.
0 95 So
S f(x
f( 4)=-0.9975
) 0 9975
Since f(.9)>f(.95) the new interval of
uncertainty L4 is [.9,1.2].

L4 .3
3 1
= = =.2 <.25
L0 1.5 5
P K Sahoo, BITS-Hyderabad Campus

13

Q. Consider the function

x2 , 0 x 1
f ( x) =
x 2, 1 x 2

Find the minimum within an interval of

uncertainty equal to 0.25
0 25 in the interval [0,2]
[0 2]
by Fibonacci method.
Fn2
F
2
L =
L0 = 2 L0 = 2 = 0.8
Fn
F4
5

1
n=
0.25 Fn 4 n = 4
Fn

*
2

Thus the positions of first two experiments in

L0 are
x1 x2
x1 = a + L*2 = 0.8
x2 = b L*2 = 1.2

f ( x1 ) = 0.64, f ( x2 ) = 0.8

a=0

b=2

-.64 -.8
.8
P K Sahoo, BITS-Hyderabad Campus

14

Since f(x1) > f(x2) we reject [0,x1]

Interval of uncertainty is [.8,2]

[ 8 2]
The third experiment is placed at
x3 = b ( x2 x1 ) = 1.6
with f ( x3 ) = 0.4
04

x1 x2 x3 b
.8 1.2 1.6 2
-.99 -.96

Since f(x
( 3) > f(x
( 2) we reject
j [[x 3,
] [ , ]
3 b]=[1.6,2]

Interval of uncertainty is [.8,1.6]

Note that the point 1.2
1 2 is the middle point i.e.
i e it
is equal distance from both end points. Take x4
=1.25.
1 25 So
S f(x
f( 4)=-0.75
) 0 75
P K Sahoo, BITS-Hyderabad Campus

15

Since f(x4)>f(x2) we reject [1.25,1.6]. The new

i t
interval
l off uncertainty
t i t L4 is
i [.8,1.25].
[ 8 1 25]
L4 .45
= =.225 <.25
L0 2

P K Sahoo, BITS-Hyderabad Campus

16

Golden Section Method

P K Sahoo, BITS-Hyderabad Campus

17

The golden section method is same as the Fibonacci

method except that in the Fibonacci method the total
number of experiments to be conducted has to be
specified before beginning of the calculation,
calculation
whereas this is not required in golden section
method.
method
In the Fibonacci method, the location of the first two
experiments is determined by the total number of
experiments, n. In the golden section method we start
with the assumption that we are going to conduct a
large number of experiments. Of course, the total
number of experiments can be decided during the
computation.
P K Sahoo, BITS-Hyderabad Campus

18

The intervals of uncertainty remaining at the

end
d off different
diff
t number
b off experiments
i
t can be
b
computed as follows:
Fn 1
L2 = lim
L0
n F
n
Fn 2
Fn 2 Fn 1
L3 = lim
L0 = lim
L0
n F
n F
n
n 1 Fn

Fn 1
L0
lim

n
Fn

..........
Fn 1
Lk = lim

n
Fn

k 1

L0

P K Sahoo, BITS-Hyderabad Campus

19

From the relation Fn=Fn-1+Fn-2

we have Fn
Fn 2
Fn 1

= 1+

Fn 1

Fn
Fn 2
lim
= 1 + lim
n F
n F
n 1
n 1
1
1+ ,
r

Fn
= lim
n F
n 1

1 1+ 4
1 = 0 =
= 1.618
2
2

1
Lk =

k 1

L0 = ( 0.618 )

P K Sahoo, BITS-Hyderabad Campus

k 1

L0

20

Procedure: The procedure is same as

Fibonacci method,
method except that the location of
the first two experiments is defined by
Fn2
Fn2 Fn1
L0
L=
L0 =
L0 = 2 = 0.382L0
Fn
Fn1 Fn

*
2

The desired accuracyy can be specified

p
to stop
p
the procedure.

P K Sahoo, BITS-Hyderabad Campus

21

Q. Find the minimum of

by
Fib
i
f (x) = x(x 5 ) on[0,20] with n = 7 Fibonacci
Method

Here

Fn2
F5
8
L =
L0 = L0 = 20 = 7.62
Fn
F7
21
*
2

Thus the positions of the first two experiments are

x1 = a + L*2 = 7.62
x2 = b L*2 = 12.38

x1
a

x2
b

f ( x1 ) = 61.63,, f ( x2 ) = 41.2

Since f(x1) < f(x2) we reject [x2,20] based on the

unimodality assumption and obtain the new interval
of uncertainty L2 =[0,x2]=[0,12.38].
P K Sahoo, BITS-Hyderabad Campus

22

The third experiment is placed at

x3 = a + ( x2 x1 ) = 4.76
4 76
with f ( x3 ) = 52.11
Since f(x1) < f(x3) we reject [0,x3]

Interval of uncertainty L3 is [x3,x

x2]=
[4.76,12.38]
The fourth experiment is placed at

x4 = x3 + ( x2 x1 ) = 4.76 + 4.76 = 9.52

with f ( x4 ) = 58.9
Since f(x1) < f(x4) we reject [x4,x2]
P K Sahoo, BITS-Hyderabad Campus

23

and obtain the new interval of uncertainty as

L4 = [x3,xx4]]=[4
[4.76,9.52]
76 9 52]
The fifth experiment is placed at

x5 = x3 + ( x4 x1 ) = 6.66
with f ( x4 ) = 60.23
Since f(x1) < f(x5) we reject [x3,x 5]
and L5=[x5,x
x4]=[6.66,9.52]
]=[6 66 9 52]
The sixth experiment is placed at

x6 = x5 + ( x4 x1 ) = 8.56
with f ( x6 ) = 61.19
P K Sahoo, BITS-Hyderabad Campus

24

Since f(x1) < f(x6) we reject [x6,x 4]

and L6=[x
[x5,x
x6]]=[6
[6.66,8.56]
66 8 56]
The seventh experiment is placed at

x7 = x5 + ( x6 x1 ) = 7.6
with f ( x7 ) = 61.62
Since f(x1) < f(x7) we reject [x5,x 7]
and L7=[x7,x
x6]=[7.6,8.56]
]=[7 6 8 56]
L7 8.56 7.6
=
= 0.048
0 048
L0
20

P K Sahoo, BITS-Hyderabad Campus

25

Q. Find the minimum of

by Golden
Section
f (x) = x(x 5 ) on[0,20] with n = 7
Method

Here

L*2 = 0.382L0 = 0.382 20 = 7.64

Thus the positions of the first two experiments are

x1 = a + L*2 = 7.64
x2 = b L*2 = 12.36

x1
a

x2
b

f ( x1 ) = 61.639,, f ( x2 ) = 41.38

Since f(x1) < f(x2) we reject [x2,20] based on the

unimodality assumption and obtain the new interval
of uncertainty L2 =[0,x2]=[0,12.36].
P K Sahoo, BITS-Hyderabad Campus

26

The third experiment is placed at

x3 = a + ( x2 x1 ) = 4.72
4 72
with f ( x3 ) = 51.86
Since f(x1) < f(x3) we reject [0,x3]

Interval of uncertainty L3 is [x3,x

x2]=
[4.72,12.36]
The fourth experiment is placed at

x4 = x3 + ( x2 x1 ) = 4.72 + 4.72 = 9.44

with f ( x4 ) = 59.16
Since f(x1) < f(x4) we reject [x4,x2]
P K Sahoo, BITS-Hyderabad Campus

27

and obtain the new interval of uncertainty as

L4 = [x3,xx4]]=[4
[4.72,9.44]
72 9 44]
The fifth experiment is placed at

x5 = x3 + ( x4 x1 ) = 6.52
with f ( x4 ) = 59.9
Since f(x1) < f(x5) we reject [x3,x 5]
and L5=[x5,x
x4]=[6.52,9.44]
]=[6 52 9 44]
The sixth experiment is placed at

x6 = x5 + ( x4 x1 ) = 8.32
with f ( x6 ) = 61.47
P K Sahoo, BITS-Hyderabad Campus

28

Since f(x1) < f(x6) we reject [x6,x 4]

and L6=[x
[x5,x
x6]]=[6
[6.52,8.32]
52 8 32]
The seventh experiment is placed at

x7 = x5 + ( x6 x1 ) = 7.2
with f ( x7 ) = 61.26
Since f(x1) < f(x7) we reject [x5,x 7]
and L7=[x7,x
x6]=[7.2,8.32]
]=[7 2 8 32]
L7 8.32 7.2
=
= 0.056
0 056
L0
20

P K Sahoo, BITS-Hyderabad Campus

29