Finding the minimum of the banana function using steepest descent

f( x , y ) := ( 1 x ) + y x dfdx( x , y ) :=
2

<= Expression defining the banana function

2 d f( x , y ) ( 2 ) + 2 x 4 y x x dx <= Expressions defining the terms of the gradient 2 d dfdy( x , y ) := f( x , y ) 2 y 2 x dy

Sx ( x , y ) := dfdx( x , y ) Sy ( x , y ) := dfdy( x , y ) x 0 := 3 y 0 := 5

<= The search direction at any (x,y) point is just the negative of the gradient - the direction of steepest descent <= Initial guess obj := f x 0, y 0 0

obj = 32
0

1 Minimize f , x 0 , y 0 = 1

<= location of minimum as found using the Mathcad minimize function - this is the 'right answer'

f( 1 , 1 ) = 0

Iteration 1

( ) Sy ( x 0 , y 0 ) = 8

Sx x 0 , y 0 = 56

x ( d ) := x 0 + d Sx x 0 , y 0 y ( d ) := y 0 + d Sy x 0 , y 0

( (

) )

<= Define x and y in terms of distance in the search direction, S

F( d ) := f( x ( d ) , y ( d ) )

<= Define the objective function in terms of d so that the multi-dimensional problem is now a one-dimensional problem

13

12 F ( d) 11

10 0.01

0.012

0.014 d

0.016

0.018

dd := .01 d star := Minimize( F , dd)

<= Initial guess for the Mathcad optimizer d star = 0.016 <= d* is the location of the minimum value of the objective function along the line defined by the current search direction x 1 = 2.101 y 1 = 5.128 <= New estimate of x* and y* obj := f x 1 , y 1 obj = 10.126
1

x 1 := x 0 d star dfdx x 0 , y 0 y 1 := y 0 d star dfdy x 0 , y 0 Iteration 2 Sx x 1 , y 1 = 0.204 Sy x 1 , y 1 = 1.427

15

( (

) ) ( (

( (

) )

x ( d ) := x 1 + d Sx x 1 , y 1 y ( d ) := y 1 + d Sy x 1 , y 1

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

F ( d) 10

2 d

dd := .01

<= Initial guess for the Mathcad optimizer d star = 2.172

d star := Minimize( F , dd) x 2 := x 1 d star dfdx x 1 , y 1 y 2 := y 1 d star dfdy x 1 , y 1 Iteration 3 Sx x 2 , y 2 = 10.103 Sy x 2 , y 2 = 1.443

( (

) ) ( (

x 2 = 1.658 y 2 = 2.028

<= New estimate of x* and y* obj := f x 2 , y 2 2

obj = 7.587
2

( (

) )

x ( d ) := x 2 + d Sx x 2 , y 2 y ( d ) := y 2 + d Sy x 2 , y 2

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

15

10 F ( d) 5

0.05

0.1

0.15

0.2 d

0.25

0.3

0.35

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.055

d star := Minimize( F , dd) x 3 := x 2 d star dfdx x 2 , y 2 y 3 := y 2 d star dfdy x 2 , y 2 Iteration 4 Sx x 3 , y 3 = 0.256 Sy x 3 , y 3 = 1.792

( (

) ) ( (

x 3 = 1.101 y 3 = 2.108

<= New estimate of x* and y* obj := f x 3 , y 3 3

obj = 5.216
3

( (

) )

x ( d ) := x 3 + d Sx x 3 , y 3 y ( d ) := y 3 + d Sy x 3 , y 3

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

5 F ( d) 4

0.5

1 d

1.5

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.965

d star := Minimize( F , dd) x 4 := x 3 d star dfdx x 3 , y 3 y 4 := y 3 d star dfdy x 3 , y 3

( (

) )

x 4 = 0.854 y 4 = 0.379

<= New estimate of x* and y* obj := f x 4 , y 4 4

obj = 3.559
4

Iteration 5 Sx x 4 , y 4 = 4.903 Sy x 4 , y 4 = 0.7

4

( (

) )

x ( d ) := x 4 + d Sx x 4 , y 4 y ( d ) := y 4 + d Sy x 4 , y 4

( (

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

F ( d) 2

0.1

0.2 d

0.3

0.4

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.347

d star := Minimize( F , dd) x 5 := x 4 d star dfdx x 4 , y 4 y 5 := y 4 d star dfdy x 4 , y 4

( (

) )

x 5 = 0.848 y 5 = 0.622

<= New estimate of x* and y* obj := f x 5 , y 5 5

obj = 0.033
5

Iteration 6 Sx x 5 , y 5 = 0.028 Sy x 5 , y 5 = 0.195

( (

) )

x ( d ) := x 5 + d Sx x 5 , y 5 y ( d ) := y 5 + d Sy x 5 , y 5

( (

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

F ( d) 0.03

0.025

0.1

0.2 d

0.3

0.4

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.326

d star := Minimize( F , dd) x 6 := x 5 d star dfdx x 5 , y 5 y 6 := y 5 d star dfdy x 5 , y 5

( (

) )

x 6 = 0.839 y 6 = 0.685

<= New estimate of x* and y* obj := f x 6 , y 6 6

obj = 0.026
6

Iteration 7 Sx x 6 , y 6 = 0.26 Sy x 6 , y 6 = 0.037

0.03 0.0283 0.0267 F ( d) 0.025 0.0233 0.0217 0.02 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 d 0.2 0.22 0.24 0.26 0.28 0.3

( (

) )

x ( d ) := x 6 + d Sx x 6 , y 6 y ( d ) := y 6 + d Sy x 6 , y 6

( (

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.143

d star := Minimize( F , dd) x 7 := x 6 d star dfdx x 6 , y 6 y 7 := y 6 d star dfdy x 6 , y 6

( (

) )

x 7 = 0.876 y 7 = 0.691

<= New estimate of x* and y* obj := f x 7 , y 7 7

obj = 0.021
7

Iteration 8 Sx x 7 , y 7 = 0.022 Sy x 7 , y 7 = 0.154

0.022

( (

) )

x ( d ) := x 7 + d Sx x 7 , y 7 y ( d ) := y 7 + d Sy x 7 , y 7

( (

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

0.02 F ( d) 0.018

0.016

0.1

0.2 d

0.3

0.4

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.322

d star := Minimize( F , dd) x 8 := x 7 d star dfdx x 7 , y 7 y 8 := y 7 d star dfdy x 7 , y 7

( (

) )

x 8 = 0.869 y 8 = 0.74

<= New estimate of x* and y* obj := f x 8 , y 8 8

obj = 0.017
8

Iteration 9 Sx x 8 , y 8 = 0.21 Sy x 8 , y 8 = 0.03

( (

) )

x ( d ) := x 8 + d Sx x 8 , y 8 y ( d ) := y 8 + d Sy x 8 , y 8

( (

) )

<= Define x and y in terms of distance in the search direction, S F( d ) := f( x ( d ) , y ( d ) )

0.018 F ( d) 0.016

0.014

0.05

0.1

0.15 d

0.2

0.25

dd := .01

<= Initial guess for the Mathcad optimizer d star = 0.137

d star := Minimize( F , dd) x 9 := x 8 d star dfdx x 8 , y 8 y 9 := y 8 d star dfdy x 8 , y 8 Now, make a convergence plot

( (

) )

x 9 = 0.898 y 9 = 0.744 i := 0 .. 9

<= New estimate of x* and y* obj := f x 9 , y 9 9

obj = 0.014
9

100

10

Iteration Number

obji

0.1

0.01

4 i

Objective Function