ECE 307 - Techniques For Engineering Decisions: George Gross

ECE 307 – Techniques for
Engineering Decisions
Dynamic Programming
George Gross
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
© 2006 University of Illinois Board of Trustees, All Rights Reserved

ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DYNAMIC PROGRAMMING
Multi-step problem solving technique
Systematic approach to sequential decision
making
Key characteristic: ability to to separate
problem into stages

1
STAGES AND STATES
We consider the problem to be composed of

multiple stages
A stage is the point in time, space, geographic
location or structural element at which we make
a decision; this point is associated with a
number of states
A state of the system describes a possible
configuration of the system at a given stage
STAGES AND STATES
decision
dn
variable
sn stage n
state
(input)
2
RETURN FUNCTION
A decision dn at the stage n transforms the state
sn at the current stage n into the state sn+1 at
the stage n + 1
The state sn and the decision dn are associated
with the value of the objective; the effect is
measured by the return function denoted by
rn ( s n , d n )
The optimal decision at stage n is the decision
d *n that optimizes the return function for state sn
RETURN FUNCTION
dn decision
variable
sn stage n
state return
rn ( d n, sn)
(input) function
3
ROAD TRIP EXAMPLE
A poor student is traveling from NY to LA
To minimize costs, the student plans to sleep
at friends’ houses each night in cities along the
trip
Based on past experience he can reach
Columbus, Nashville or Louisville after 1
day
Kansas City, Omaha or Dallas after 2 days
San Antonio or Denver after 3 days
LA after 4 days
ROAD TRIP EXAMPLE
680 610
2 Columbus 5 K. City 8 Denver
580
550 515 1030
790
900 6 Omaha
NY 1 3 Nashville 760 10 LA
700 790
770 510 1050 1390
940
660 790
830
4 Louisville 7 Dallas 9 S. Antonio
270
stage 1 stage 2 stage 3 stage 4 stage 5

4
ROAD TRIP
The student wishes to minimize the number of

miles driven and so he wishes to determine the
shortest path between NY and LA
To solve the problem, he works backwards
We adopt the following notation
cij = distance between states i and j
fk( i ) = distance of the shortest path to
LA from state i in stage k
ROAD TRIP EXAMPLE CALCULATIONS
stage 4 : f 4 (8) = 1030 f 4 (9) = 1390
⎧ ⎫
⎪ ⎪
stage 3 : f 3 (5) = min ⎨(610 + 1030),(790 + 1390)⎬ = 1640

⎪⎩ 1640 2180 ⎪⎭
⎧ ⎫
⎪ ⎪
f 3 (6) = min ⎨(540 + 1030),(940 + 1390)⎬ = 1570

⎪⎩ 1570 2330 ⎪⎭
⎧ ⎫
⎪ ⎪
f 3 (7) = min ⎨(790 + 1030),(270 + 1390)⎬ = 1660

⎪⎩ 1820 1660 ⎪⎭
5
ROAD TRIP EXAMPLE CALCULATIONS
stage 2 :
⎧⎪ ⎫⎪
f 2 (2) = min ⎨(680 + 1640) , (790 + 1570) , (1050 + 1660) ⎬ = 2320

⎩⎪ 2320 2360 2710 ⎭⎪
⎧⎪ ⎫⎪
f 2 (3) = min ⎨(580 + 1640) , (760 + 1570) , (660 + 1660)⎬ = 2220

⎩⎪ 2220 2330 2320 ⎭⎪
⎧⎪ ⎫⎪
f 2 (4) = min ⎨(510 + 1640) , (700 + 1570) , (830 + 1660)⎬ = 2150

⎩⎪ 2150 2270 2490 ⎭⎪
stage 1 :
⎧⎪ ⎫⎪
f1 (1) = min ⎨(1550 + 2320) , (900 + 2220) , (770 + 2150) ⎬ = 2870

⎩⎪ *2870* 3120 292 ⎭⎪
ROAD TRIP EXAMPLE
The shortest path is 2,870 miles and corresponds

to the trajectory { (1, 2) , ( 2, 5 ) , ( 5, 8 ) , ( 8, 10 ) } ,
i.e., from New York, the student reaches
Columbus on the first day, Kansas city on the
second day, Denver the third day and then Los
Angeles
Any other trajectory to LA leads to higher costs
and so is, by definition, suboptimal
6
WORKING BACKWARDS: PICK UP
MATCHES GAME
There are 30 matches on a table and 2 players
Each player can pick up 1, 2, or 3 matches and
continue until the last match is picked up
The loser is the person who picks up the last
match
How can the player P1 who goes first ensure a
win?

MATCHES GAME
We solve this problem by reasoning in a back-
wards fashion so as to ensure that when a
single match remains, P2 has the turn
Consider the situation where 5 matches remain
and it is P2’s turn; for P1 to win we, consider all
possible situations:
7
MATCHES GAME
1 ⇒ 4 left P1 removes 3
P2’s move is to pick 2 ⇒ 3 left P1 removes 2
3 ⇒ 2 left P1 removes 1
We can reason similarly for the cases of 9, 13,
17, 21, 25, and 29 matches
Therefore, P1 wins if P1 picks 30 – 29 = 1 match
in the first move
In this manner, we can assure a win for any
arbitrary number of matches in the game
OIL TRANSPORT TECHNOLOGY
We consider the development of a transport

network from north slope of Alaska to one of 6
possible shipping points in the U.S
The network must meet the problem feasibility
requirements
7 pumping stations from a north slope
ground storage plant to a shipping port
use of only those paths that are physically
and environmentally feasible
8
intermediate
region
oil
substations
storage final
destinations
Objective: determine a feasible pumping
configuration that minimizes the
construction costs of branches

total
= of network of the feasible
costs
pumping configuration
9
Possible approaches to solving such a problem
include:
enumeration: exhaustive evaluation of all
possible paths; too costly since there are
more than 100 possible paths
myopic decision rule: at each node, pick
as the next mode the one reachable by the
cheapest path (in case of ties the pick is
arbitrary) ; for example,
0 3 11 15 19 26 29 31 36
oil I-E II-E III-D IV-E V-D VI-D VII-C B
storage
but such a path is not unique and not
guaranteed to be optimal
serial dynamic programming (DP) : we
need to construct the problem solution by
defining the stages, states and decisions
10
DP SOLUTION
We define a stage to represent each pumping
region and so each stage corresponds to the
set of vertical nodes I, II, . . . , VII
We use backwards recursion: start from a final
destination and work backwards to the oil stage
We define a state to denote a particular
pumping station ↔ sk
A decision refers to the selection of the branch
from each state sk , so there are at most three
choices for a decision dk :
L ↔ left F ↔ forward R ↔ right
OIL TRANSPORT TECHNOLOGY :

DP SOLUTION
The return function is defined as the costs
associated with the decision at the state
dk ↔ rk ( sk , d k )
The transition function is the total costs in
proceeding from one stage to another
We proceed to solve the problem by moving
backwards from each final state to the states in
the previous stage
11
DP SOLUTION:
STAGE 1 REGION VII
optimal decision return
d1
s1 d *1 f *1 ( s1 )
R L F
A 7 R 7
B 6 3 F 3
C 7 5 6 L 5
D 6 5 3 F 3
E 7 8 5 F 5
F 4 2 6 L 2
DP SOLUTION
STAGE 2 REGION VI
optimal decision
cumulative cost in proceeding
from s2 to a final destination
d2
s2 d 2* f 2* ( s2 )
R L F
A 10 12 R 10
B 9 12 7 F 7
C 5 6 7 R 5
D 8 7 6 F 6
E 7 6 11 L 6
12
OIL TRANSPORT : STAGE 2
⎛ ⎞
f ( s2 ) = min r2 ( s2 , d 2 ) + f 1 ( s1 ) ⎟
* ⎜ *
d2 ⎜
⎟
2
⎝ ⎠
a function of only s1
⇓
for given d 2 , s1 is set
DP SOLUTION:
STAGE 3 REGION V
{
f 3* ( s3 ) = min r3 ( s3 , d 3 ) + f 2* ( s2 )
d3
}
d3
s3 d *3 f *3 ( s3 )
R L F
A 14 16 R 14
B 14 17 15 R 14
C 10 5 6 R 10
D 9 12 9 R, F 9
E 12 15 L 12
13
DP SOLUTION:
STAGE 4 REGION IV
{
f 4* ( s4 ) = min r4 ( s4 , d 4 ) + f 3* ( s3 )
d4
}
d4
s4 d 4* f 4* ( s4 )
R L F
B 17 18 23 R 17
C 15 22 16 R 15
D 18 17 16 F 16
E 16 21 L 16
DP SOLUTION:
STAGE 5 REGION III
{
f 5* ( s5 ) = min r5 ( s5 , d 5 ) + f 4* ( s4 )
d5
}
d5
s5 d *5 f *5 ( s5 )
R L F
A 19 R 19
B 18 18 R,F 18
C 24 23 17 F 17
D 20 19 25 L 19
E 21 17 F 17
F 20 L 20
14
DP SOLUTION:
STAGE 6 REGION II
{
f 6* ( s6 ) = min r6 ( s6 , d 6 ) + f 5* ( s5 )
d6
}
d6
s6 d *6 f 6* ( s6 )
R L F
A 25 R 19
B 21 25 18 R,F 18
C 28 21 23 F 17
D 27 26 29 L 19
E 26 23 22 F 17
F 18 23 L 20
DP SOLUTION:
STAGE 7 REGION I
{
f 7* ( s7 ) = min r7 ( s7 , d 7 ) + f 6* ( s6 )
d7
}
d7
s7 d *7 f 7* ( s7 )
R L F
A 27 32 R 27
B 26 33 26 R,F 26
C 34 25 27 L 25
D 25 27 33 R 25
E 27 35 30 R 27
15
THE OPTIMAL TRAJECTORY
For the last stage

I-A I-B I-C I-D I-E
f 8* ( s8 ) = min{27 + 6, 26 + 4, 25 + 7, 25 + 8, 27 + 3 }
= 30
To find trajectory we retrace forwards through
the stages 7, 6, . . . , 1 and obtain

THE OPTIMAL TRAJECTORY
d8=L d7=R d6=F d5=R d4=R d3=R d2=F
oil
storage
I II III VI V VI VII
B A A B C D D
In addition to this optimal solution, others are
possible since the path need not be unique

16
OIL TRANSPORT PROBLEM
SOLUTION
We obtain the diagram shown on the next slide
by retracing the steps at each stage
The solution
provides all the optimal solutions
is based on logically breaking up the problem
into stages with computation in each stage
being a function of the number of states in the
stage
provides all the suboptimal paths

SOLUTION final destinations
3 3
6 3
9 6
oil substations
storage
17
SOLUTION
For example, we may calculate the least cost
optimal path to any sub optimal shipping point
different than D
From the solution, we can also determine the
sub optimal path if the construction of a feasible
path is not undertaken

OIL TRANSPORT : SENSITIVITY CASE

Consider the case where we got to stage VI but
the arc VI – D to VII – D cannot be built due to
some environmental intervention
We determine, then, the least-cost path from VI –
D to find the final destination D whose value is 9
instead of 6
2 7 final
VI - D VII - C destination
D
and so the sub optimal cost solution costs are 33
18
FACILITIES SELECTION
A company is expanding to meet a wider market
and considers:
3 location alternatives
4 different building types (sizes) at each site
Revenues and costs vary with each location and
building type
Revenues R increase monotonically with
building size; these are net revenues or profits
Costs C increase monotonically with building
size
The data for building sizes, revenues and costs
are given in the table

19
building size
B1 B2 B3 B4 none
site
R1 C1 R2 C2 R3 C3 R4 C4 R0 C0
I 0.50 1 0.65 2 0.8 3 1.4 5 0 0
II 0.62 2 0.8 5 0.96 6 1.8 8 0 0
III 0.71 4 1.2 7 1.6 9 2 11 0 0
THE FACILITIES SELECTION PROBLEM
The company can afford to invest at most 21
million $ in the total expansion project
Determine the optimal expansion policy, i.e., the
buildings to be built at each site

20
DP SOLUTION APPROACH
We use the DP approach to solve this problem;
first, however, we need to define the DP
parameters
For the facilities siting problem we realize that
without the choice of a site, the building type is
irrelevant and so the elements that control the
entire decision process are building sites

stage ↔ site
amount of funds available
state ↔
for construction
decision ↔ building type
profits attained, i.e., the
return function ↔
revenues
impact of a decision on the
transition function ↔
availablity of resources
21
As we move from stage n backwards to stage n-1
after a decision dn is taken the funds available
for construction are
sn−1 = sn − cn ← costs of decision d n
The recursion relation is
{
f n* ( sn ) = max f n ( sn , d n ) + f n−*1 ( sn−1 )
dn
} n = 1,2, 3
with
s n − 1 = sn − c n
and
f n ( sn , d n ) = rn ( sn , d n ) = Rn← profits of decision d n
We use backwards DP and start with site I, a
purely arbitrary choice, as stage 1, where this
stage 1 represents the last decision in the 3 -
stage sequence
The amount of funds available is unknown since

the decision at sites II and III are assumed to
have been previously made, but, clearly,
0 ≤ s1 ≤ 21
22
DP SOLUTION : STAGE 1 SITE I
f1 ( s1 ) = max { r1 ( s1 , d1 )}

0 ≤ d1 ≤ 4 R1
s1 0 1 2 3 4 d 1* f 1* ( s1 )
21 ≥ s1 ≥ 5 0 .5 .65 .80 1.4 4 1.4
4 ≥ s1 ≥ 3 0 .5 .65 .80 3 .80
2 0 .5 .65 2 .65
1 0 .5 1 .50
0 0 0 0 0 0 0 0
DP SOLUTION : STAGE 2 SITE II
The amount of funds s2 available is unknown
since the decision at site III is assumed to have
been already made
The value of d2 is a function of s2 and we
construct a decision table using
f 2* ( s2 ) = max { r2 ( s2 , d 2 ) + f 1* ( s1 )}

0 ≤ d2 ≤ 4 R2
23
DP SOLUTION : STAGE 2 SITE II
s2 0 d2 1 2 3 4 dd **22 ff**22(S
( s 22))
21 ≥ s ≥ 13 1.40
2 2.02 2.18 2.36 3.20 4 3.20
12 1.40 2.02 2.18 2.36 2.60 4 2.60
11 1.40 2.02 2.18 2.36 2.60 4 2.60
10 1.40 2.02 2.18 1.76 2.45 4 2.45
9 1.40 2.02 1.58 1.61 2.30 4 2.30
8 1.40 2.02 1.58 1.61 1.80 1 2.02
7 1.40 2.02 1.43 1.61 1 2.02
6 1.40 1.42 1.28 1.46 3 1.46
5 1.40 1.42 0.78 0.96 1 1.42
4 0.80 1.27 1 1.27
3 0.80 1.12 1 1.12
2 0.65 0.62 0 0.65
1 0.50 0 0.50
0 0.00 0 0.00
SAMPLE CALCULATIONS
Consider the case s2 = 10 and d2 = 0 :
C2 = 0 and R2 = 0
then, s1 = 10 and d 1* = 4 so that
f 1* ( s1 ) = 1.4
consequently, f2(s2) = 1.4
Consider the case s2 = 10 and d2 = 4 :
C2 = 8 and R2 = 1.8
then, s1 = 2 and d 1* = 2 so that
f 1* ( s1 ) = .65
consequently, f2(s2) = 2.45
We can, then, show that
f 2* ( s2 ) = 2.45
24
DP SOLUTION : STAGE 3 SITE III
At stage 3 , the first actual decision is made and
so exactly 21 million is available and s3 = 21
We compute the elements in the table using
f 3* ( s3 ) = max{ r3 ( s3 , d 3 ) + f *2 ( s2 )}
d3 N
where R3
s2 = s 3 − C 3
d3
d 3* f *3 ( s3 )
s3 0 1 2 3 4
21 3.2 3.91 4.4 4.1 4.45 4 4.45
OPTIMAL SOLUTION
Optimal profits are 4.45 million
The optimal decision is obtained by retracing
steps from stage 3 to stage 1:
d *3 = 4 ↔ construct B2 at site III
s2 = s3 – C3 = 21 – 11 = 10
d *2 = 4 ↔ construct B4 at site II
s1 = s2 – C2 = 10 – 8 = 2
d 1* = 2 ↔ construct B2 at site 1
C1 = 5 and C1+C2+C3 = 21
25
SENSITIVITY CASE
We next consider the case where the maximum
investment available is 15 million
By inspection, the results in stages 1 and 2
remain unchanged; however, we must recom-
pute stage 3 results with the 15 million limit
d3
s3 d 3* f *3 ( s3 )
0 1 2 3 4
15 3.2 3.31 3.22 3.06 3.27 1 3.31
SENSITIVITY CASE
The optimal solution for this sensitivity case
obtains maximum profits of 3.31 million and the
decision is
d *3 = 1 ↔ construct B1 at site III
s 2 = s3 − C 3 = 15 − 4 = 11
d *2 = 4 ↔ construct B4 at site II
s1 = s2 − C 2 = 11 − 8 = 3
d 1* = 3 ↔ construct B3 at site I
C1 = 3 and C1 + C 2 + C 3 = 15
26
OPTIMAL CUTTING STOCK PROBLEM
A paper company gets an order for:
8 rolls of 2 ft paper at 2.50 $/roll
6 rolls of 2.5 ft paper at 3.10 $/roll
The company only has 13 ft of paper to fill these

orders; partial orders can be filled
Determine how to fill orders to maximize profits

A stage is an order and since there are 4 orders
we construct a 4 – stage DP
d4 d3 d2 d1
s4 stage s3 stage s2 stage s1 stage

4 3 2 1
r4 r2 r3
r1
A state in stage n is the remaining ft of paper
left for the order being processed at stage n and
all the remaining stages
27
A decision in stage n is the amount of rolls to
produce in stage n :
⎡F ⎤ F
d n = ⎢ 0 ⎥ , the largest integer in 0
⎣ Ln ⎦ Ln
where
Ln = length of order n ( ft )
F0 = length of available paper ( ft )

The return function at stage n is the additional
revenues gained from producing dn rolls
The transition function measures amount of
paper remaining at stage n
sn − 1 = sn − d n Ln n = 2,3,4
s0 = s1 − d1 L1
and s0 should be as close as possible to 0
⎡s ⎤
Clearly, d1 = ⎢ 1 ⎥
⎣ L1 ⎦
28
The recursion relation is
f n* ( sn ) = max {r ( s , d
n n n ) + f n−*1 ( sn−1 ) }
⎡ ⎤
0 ≤ dn ≤ ⎢ sn ⎥
L
⎣ n⎦
n = 1, … , 4
where
sn−1 = sn − d n Ln
and
f 0* ( s0 ) = 0
f n ( sn , d n ) = rnd n + f n−*1 ( sn − d n Ln )
Arbitrary order of the stages and we pick
stage n 1 2 3 4
length of
2.5 4 3 2
order ( ft )
We proceed backwards from stage 1 to stage 4
and we know that

29
DP SOLUTION : STAGE 1
f 1* ( s1 ) = max
0 ≤ d1 ≤ 5
{r1 ( s1 , d1 )} = max {3.10 d1 }
0 ≤ d1 ≤ 5
⎡ 13 ⎤
d1 ≤ ⎢ = 5
⎣ 2.5 ⎥⎦
s1
f *1 ( s 1 )
DP : SOLUTION STAGE 2
f *2 ( s2 ) = max
0 ≤ d2 ≤ 3
{5.25 d 2 + f 1* ( s2 − 4 d 2 ) }
⎡ 13 ⎤
d2 ≤ ⎢ ⎥ = 3
⎣4⎦
s2
f *2 ( s 2 )
30
f *3 ( s3 ) = max
0 ≤ d3 ≤ 4
{4.40 d 3 + f *2 ( s3 − 3 d 3 ) }
⎡ 13 ⎤
d3 ≤ ⎢ ⎥ = 4
⎣3⎦
s2
f *3 ( s 3 )
f *4 ( s4 ) = max
0 ≤ d4 ≤ 6
{ 2.5 d 4 + f *3 ( s4 − 2 d 4 ) }
⎡ 13 ⎤
d4 ≤ ⎢ ⎥ = 6
⎣2⎦
*
d4 0 1 2 3 4 5 6 d 4* f 4 ( s4 )
s4 = 13 18.45 17.5 18.2 17.15 16.2 16.9 15 0 18.45
31
DP OPTIMAL SOLUTION
The maximum profits are $18.45
The optimal solution is obtained by retracing
f 4* ( s4 = 13) = 18.45 with d 4* = 0 ↔ no rolls of 2 ft
f 3* ( s3 = 13) = 18.45 with d 3* = 3 ↔ 3 rolls of 3 ft
f 2* ( s2 = 4) = 5.25 with d 2* = 1 ↔ 1 roll of 4 ft
f 1* ( s1 = 0 ) = 0 with d 1* = 0 ↔ no rolls of 2.5 ft
SENSITIVITY CASE
Consider the case that due to an incorrect
measurement, in truth, there are only 11 ft
available for the rolls
We note that the solution for the original 13 ft
covers this possibility in the stages 1, 2 and 3 but
we need to recompute the results of stage 4,
which we now call stage 4′

32
SENSITIVITY CASE : STAGE 4′
The stage 4′ computations become
⎡ 11 ⎤
d 4′ ≤ ⎢ ⎥ = 5
⎣2⎦
d 4′ 0 1 2 3 4 5 d 4*′ f 4*′ ( s4 )
s4 = 11 15 15.7 14.65 13.7 14.4 12.5 1 15.7
SENSITIVITY CASE OPTIMUM
The optimal profits in this sensitivity case are
$15.7
d 4*′ = 1, ↔ 1 roll of 2 ft
d 3*′ = 3 ↔ 3 rolls of 3 ft
d 2′ = d 1′ = 0 ↔ no rolls of 4 ft
* *
↔ no rolls of 2.5 ft
33
ANOTHER SENSITIVITY CASE
We consider the case with the initial 13 ft, but in
addition we get the constraint that at least 1 roll
of 2 ft must be produced:
d4 ≥ 1
Note that no additional work is needed since the

computations in the first tables have all the
necessary data
This sensitivity case optimum profits are $18.2
The optimum solution is :
OPTIMAL CUTTING STOCK PROBLEM
f 4*′′ ( s4 = 13) = 18.2 with d 4*′′ = 2 ↔ 2 rolls of 2 ft

f 3*′′ ( s3 = 9) = 13.2 with d 3*′′ = 3 ↔ 3 rolls of 3 ft
and since s2 = s1 = 0
d 2*′′ = 0 ↔ 0 rolls of 4 ft
d 1*′′ = 0 ↔ 0 rolls of 2.5 ft
The constraint reduces optimum from $ 18.45 to
$18.2 and so it costs $ .25
34
INVENTORY CONTROL PROBLEM
This problem is concerned with the development
of an optimal ordering policy for a retailer
The sales of a seasonal item has the demands
month Oct Nov Dec Jan Feb Mar
demand 40 60 30 40 30 20
All units sold are purchased from a vendor at
4 $/unit `; units are sold in lots of 10, 20, 30, 40 or
50 with the corresponding discount
lot size 10 20 30 40 50
discount
4 5 10 20 25
%
35
There are ordering costs: each order incurs
fixed costs of $ 2 and $ 8 for shipping, handling
and insurance
The storage limitations of the retailer require
that no more than 40 units be in inventory at the
end of the month and the storage charges are
0.2 $/unit; inventory at time 0 and inventory at the
e.o.p. 6 are 0
Assumption: demand occurs at a constant rate
throughout the month
We formulate the problem as a DP and use a
backward process
Each stage corresponds to a month
month Oct Nov Dec Jan Feb Mar
stage
6 5 4 3 2 1
n
36
d6 d5 d4 d3 d2 d1
s6 stage s5 stage s4 stage s3 stage s2 stage s1 stage s0

6 5 4 3 2 1
r6 r5 r4 r3 r2 r1
The state variable at stage n is defined as the
amount entering inventory given that there are
n months remaining – the present month n
plus months n – 1 , n – 2 , . . . , 1
The decision variable at stage n is the amount of
units ordered to satisfy demands for the n
months
The transition function is defined by
sn-1 = sn + dn – Dn n = 1, 2, . . . , 6
s0 = 0 s6 = 0 demand in month n
37
The return function at stage n is given by
rn ( d n , sn ) = φ ( d n ) + hn ( sn + d n − Dn )

0.2( sn + dn − Dn )
with storage costs
φ ( dn ) = 10
+ [1 − ρ ( d n ) ] d n

fixed discount
costs factor
d n = 10,20,30,40,50
dn 0 10 20 30 40 50
φ (d n ) 0 48 86 118 138 160
In the DP approach, at each stage we minimize
{
f n* ( sn ) = min φ (d n ) + hn ⎡⎣ sn + d n − Dn ⎤⎦ + f n−*1 ( sn−1 )
dn
}
n = 1,...,6
s0 = 0 and so f 0* ( s0 ) = 0
38
s0 = 0 ⎫
⎬ ⇒ s1 = 20, 10 or 0 ⇒ d *1 = 0 , 10 or 20
D1 = 20 ⎭
f 1* ( s1 ) = min{ φ (d1 ) + 0 } = φ (d1* )

d1
s1 20 10 0
d *1 0 10 20
φ ( d *1 ) 0 48 86
D2 = 30 and s1 = s2 + d 2 − 30
{
f *2 ( s2 ) = min φ (d 2 ) + 0.2 ⎡⎣ s2 + d 2 − 30⎤⎦ + φ (d *1 )
d2
}
d2
d *2 f 2* ( s2 )
s2 0 10 20 30 40 50
0 204 188 164 50 164
10 172 168 142 40 142
20 134 136 122 122 30 122
30 86 98 90 0 86
40 50 52 0 50
39
D3 = 40 and s2 = s3 + d 3 − 40
*
3
d3
{
f ( s3 ) = min φ (d 3 ) + 0.2 ⎡⎣ s3 + d 3 − 40⎤⎦ + f *2 ( s2 ) }
d3
d *3 f 3* ( s3 )
s3 0 10 20 30 40 50
0 302 304 40 302
10 282 282 286 30, 40 282
20 250 262 264 252 20 250
30 212 230 244 230 218 10 218
40 164 192 212 210 196 0 164
D4 = 30 and s3 = s4 + d 4 − 30
{
f *4 ( s4 ) = min φ (d 4 ) + 0.2 ⎡⎣ s4 + d 4 − 30⎤⎦ + f *3 ( s3 )
d4
}
d4
d 4* f 4* ( s4 )
s4 0 10 20 30 40 50
0 420 422 414 50 414
10 388 402 392 384 50 384
20 350 370 372 362 332 50 332
30 302 332 340 342 210 0 302
40 284 302 310 290 0 284
40
D5 = 20 and s4 = s5 + d 5 − 20
{
f 5* ( s5 ) = min φ (d 5 ) + 0.2 ⎡⎣ s5 + d 5 − 20⎤⎦ + f *5 ( s5 )
d5
}
d5
d 5* f *5 ( s5 )
s5 0 10 20 30 40 50
0 500 504 474 468 50 468
10 462 472 454 446 452 40 446
20 414 434 422 426 430 0 414
30 286 384 394 410 10 384
40 336 356 378 0 336
n=6 D6 = 40 and s6 = 0
s5 = s6 + d 6 − 40 = d 6 − 40
{
f *6 ( s6 ) = min φ (d 6 ) + 0.2 ⎡⎣ s6 + d 6 − 40⎤⎦ + f *5 ( s5 )
d
}
6
d6 0 10 20 30 40 50 d *6 f 6* ( s6 )
f6(s6) 606 608 40 606
d *6 = 40 ⇒ d *5 = 50 ⇒ d *4 = 0 ⇒ d *3 = 40 ⇒ d *2 = 50 ⇒ d 1* = 0
41
OPTIMAL SOLUTION
d *6 = 40 which corresponds to s5 = 0 and costs 606
d 1* = 0 with costs 0
OPTIMAL SOLUTION
d *6 = 40 d *5 = 50 d *4 = 0 d *3 = 40 d *2 = 50 d *1 = 0
s6 = 0 s5 = 0 s4= 30 s3= 0 s2= 0 s1 = 20 s0=0
stage stage stage stage stage stage
6 5 4 3 2 1
r6 (40,0) r4 (0,30) r2 (50,0)

= 138 r5 (50,0) =0 r3 (40,0) = 164 r1 (0,20)
= 166 = 138 =0
42
OPTIMAL SOLUTION
optimal trajectory is
s0 = 0 s1 = 20 s2 = 0 s3 = 0 s4 = 30 s5 = 0
with total costs
0 + 164 + 138 + 0 + 166 + 138 = 606

MUTUAL FUND INVESTMENT

STRATEGIES
We consider a 5-year investment of
10 k$ invested in year 1
1 k$ invested in each year 2, 3, 4 and 5
into 2 mutual funds with different yields for

both the short-term (1 year) and the long-term
(up to 5 years)
A decision at the beginning of each year is the

allocation of investment in each fund
43
STRATEGIES
We operate under the protocol that
once invested, the money cannot be
withdrawn until the end of the 5-year horizon
all short term gains may be reinvested in
either of the two funds or withdrawn in which
case the withdrawn funds earns no more
further interest
The objective is to maximize the total returns at
the end of 5 years

STRATEGIES
The earnings on the investment are
LTD : the long-term dividend specified as

% return / year on the accumulated capital
STD : the short-term interest dividend is the

cash returned to the investor at the end of
the period; cash may be reinvested and any
money not invested in one of the funds earns
nothing
44
STRATEGIES
STD rate in for year n

LTD
fund
1 2 3 4 5 rate I
A 0.02 0.0225 0.0225 0.025 0.025 0.04
B 0.06 0.0475 0.05 0.04 0.04 0.03
We use backwards DP to solve the problem
The stages are the 5 investment periods
stage n Δ year 6 − n n = 1,2,3,4,5

r5 r4 r3 r2 r1
s5 stage s4 stage s3 stage s2 stage s1 stage s0

5 4 3 2 1
d5 d4 d3 d2 d1
45
DP SOLUTION METHOD
For stage n , the state sn is the amount of capital
available for investment in the year 6 – n
The decision dn is the amount of capital

invested in fund A in year 6 – n ; the amount of
capital invested in fund B in the year 6 – n is
therefore sn – dn
In each year, we need to determine the amount

to invest in each fund
DP SOLUTION METHOD
The use of backward recursion considers year 5
first and each of the previous years sequentially
Basic considerations:
for each year 6 – n
dn is invested in fund A returns d n i A ( SDT )
( sn − d n )is invested in fund B returns ( sn − d n )i B ( SDT )
for the year 6 – n + 1
sn−1 = d n i A + ( sn − d n ) iB + 1000 n = 2, 3,4,5
s5 = 10,000
46
THE OBJECTIVE
The objective is to maximize the total returns
5
max R = ∑r
n =1
n
We express all returns in end of the year 5

dollars : rn is the future value of long term
earnings in the years 1, 2, 3 and 4
rn = (1 + I A )n d n + (1 + I B )n ( sn − d n ) n = 1, ... ,5
But for n = 1, r1 is the present value of all earn-
ings in stage 1
r1 = (1 + I A ) d1 + (1 + I B ) ( s1 − d1 ) + i A d1 + i B ( s1 − d1 )
For stage 1
r1
s1
d1
where
r1 = (1 + I A )d1 + (1 + I B )( s1 − d1 ) + i Ad1 + i B( s1 − d1 )
= ( I A + i A − I B − iB ) d1 + (1 + I B + i B ) s1
47
r1 = earnings in stage 1 (returns realized at
the end of 5 years)
⎧ d1 ( I A + i A − I B − i B ) + ⎫
f 1* ( s1 ) = max { r1 } = max ⎨ ⎬
d1 d1 ⎩ s1 (1 + I B + i B ) ⎭
⎧ d ( 0.04 + 0.025 − 0.03 − 0.04) + ⎫
= max ⎨ 1 ⎬
0 ≤ d1 ≤ s1 ⎩ s1 (1 + 0.03 + 0.04) ⎭
= max {d1 ( − 0.005) + s1 (1.07)} maximum
optimal d1
return in
decision stage 1
d *1 = 0 f 1* ( s1 ) = 1.07 s1
r2 = returns realized at the end of 5 years due to
the decision in stage 2
= d 2 (1 + I A )2 + ( s2 − d 2 )(1 + I B )2
= d 2 ⎡⎣(1 + I A )2 − (1 + I B )2 ⎤⎦ + s2 (1 + I B )2
s1 = s2 iB + d 2 ( i A − iB ) + 1,000
48
We select d 2* to maximize
f 2* ( s2 ) = max
d2
{r 2 + f 1* ( s1 ) }
= max
d2
{d 2 (1.042 − 1.032 ) + s2 (1.03)2 + f 1* ( s1 ) }
⎧ d 2 ( .0207) + 1.0609 s2 + ⎫
= max ⎨ ⎬
0 ≤ d2 ≤ s2 ⎩1.07[ .04 s2 + d 2 ( − .015) + 1,000]⎭
= max
d2
{d 2 (.0046) + 1.1037 s2 + 1070}
d 2* = s2 f 2* ( s2 ) = 1.108 s2 + 1070
the decision d3
= d 3 (1 + I A )3 + ( s3 − d 3 )(1 + I B )3
= d 3 ⎡⎣(1 + I A )3 − (1 + I B )3 ⎤⎦ + s3 (1 + I B )3
s2 = s3 iB + d 3 ( i A − iB ) + 1,000
49
f 3* ( s3 ) = max
d3
{r 3 + f 2* ( s2 ) }
⎧ d 3 (1.043 − 1.033 ) + s3 (1.03)3 + ⎫
= max ⎨ ⎬
d3
⎩1.108 s2 + 1,070 ⎭
= max {2,178 + 1.1481s3 + .0018d 3 }
0 ≤ d3 ≤ s3
d 3* = s3 f 3* ( s3 ) = 1.15 s3 + 2,178

the decision d4
= d 4 (1 + I A )4 + ( s4 − d 4 )(1 + I B )4
= d 4 ⎡⎣(1 + I A )4 − (1 + I B )4 ⎤⎦ + s4 (1 + I B )4
s3 = s4 i B + d 4 ( i A − iB ) + 1,000
50
f 4* ( s4 ) = max
d4
{r + f
4
*
3 ( s3 )}
= max
d4
{d 4 (1.044 − 1.034 ) + s4 (1.03)4 + 1.15 s3 + 2,178 }
= max
0 ≤ d4 ≤ s4
{3328 + 1.1772 s4 + .0156d 4}
d 4* = s4 f 4* ( s4 ) = 1.193 s4 + 3,328
the decision d5
= d 5 (1 + I A )5 + ( s5 − d 5 )(1 + I B )5
= d 5 ⎡⎣1.045 − 1.035 ⎤⎦ + s5 (1.03)5
s5 = 10,000 ← capital available for investment
s4 = s5 i B + d 5 ( i A − iB ) + 1,000
= 10,000 iB + d 5 ( i A − iB ) + 1,000
51
⎧ ⎫
⎪ ⎪
f 5 ( s5 ) = max ⎨10,000(1.03) + d 5 (1.04 − 1.03 ) + f 4 ( s4 ) ⎬
* 5 5 5 *
0 ≤ d5 ≤ s4 ⎪
⎪⎭
⎩ 11,593 0.0574
⎡⎣1,000 + 600 + d 5 ( −.04)⎤⎦ 1.193 + 3,328
⎧ (.0574 − 0.048) ⎫
= max ⎨16,830 + d 5 ⎬
0 ≤ d5 ≤ s5 ⎩ 0.097 ⎭
= 16,830 + 0.097(10,000)
d 5* = 10,000 f 5* ( s5 ) = 16,927
OPTIMAL SOLUTION
optimal return at end of 5 years is 16,927 using
the following strategy
beginning investment in
of year fund A fund B
1 10,000 0
2 STD returns + 1,000 0
5 0 STD returns + 1,000
52

ECE 307 - Techniques For Engineering Decisions: George Gross

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

ECE 307 - Techniques For Engineering Decisions: George Gross

Caricato da

Copyright:

Formati disponibili

ECE 307 – Techniques for

© 2006 University of Illinois Board of Trustees, All Rights Reserved

 Multi-step problem solving technique

 Systematic approach to sequential decision

 Key characteristic: ability to to separate

problem into stages

 We consider the problem to be composed of

STAGES AND STATES

ROAD TRIP EXAMPLE

stage 1 stage 2 stage 3 stage 4 stage 5

 The student wishes to minimize the number of

ROAD TRIP EXAMPLE CALCULATIONS

stage 4 : f 4 (8) = 1030 f 4 (9) = 1390

ROAD TRIP EXAMPLE

 The shortest path is 2,870 miles and corresponds

 Each player can pick up 1, 2, or 3 matches and

continue until the last match is picked up

 The loser is the person who picks up the last

 How can the player P1 who goes first ensure a

WORKING BACKWARDS: PICK UP

 We solve this problem by reasoning in a back-

wards fashion so as to ensure that when a

single match remains, P2 has the turn

 Consider the situation where 5 matches remain

and it is P2’s turn; for P1 to win we, consider all

OIL TRANSPORT TECHNOLOGY

 We consider the development of a transport

OIL TRANSPORT TECHNOLOGY

 Objective: determine a feasible pumping

configuration that minimizes the

construction costs of branches

OIL TRANSPORT TECHNOLOGY

OIL TRANSPORT TECHNOLOGY :

 For the last stage

 To find trajectory we retrace forwards through

the stages 7, 6, . . . , 1 and obtain

THE OPTIMAL TRAJECTORY

d8=L d7=R d6=F d5=R d4=R d3=R d2=F

 In addition to this optimal solution, others are

possible since the path need not be unique

OIL TRANSPORT PROBLEM

 For example, we may calculate the least cost

optimal path to any sub optimal shipping point

 From the solution, we can also determine the

sub optimal path if the construction of a feasible

path is not undertaken

OIL TRANSPORT : SENSITIVITY CASE

 A company is expanding to meet a wider market

 4 different building types (sizes) at each site

 Revenues and costs vary with each location and

 Revenues R increase monotonically with

building size; these are net revenues or profits

 Costs C increase monotonically with building

 The data for building sizes, revenues and costs

are given in the table

I 0.50 1 0.65 2 0.8 3 1.4 5 0 0

II 0.62 2 0.8 5 0.96 6 1.8 8 0 0

III 0.71 4 1.2 7 1.6 9 2 11 0 0

THE FACILITIES SELECTION PROBLEM

 The company can afford to invest at most 21

million $ in the total expansion project

 Determine the optimal expansion policy, i.e., the

buildings to be built at each site

Multi-step problem solving technique

Systematic approach to sequential decision

Key characteristic: ability to to separate

We consider the problem to be composed of

The student wishes to minimize the number of

The shortest path is 2,870 miles and corresponds

Each player can pick up 1, 2, or 3 matches and

The loser is the person who picks up the last

How can the player P1 who goes first ensure a

We solve this problem by reasoning in a back-

Consider the situation where 5 matches remain

We consider the development of a transport

Objective: determine a feasible pumping

For the last stage

To find trajectory we retrace forwards through

In addition to this optimal solution, others are

For example, we may calculate the least cost

From the solution, we can also determine the

A company is expanding to meet a wider market

4 different building types (sizes) at each site

Revenues and costs vary with each location and

Revenues R increase monotonically with

Costs C increase monotonically with building

The data for building sizes, revenues and costs

The company can afford to invest at most 21

Determine the optimal expansion policy, i.e., the

We use the DP approach to solve this problem;

For the facilities siting problem we realize that

The amount of funds available is unknown since

The amount of funds s2 available is unknown

The value of d2 is a function of s2 and we

8 rolls of 2 ft paper at 2.50 $/roll

6 rolls of 2.5 ft paper at 3.10 $/roll

5 rolls of 4 ft paper at 5.25 $/roll

4 rolls of 3 ft paper at 4.40 $/roll

The company only has 13 ft of paper to fill these

Determine how to fill orders to maximize profits

Arbitrary order of the stages and we pick

We proceed backwards from stage 1 to stage 4

The maximum profits are $18.45

The optimal solution is obtained by retracing

Consider the case that due to an incorrect

We note that the solution for the original 13 ft

The stage 4′ computations become

The optimal profits in this sensitivity case are

Note that no additional work is needed since the

f 4′′ ( s4 = 13) = 18.2 with d 4′′ = 2 ↔ 2 rolls of 2 ft

The constraint reduces optimum from $ 18.45 to

This problem is concerned with the development

The sales of a seasonal item has the demands

All units sold are purchased from a vendor at

We formulate the problem as a DP and use a

Each stage corresponds to a month

In the DP approach, at each stage we minimize