Dynamic Programming

We already talked about dynamic programming and how MergeSort and QuickSort and even
BinarySearch and RandomizedSelect are examples of that kind of algorithms divide into
pieces! solve each one! then merge the solutions"
#h"$% is a little different version of the same thing" &he algorithms are not
recursive in the same way we saw before" &he structure of divide'con(uer'merge is
still there! but the )*+*), is not a -clean- one'cut divide like we saw before"
MergeSort divides exactly in half! always" QuickSort divides into two -halves- exactly
at the point where .artition calculates the cut to be"
But #h"$% algorithms do not have such a clean cut""" *nstead! they &R/ ,+,R/ .0SS*B1,
#2&! and then pick the best one" &hat3s where the -minimum out of many numbers-
algorithm was needed for the Review 4omework" We will use it here"
&he conse(uence of trying every possible cut is that different cuts can overlap"
&herefore! the issue that makes dynamic programming problems a special case of divide'
and'con(uer problems is that the divide is so variable! and leads to overlapping
subproblems"
&he reason why #h"$% algorithms work in that way is because they are optimization
algorithms" &herefore! they have to find the best solution"
When you read the assembly line problem! try to visualize all different possibilities
for going from start to end" )ynamic programming algorithm is really an intelligent
brute force method that will try all possible paths to find the shortest one"
&herefore! the first part of solving a dynamic programming problem is to come up with
a formula that describes the subproblems" *n this class! we are not too interested in
becoming extremely proficient with discerning what the formula should be"
&he interesting part about #h"$% is going from formulating problems in math notation
to coding" /ou will notice that the math formula can be coded in two ways recursively
5assembly line recursive code is in the notes on the web6 and iteratively 5in the
book6" &his part is of great interest to us in this class! and we will focus a lot of
attention on it" Why7 Because there are tremendous conse(uences if we code the formula
one way or the other"
Recursive code can be 8plain9 recursive where everything is calculated 8as needed9 and
then discarded 5think of it as 8use once and throw away96! versus recursive code where
returned results are stored for future use" &hat kind of code is called 8memoized"9
Mini (uiz what issue is the memoized code trying to solve7
Dynamic Programming Example: Assembly Line
Goal: assembly a car as fast as possible, using two identical assembly lines, each with n machines.
e
i
: time to enter a line i=1, 2
x
i
: time to exit a line
a
ij
: time to process a job at station S
ij
i=1, 2 j= 1, , n
t
ik
: time to transfer from line i after ha!ing gone through station S
i"
i=1, 2 "= 1, , n#1
f*: fastest time through the whole factory
f[i, j]: fastest time through station j on line i i=1,2 j = 1, 2, ., n.
line
ij
: line whose station j#1 was used in a fastest way through S
ij
i=1, 2 j= 2, , n
l*: line whose station n was used in the fastest way through the whole factory
Dynamic programming soltion: decide beforehand how to split the problem into independent $and most
li"ely related and o!erlapping% optimal subproblems. &hen sol!e the subproblems bottom#up, remembering
the intermediate solutions.
Steps'
!" (efine the structure of the optimal solution
#" )rite recursi!e formula describing the optimal solution
$" *ompute the formula bottom%p
&" *onstruct the optimal solution from computed information
+n the case of assembly line'
1. write the formula for the fastest way through station Sij
2. using formula from 1, write recursi!e formula to calculate the fastest time through the whole factory
,. write pseudocode to calculate the fastest time through factory is $e.g. 1- hours%
.. calculate the fastest route through the factory $e.g. for n=,, through stations S11, S12, and S2,%.
Gree'y algorit(m soltion: at each decision point, ma"e a choice that loo"s the best at the moment,
hoping that the final solution will be optimal. Sol!e the resulting subproblems top#down, and do not
remember them.
a
2n
x
2
x
1
car
e
1
e
2
chasis
a
1n
a
11
a
12
a
21
a
22
t
11
t
21
a
2,j#1
a
2j
t
1,j#1
t
2,j#1
a
1,j#1
a
1j
.
.
So, we ha!e to write the algorithm to calculate the fastest time through the factory.
FastestTimeTruFactory
a[2][n]
t[2][n-1]
e[2] f*, l*,
x[2] list of stations in the shortest path (n long)
n
&here are se!eral ways in which we can write this algorithm. )e can write it recursi!ely top#down, or we
can write it iterati!ely bottom#up. +t is usually easier to start recursi!ely, because it matches the informal
human way of thin"ing.
/ecursi!e call should return f0.
f* = min(f[1,n] x
1
, f[2,n] x
2
) !! "e either come off line 1 or line 2
f[1, n] = min(f[1, n-1] a
1,n
, f[2, n-1] t
2,n-1
a
1,n
) !! "e either come #irectly from the pre$ious
!!station on line1, or "e come from the
!!pre$ious station from line 2 an# transfer
from !!line 2 to line 1%
So we ha!e to "eep on unwrapping bac"wards. +n general, we write a recursi!e function'
f[1, &] = min(f[1, &-1] a
1,&
, f[2, &-1] t
2,&-1
a
1,&
)
f[1,1] = e
1
a
1,1
1nd call it with the initial call of f21,n3.
4ow to map math into code'
'alcf*(a,t,x,e,n) (
return min('alcf(a,t,x,e,n, 1, n) x[1], 'alcf(a,t,x,e,n, 2, n) x[2])
)
'alcf(a,t,x,e,n, 1, &) (
*f &==1
+eturn a[1][1]e[1]
%
return min('alcf(a,t,x,e,n, 1, &-1) a[1][&], 'alcf(a,t,x,e,n, 2, &-1) t[2][&-1] a[1][&])
)
*alculate fastest time
through the entire factory
'alcf* = min('alcf,*T-.T/+012(3,1,n) x[1] , 'alcf,*T-.T/+012(3,2,n) x[2])
'alf,*T-.T/+012(3%) (
'hec4 if "e ha$e it
*f yes, return it
2lse calculate, store, return
)
5ain'alc,*T-.T/+012 (3) (
*nitiali6e f[ ][ ] to 77777
)
'alf,*T-.T/+012() (
*f f[ ] [ ] is not empty, "e alrea#y ha$e the solution
+eturn f[ ] [ ]
2lse
*f &==1
f[1,1] = a[1][1]e[1]
+eturn f[1,1]
f[1,&-1] = % 'alcf,*T-.T/+012(a,t,x,e,n, 1, &-1)
f[2, &-1] = 'alcf,*T-.T/+012(a,t,x,e,n, 2, &-1)
return min(f[1, &-1] a[1][&], f[2, &-1] t[2][&-1] a[1][&])
)
&hen of course we ha!e to write function *alcf$a, t, x, e,n, 2, j% which is a mirror image of
*alcf$a,t,x,e,n, 1, j%.
5&), +t is also possible to write the algorithm using functions *alcf1$a, t, x, e,n, j% and *alcf2$a, t, x, e,n,
j%.
So far so good6 this code will return the fastest time through the factory, for example 7 hours. 5ut we will
not "now which stations to go though to achie!e this fastest time. So, we ha!e to modify the code to ma"e
sure to remember which stations we passed through. &herefore, we cannot use the min function out of the
box, we ha!e to write min from scratch and remember which side ga!e us the minimum.
!!assume line[][] an# l* are glo8als
'alcf*(a,t,x,e,n) (
if ('alcf(a,t,x,e,n, 1, n) x[1] 9= 'alcf(a,t,x,e,n, 2, n) x[2]) (
f* = 'alcf(a,t,x,e,n, 1, n) x[1]
l* = 1 !! the car came off line 1
)
else (
f* = 'alcf(a,t,x,e,n, 2, n) x[2]
l* = 2 !! the car came off line 2
)
return f*
)
'alcf(a,t,x,e,n, 1, &)
if ('alcf(a,t,x,e,n, 1, &-1) a[1][&] 9= 'alcf(a,t,x,e,n, 2, &-1) t[2][&-1] a[1][&]) (
line[1, &] = 1
f[1,&] = 'alcf(a,t,x,e,n, 1, &-1) a[1][&]
)
else (
line[1, &] = 2
f[1,&] = 'alcf(a,t,x,e,n, 2, &-1) t[2][&-1] a[1][&])
)
return f[1,&]
)
&his code wor"s, (o)e*er it is going to be !ery slow, because it "eeps on calculating the same 8uantities
o!er and o!er again. 1s soon as a 8uantity is calculated, it is discarded 9 we do not store it anywhere # so
when we need it the next time, we ha!e to calculate it again.
1 remedy for that is to sa!e e!erything we calculate, and when we need it the next time, just pull it out of
the storage. &his approach is called memoi:ed. ;emoi:ed approach is the recursi!e approach with the
storage.
&he algorithm is'
*nitiali6e storage
5a4e the initial call
5emoi6e#'alcf( 3, 1, &) (
if f[1,&] is not empty, return it
else calculate it recursi$ely ("atch out, the recursi$e call "ill 8e to 5emoi6e#'alcf)
)
&he third way is to calculate it iterati!ely, using bottom up approach. &hat code is in the boo".
<< +terati!e !ersion
<<&he fastest way through a factory with 2 assembly lines
<< =rom the textboo", p.,2>
<<+nput'
<< matrices a and t6 a is of si:e 2xn, t is of si:e 2xn#1.
<< !ectors e and x, of si:e 2
<< integer n $number of stations%
<<
<<?utput'
<< real number f0, the fastest time through the whole factory
<< integer l0, the line whose station was used to pass through n
th
station
<< matrix f2 , 3 of si:e 2xn6 or !ectors f!23 and f#23 of si:e n, containing fastest times
through each station
<< matrix line2 , 3 of si:e 2xn#1 or !ectors line!23 and line#23 of si:e n#1, containing the
fastest path through each station
=1S&@S&#)1A$a, t, e, x, n% B
f
1,1
= e
1
C a
1,1
<<can be coded as' f21,13 = e213 C a21,13
<<or' f1213 = e213 C a21,13
f
2,1
= e
2
C a
2,1
for j = 2, n
if $ f
1,j#1
C a
1,j
D f
2,j#1
C t
2,j#1
C a
1,j
% <<line 1 is faster
f
1j
= f
1,j#1
C a
1j

line
1j
= 1 <<code as' line21,j3 or line12j3
else <<line 2 is faster
f
1,j
= f
2,j#1
C t
2,j#1
C a
1,j
line
1,j
= 2
if $ f
2,j#1
C a
2,j
D f
1,j#1
C t
1,j#1
C a
2,j
% << line 2 is faster
f
2,j
= f
2,j#1
C a
2,j

line
2,j
= 1
else <<line 1 is faster
f
2,j
= f
1,j#1
C t
1,j#1
C a
2,j
line
2,j
= 2
if $f
1,n
C x
1
D f
2,n
C x
2
%
f0 = f
1,n
C x
1
l0 = 1
else
f0 = f
2,n
C x
2
l0 = 2
E
4ow do we "now which stations are in the shortest pathF 1gain, we ha!e to use a recursi!e
approach.
l0 tells what line was used for station n.
l2i,j3 tells what line was used for station j#1 when getting to station j on line i.
So, we need'
l0 << this is line for station n
l2l0, n3 <<this is line for station n#1
l2l2l0,n3, n#13 <<this is line for station n#2

4ow do we read that from line2323 matrixF

l0 tells us which line to read for station n#1, etc. "eep on bac"trac"ing.
=or example'
l0=2, n=,
line = G+H 2 2 <<G+H means that line21,13 and line22,13 donIt exist
G+H 1 1
l0=2 means that we got off station S
2,,
So we loo" up l22,,3 which is 1, which means we came through S
1,2
So we loo" up l21, 23, which is 2, which means we came through S
2,1
So the shortest path through factory is S
2,1
S
1,2
S
2,,

&his is a !ery important approach that we will use for the majority of algorithms for the rest
of the semester.
J/+G&#S&1&+?GS$line, n% B
i = l0
print line i , station n
for $j = n, j K 2, jL%
i = line
i,j
print line i , station j#1
E