Branch and bound

General search strategies

Backtracking

Explore all alternatives

• Solution constructed by stepwise choices
• Decision tree
• Guarantees optimal solution
• Exponential time (slow)
 Knapsack problem
Simplified
Input: Weight of N items {w1, w2, ..., wn}
Knapsack limit S
Output: Selection for knapsack: {x1,x2,…xn}
where xi {0,1}.
5
3
Sample input: 7

c =wi={2,3,5,7,11}
i
2
11

S=15
Max 15
 Knapsack as decision tree
wi={2,3,5,7,11}
Yes No

Chooce 2 2 -

Yes No Yes No
Max 15
Chooce 3 5 2 3 -

Yes No Yes No Yes No Yes No

Chooce 5 10 5 7 2 8 3 5 -

Yes Yes No

Chooce 7 17 10 12 5 5 14 9 2 15 8 10 3 12 5 7 -

… … No
… …
28 12 14 13 26 15 12 -
 Bounding criterion

Brute force:
search full tree
Greedy: 2 -
take if fits in
O(2N)
N choices

5 2 3 -

10 5 7 2 8 3 5 -

17 10 12 5 5 14 9 2 15 8 10 3 12 5 7 -

… … …
28 10 12 14 13 26 15 12 -
 Bounding criterion
+wnext > S

Must be sorted!
wi={2,3,5,7,11}
Chooce 2 2 -

Chooce 3 5 2 3 -

Chooce 5 10 5 7 2 8 3 5 -

Chooce 7 12 5 5 14 9 2 15 8 10 3 12 5 7 -

Chooce 11 13 14 11
 Traveling salesman problem
Input: Graph (V,E)
Problem: Shortest tour visiting all nodes.

Closed loop: Open loop:

B C B C
2 2
2 2 2 2
4 4 4 4
3 5 4 3 5 4
A D E F A D E F
3 3 3 3
2 3 2 3
2 2
G H G H

Optimal = A-B-C-F-H-E-D-G-A Optimal = D-G-A-B-C-F-H-E

Length = 22 Length = 17
 Brute force TSP

Brute force:
search full tree

O(N!)
2
1
2 277
153 157
283

2 2 2 2
2
1
2
 Bounding criterion 1
Best found so far

Best candidate: 28 B
6
A
4
2 5 G
C E
1 D
6 4
H
F

New so far: 32 17 B
A 15
G
C E
5! = 120 D
H combinations
F
 Bounding criterion 2
Nearest neighbors

Best candidate: 28 All nodes must be visited.

Find the smallest possible incoming link.

17 B
A
4
G
C E
4 D
1 1 3 3
H
F
16
4+1+1+3+3+4
Lower bound: 33
 Branch and Bound Algorithm:

Scheduling Problem
Material by A.Mirhashemi

Input of the problem:

 A number of resources
A B C
 A number of tasks
1
Output of the problem: 2
 A sequence of feeding the tasks to resources 3
to minimize the required processing time
4
Application 1
Digital processing:
Each resource is a processor. All tasks need to pass trough
all processors in the fix sequence A,B,C but depending on
the task it takes different time for each processor to
process them. For example :

Processor A: Scanning
Processor B: Making a PDF
Processor C: Exporting a PDF

Task 1: A one page plain text document

Task 2: A 10 page document with pictures
Task 3: A 5 page html document.
Task 4: …
Application 2
Production line:
Each product (task) need to pass trough all machines
(resources) in the production line but, the time depends on
what kind of customization the customer has ordered for
that production. For example:

Machine A: Solding
Machine B: Painting
Machine C: Packaging

Task 1: A black car with airbag

Task 2: A red car without airbag with CD player
Task 3: A white car with leather seats
Task 4: …
Different tasks take different time
to be processed in each resource
A B C
1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3
Tasks can be done in any order

N! Possible different sequences

1
2

N
P
3
4
4 2
3 1
2 1 4 3
Decision tree (Brute force)
Greedy Algorithm
A possible greedy algorithm might start with selecting
the fastest tasks for processor A.

A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3

A : 4 2 3 1
Greedy solution A B C

1 7 6 7
T(4,2,3,1) = 34 2 5 5 2
3 6 4 1
4 3 4 3

4 2 3 1

4 2 3 1

4 2 3 1
Optimal solution A B C

1 7 6 7
T(4,1,2,3) = 26 2 5 5 2
3 6 4 1
4 3 4 3

4 1 2 3

4 1 2 3

4 1 2 3
Branch and bound Algorithm
Define a bounding criteria for a minimum time
required by each branch of the decision tree
For level 1:

For level 2:
Level 1 A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3

b(1)= 7+(6+5+4+4)+1=27
b(2)= 5+(6+5+4+4)+1=25
b(3)= 6+(6+5+4+4)+2=27 Minimum
b(4)= 3+(6+5+4+4)+1=23 This next

Bounds: T ≥27 T ≥25 T ≥27 T ≥23

Level 2 A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3
b(4,1)= (3+7)+(6+5+4)+1=26
b(4,2)= (3+5)+(6+5+4)+1=24
b(4,3)= (3+6)+(6+5+4)+2=26 Minimum
This next

Bounds: T ≥26 T ≥24 T ≥26

Solve the branch 4-2-x-x A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3

Bounds: T ≥26 T ≥24 T ≥26

Tmin(4,2,x,x)= 29

Actual: T(4,2,1,3) = 29
T(4,2,3,1) = 34
Solve the branch 4-2-x-x A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3

Bounds: T ≥26 T ≥24 T ≥26

Tmin(4,2,x,x)= 29
Tmin(4,1,x,x)= 26 Tmin(4,3,x,x)= 31

Actual:
T(4,1,2,3) = 26 Actual:
T(4,3,1,2) = 31
T(4,1,3,2) = 28 T(4,3,2,1) = 34
Solve the other branches A B C

1 7 6 7
2 5 5 2
3 6 4 1
4 3 4 3

Can be
skipped

Bounds: T ≥27 T ≥25 T ≥27 T ≥23

Actual Time: T = 26
Must be
solved
 A B C

b(2,1)= (5+7)+(6+4+4)+1=27 1 7 6 7
2 5 5 2
b(2,3)= (5+6)+(6+4+4)+3=28 6 4 1
3
b(2,4)= (5+3)+(6+4+4)+1=23 4 3 4 3

The only candidate that can outperform

T(4,1,2,3) is T(2,4,…) so we calculate it:
Actual T(2,4,1,3) = 29
Actual T(2,4,3,1) = 34
 A B C
So the best time is T(4,1,2,3) and 1 7 6 7
we don’t need to solve the 2 5 5 2
3 6 4 1
problem for any other branch
4 3 4 3
because we now their minimum
time, already.

Bounds: T ≥27 T ≥25 T ≥27 T ≥23

Actual Time: T = 29 T = 26

Bounds greater than 26!

Summary
• Using only the first level criteria we reduce
the problem by 50% (omitting 2 main
branches).
• Using the second level criteria we can reduce
even more.
 Knapsack problem wi={2,3,5,7,11}
Branch-and-bound
0
2
Max 15

2 0
3 3

5 2 3 0
5 5 5 5

12 5 9 2 10 3 7 0

7 7 7 7

12 5 9 2 10 3 11 0
11 11 11

13 2 14 3 11 0
 Traveling salesman problem
A
2 G 2
B D 3

4 6 7 6
D B G
C

8 6 9
E F 11 E G 9 C
12 11 13 11
13
F
F H D 15 C F 15 E

15 14 13 17
F 17 F
H F G 16 20
17 D H
H 20
G 22 22
20
G 24 27 G 24
D 23
A D A A
 Traveling salesman problem
Input: graph (V,E)
Problem: Find shortest path via all nodes and
returning to start node.
B C
2
2 2
4 4
3 5 4
A D E F
3 3
2 3
2
G H
 117
Just in case
if needed
216
110

246

199 182

170
121 315 231
142
79
242 136
191 148 78

191 126 120

178 149 89
116 234
170
51 112 79 131 109
73 86
163 143
90 72 63 53
59 105 27
58
135
116