.4.

2 Optimal Solution for TSP using Branch and Bound

Principle Suppose it is required to minimize an objective function. Suppose that we have a method for getting a lower bound on the cost of any solution among those in the set of solutions represented by some subset. If the best solution found so far costs less than the lower bound for this subset, we need not explore this subset at all. Let S be some subset of solutions. Let
L(S) = a lower bound on the cost of

Let C = cost of the best solution found so far If C L(S), there is no need to explore S because it does not contain any better solution. If C > L(S), then we need to explore S because it may contain a better solution.

A Lower Bound for a TSP Note that: Cost of any tour

= Now: The sum of the two tour edges adjacent to a given vertex v

sum of the two edges of least cost adjacent to v

Therefore: Cost of any tour

Example: See Figure 8.17.

Figure 8.17: Example of a complete graph with five vertices

Node Least cost edges Total cost

a b c d e

(a, d), (a, b) (a, b), (b, e) (c, b), (c, a) (d, a), (d, c) (e, b), (e, f)

5 6 8 7 9

Thus a lower bound on the cost of any tour = (5 + 6 + 8 + 7 + 9) = 17.5 A solution Tree for a TSP instance: (edges are considered in lexicographic order): See Figure 8.18

Figure 8.18: A solution tree for a TSP instance

Suppose we want a lower bound on the cost of a subset of tours defined by some node in the search tree. In the above solution tree, each node represents tours defined by a set of edges that must be in the tour and a set of edges that may not be in the tour.

These constraints alter our choices for the two lowest cost edges at each node. e.g., if we are constrained to include edge (a, e), and exclude (b, c), then we will have to select the two lowest cost edges as follows:
a (a, d), (a, e) 9 b (a, b), (b, e) 6 c (a, c), (c, d) 9 d (a, d), (c, d) 7 e (a, e), (b, e) 10

Therefore lower bound with the above constraints = 20.5

Each time we branch, by considering the two children of a node, we try to infer additional decisions regarding which edges must be included or excluded from tours represented by those nodes. The rules we use for these inferences are: 1. If excluding (x, y) would make it impossible for x or y to have as many as two adjacent edges in the tour, then (x, y) must be included. 2. If including (x, y) would cause x or y to have more than two edges adjacent in the tour, or would complete a non-tour cycle with edges already included, then (x, y) must be excluded.

See Figure 8.19. When we branch, after making what inferences we can, we compute lower bounds for both children. If the lower bound for a child is as high or higher than the lowest cost found so far, we can ``prune'' that child and need not consider or construct its descendants.

Interestingly, there are situations where the lower bound for a node n is lower than the best solution so far, yet both children of n can be pruned because their lower bounds exceed the cost of the best solution so far.

If neither child can be pruned, we shall, as a heuristic, consider first the child with the smaller lower bound. After considering one child, we must consider again whether its sibling can be pruned, since a new best solution may have been found.
Figure 8.19: Branch and bound applied to a TSP instance

Next:8.5 To Probe FurtherUp:8.4 Traveling Salesman ProblemPrevious:8.4.1 A Greedy Algorithm for TSP eEL,CSA_Dept,IISc,Bangalore

Boyer-Moore Algorithm

The Boyer-Moore algorithm is consider the most efficient string-matching algorithm

in usual applications, for example, in text editors and commands substitutions. The reason is that it woks the fastest when the alphabet is moderately sized and the pattern is relatively long. The algorithm scans the characters of the pattern from right to left beginning with the rightmost character. During the testing of a possible placement of pattern P against text T, a mismatch of text character T[i]

= c with the corresponding pattern character P[j] is handled as follows: If c is not contained anywhere in P, then shift the pattern P completely pastT[i]. Otherwise, shift P until an occurrence of character c in P gets aligned with T[i].
This technique likely to avoid lots of needless comparisons by significantly shifting pattern relative to text.

Last Function

We define a function last(c) that takes a character c from the alphabet and specifies how far may shift the pattern P if a character equal to c is found in the text that does not match the pattern

index of the last occurrence

if c is in P

last(c) of c in pattern P = -1 otherwise

For example consider

T:

0 1 2 3 4 5 6 7 8 9 a b a c a a b a c c

P:

a b a c a b 0 1 2 3 4 5

last(a) is the index of the last (rightmost) occurrence of 'a' in P, which is 4. last(c) is the index of the last occurrence of c in P, which is 3
'd' does not exist in the pattern there we have last

(d) = -1.

c 3

d -1

last(c) 4

Now, for 'b' notice

T: a b a c a a b a c c P: a b a c a b

Therefore, last(b) is the index of last occurrence of b in P, which is 5

The complete last(c) function

b 5

c 3

d -1

last(c) 4

Boyer-Moore algorithm

BOYER_MOORE_MATCHER (T, P) Input: Text with n characters and Pattern with m characters Output: Index of the first substring of T matching P Compute function last 2. i m-1 3. j m-1 4. Repeat 5. If P[j] = T[i] then 6. if j=0 then 7. return i // we have a match 8. else 9. i i -1 10. j j -1 11. else 12. i i + m - Min(j, 1 + last[T[i]]) 13. j m -1 14. until i > n -1 15. Return "no match"
1.

Analysis
The computation of the last function takes O(m+||) time and actual search takes O(mn) time. Therefore the worst case running time of Boyer-Moore algorithm is O(nm

+ ||). Implies that the worst-case running time is quadratic, in case of n

= m, the same as the nave algorithm.

Boyer-Moore algorithm is extremely fast on large alphabet (relative to the length of the pattern). The payoff is not as for binary strings or for very short patterns.

For binary strings Knuth-Morris-Pratt algorithm is recommended. For the very shortest patterns, the nave algorithm may be better.

Time
Sort Bubble sort Average O(n^2) Best O(n^2) O(n) O(n^2) O(n) Worst O(n^2) O(n^2) O(n^2) O(n^2) Space Stability Remarks Always use a modified bubble sort Stops after reaching a sorted array Even a perfectly sorted input requires scanning the entire array In the best case (already sorted), every insert requires constant time Constant Stable Constant Stable Constant Stable Constant Stable Modified Bubble O(n^2) sort Selection Sort Insertion Sort Heap Sort Merge Sort Quicksort O(n^2) O(n^2)

O(n*log(n)) O(n*log(n)) O(n*log(n)) Constant Instable By using input array as storage for the heap, it is possible to achieve constant space O(n*log(n)) O(n*log(n)) O(n*log(n)) Depends Stable O(n*log(n)) O(n*log(n)) O(n^2) Constant Stable On arrays, merge sort requires O(n) space; on linked lists, merge sort requires constant space Randomly picking a pivot value (or shuffling the array prior to sorting) can help avoid worst case scenarios such as a perfectly sorted array.

The sorting algorithms

Insertion sort

Insertion sort is good only for sorting small arrays (usually less than 100 items). In fact, the smaller the array, the faster insertion sort is compared to any other sorting algorithm. However, being an O(n2) algorithm, it becomes very slow very quick when the size of the array increases. It was used in the tests with arrays of size 100. Implementation.
Shell sort

Shell sort is a rather curious algorithm, quite different from other fast sorting algorithms. It's actually so different that it even isn't an O(nlogn) algorithm like the others, but instead it's something between O(nlog2n) and O(n1.5) depending on implementation details. Given that it's an in-place non-recursive algorithm and it compares very well to the other algorithms, shell sort is a very good alternative to consider. Implementation.

Heap sort

Heap sort is the other (and by far the most popular) in-place non-recursive sorting algorithm used in this test. Heap sort is not the fastest possible in all (nor in most) cases, but it's the de-facto sorting algorithm when one wants to make sure that the sorting will not take longer than O(nlogn). Heap sort is generally appreciated because it is trustworthy: There aren't any "pathological" cases which would cause it to be unacceptably slow. Besides that, sorting in-place and in a non-recursive way also makes sure that it will not take extra memory, which is often a nice feature. One interesting thing I wanted to test in this project was whether heap sort was considerably faster or slower than shell sort when sorting arrays. Implementation.
Merge sort

The virtue of merge sort is that it's a truely O(nlogn) algorithm (like heap sort) and that it's stable (iow. it doesn't change the order of equal items like eg. heap sort often does). Its main problem is that it requires a second array with the same size as the array to be sorted, thus doubling the memory requirements. In this test I helped merge sort a bit by giving it the second array as parameter so that it wouldn't have to allocate and deallocate it each time it was called (which would have probably slowed it down somewhat, especially with arrays of the bigger items). Also, instead of doing the basic "merge to the second array, copy the second array to the main array" procedure like the basic algorithm description suggests, I simply merged from one array to the other alternatively (and in the end copied the final result to the main array only if it was necessary). Implementation.
Quicksort

Quicksort is the most popular sorting algorithm. Its virtue is that it sorts inplace (even though it's a recursive algorithm) and that it usually is very fast. On reason for its speed is that its inner loop is very short and can be optimized very well. The main problem with quicksort is that it's not trustworthy: Its worse-case scenario is O(n2) (in the worst case it's as slow, if not even a bit slower than insertion sort) and the pathological cases tend to appear too unexpectedly and without warning, even if an optimized version of quicksort is used (as I noticed myself in this project).

I used two versions of quicksort in this project: A plain vanilla quicksort, implemented as the most basic algorithm descriptions tell, and an optimized version (called MedianHybridQuickSort in the source code below). The optimized version chooses the median of the first, last and middle items of the partition as its pivot, and it stops partitioning when the partition size is less than 16. In the end it runs insertion sort to finish the job. The optimized version of quicksort is called "Quick2" in the bar charts. The fourth number distribution (array is sorted, except for the last 256 items which are random) is very expectedly a pathological case for the vanilla quicksort and thus was skipped with the larger arrays. Very unexpectedly this distribution was pathological for the optimized quicksort implementation too, with larger arrays, and thus I also skipped it in the worst cases (because else it would have affected negatively the scale of the bar charts). I don't have any explanation of why this happened. Implementation.
std::sort

I also used the C++ standard sorting function (as implemented in gcc) for comparison. This is a highly-developed sorting function which is somewhat similar to the optimized quicksort I described above, but with further optimizations (such as reverting to heapsort if the sorting seems to be too slow).

How to read the comparisons/assignments tables

Besides a bar chart, each testcase includes a table of numbers. This table presents the amount of comparison and assignment operations performed by the sorting algorithm in average during the sorting of one input array. The amount of comparisons and assignments and their relative ratio is not extremely relevant when comparisons and assignments are fast (such as with integers). However, when comparison or assignment of the elements is a heavy operation then it may be much more interesting and relevant to compare different sorting algorithms with regard to how much they perform those operations compared to the other algorithms. Each table has the following format:
Random Almost sortd Almost rev. Random end

Alg.

Comp. Assig. Comp. Assig. Comp. Assig. Comp. Assig.

(name) (value) (value) (value) (value) (value) (value) (value) (value) ... ... ... ... ... ... ... ... ...

The first column specifies the sorting algorithm. Each row has pairs of numbers, one pair for each input data distribution type. They represent the amount of comparisons and assignments the algorithm needed (in average) to sort the array once.
There are three things to consider here. First, insertion sort is much faster (O(n) vs O(n log n)) than quicksort IF the data set is already sorted, or nearly so; second, if the data set is very small, the 'start up time" to set up the quicksort, find a pivot point and so on, dominates the rest; and third, Quicksort is a little subtle, you may want to re-read the code after a night's sleep.