five thirty-seven

five thirty-seven

Introduction

Lecture 21:
Permutations and Backtracking

Slide 1

abc, acb, bac, bca, cab, cba

There are n! permutations of a set of n elements since

the first element of the sequence can be chosen in n
ways, the second in n 1 ways, the third in n 2
ways, etc.
Among other things, permutations are useful in
characterizing (either explicitly or implicitly) the
solution sets of a variety of arrangement problems,
notably the traveling salesman problem.
Slide 2

An Algorithm for Permutations

Algorithms for generating permutations abound, and

may produce permutations in lexicographic order,
randomly, exhaustively or partially, and may do so
either more or less efficiently.
Heres one way to generate permutations:
For a one-item set, print the item
For a two-item set, print the items and then reverse them
For a three-item set, print:
the first followed by permutations of the second and third
the second followed by permutations of the first and third
the third followed by permutations of the first and second
For an n-item set, . . .
Slide 3

five thirty-seven

five thirty-seven

For example, if S = {a, b, c} there are 6 permutations:

Lecture 21: Permutations and Backtracking

Generating Permutations

Lecture 21: Permutations and Backtracking

A permutation of a finite set S is an ordered sequence

of all of the elements of S, with each element
appearing exactly once.

Compute all permutations of an array by exchanging

each element to the beginning, then recursively
permuting the others:
Print-Perms(A, i)
1 if i = Length(A)
2
print A
3 else for j = i to Length(A) 1
4
swap A[i] and A[j]
5
Print-Perms(A, i + 1)
6
swap A[i] and A[j]

Note that the algorithm performs more swaps (2n!)

than some other algorithms, but the approach is
convenient for implementing backtracking (a type of
pruning of the search for permutations).
Lecture 21: Permutations and Backtracking

Slide 4

Print-Perms(A, i)
1 if i = Length(A)
2
print A
3 else for j = i to Length(A) 1
4
swap A[i] and A[j]
5
Print-Perms(A, i + 1)
6
swap A[i] and A[j]
Note that a typical call is of the
form Print-Perms([1, 2, 3], 0),
but that portions of the input
array can be left immune to
permutation by choosing a second
argument i > 0.

Traveling Salesman Problem

Print-Perms([1, 2, 3], 2)
[1, 2, 3]
Print-Perms([1, 2, 3], 1)
[1, 2, 3]
[1, 3, 2]
Print-Perms([1, 2, 3], 0)
[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 2, 1] not lexicographic
[3, 1, 2]

Lecture 21: Permutations and Backtracking

five thirty-seven

five thirty-seven

The Algorithm in Action

Slide 5

Notation: let c(A) denote the total cost of the edges in

the subset A E
X
c(u, v)
c(A) =
(u,v)A

In the euclidean traveling salesman problem, we are

given a set of n points in the plane, and we wish to
find the shortest closed tour that connects all n points.
Slide 6

4-City Tours with Fixed Start

In the euclidean TSP, each ordering of points

constitutes a legal tour.
An optimal TSP tour through
Germanys 15 largest cities. It is
the shortest among
43,589,145,600 possible tours
visiting each city exactly once
(from Wikipedia).
Note that this number is 14!/2 . . .
why?
How many feasible tours exist for
a 3-city problem? A 4-city
problem?

Slide 7

five thirty-seven

five thirty-seven

(a tour is a hamiltonian cycle: a simple cycle that

contains each vertex in V )

Lecture 21: Permutations and Backtracking

Euclidean TSP

Lecture 21: Permutations and Backtracking

Given a complete undirected graph G = (V, E) that has

a nonnegative integer cost c(u, v) associated with each
edge (u, v) E, find a tour of G with minimum cost.

Lecture 21: Permutations and Backtracking

Slide 8

five thirty-seven

An Algorithm for Exhaustive Search

five thirty-seven

5-City Tours with Fixed Start

Lecture 21: Permutations and Backtracking

Slide 9

An optimal solution can be found by generating all

permutations, examining the tour length of each, and
returning the shortest one:
Tsp-Exhaustive(P )
// input is a list of points
1 cand (, [])
2 for tour in Perms([0..Length(P ) 1], 1)
3
curr Tour-Length(tour , P )
4
if curr < cand[0]
// better candidate found
5
curr (curr, tour)
6
return cand

For all but the smallest problems, however, such an

approach is intractable. How large a problem can we
solve in this manner?

Lecture 21: Permutations and Backtracking

five thirty-seven

A Random 10-City Problem

five thirty-seven

A Random 10-City Problem

Lecture 21: Permutations and Backtracking

Slide 10

Slide 11

$ cat tsp10.txt
0.70501 0.58793
0.26077 0.84765
0.12284 0.19949
0.20125 0.85198
0.48505 0.08244
0.94810 0.30570
0.69991 0.32967
0.15261 0.59770
0.56046 0.39271
0.80336 0.23430

$ time python tsp.py tsp10.txt

(2.7600550582605496, [0, 5, 9, 6, 8, 4, 2, 7, 3, 1])
[(0.70501, 0.58793),
(0.94810, 0.30570),
(0.80336, 0.23430),
(0.69991, 0.32967),
(0.56046, 0.39271),
(0.48505, 0.08244),
(0.12284, 0.19949),
(0.15261, 0.59770),
(0.20125, 0.85198),
(0.26077, 0.84765)]
8.547u 0.030s 0:08.75 97.9% 0+0k 62+1io 0pf+0w

Lecture 21: Permutations and Backtracking

Slide 12

TSP with Backtracking

If a 10-city problem takes about 9 seconds, how long

would it take to solve a 20-city problem?

What about just an 11-city problem?

A 12-city problem?

Lecture 21: Permutations and Backtracking

five thirty-seven

five thirty-seven

Running Times

Slide 13

The algorithm can be called as follows:

Tspwb(P , [0..Length(P ) 1], 1)

Slide 14

Performance Comparison

Two simple changes are made:

Instead of printing a permutation for the base case, we
check the length of the tour against a global candidate
solution, which should be initialized with a large number.
The recursive call is guarded by a test to ensure that the
call is not made if the current partial tour already exceeds
the length of the candidate tour.
Note that Tour-Length(A, P, i) computes the length only up
to i in list A.

Slide 15

five thirty-seven

five thirty-seven

Tspwb(P , A, i)
// points, order, level
1 if i = Length(A)
2
curr Tour-Length(A, P , Length(A))
3
if curr < cand[0]
4
cand (curr, Copy(A))
5 else for j = i to Length(A) 1
6
swap A[i] and A[j]
7
if Tour-Length(A, P , i) < cand[0] // branch only if
8
Tspwb(P , A, i + 1)
// improvement possible
9
swap A[i] and A[j]

Lecture 21: Permutations and Backtracking

Comments on Backtracking

Lecture 21: Permutations and Backtracking

Using the original algorithm for printing permutations:

Lecture 21: Permutations and Backtracking

Slide 16

Applegate et al. 1998 Record

Year

Researchers

1954

Dantzig, Fulkerson, and Johnson

1971

Held and Karp

1975

Camerina, Fratta, and Maffioli

100

1977

Grotschel

120

1980

Crowder and Padbert

318

1987

Padberg and Rinaldi

532

1987

Grotschel and Holland

666

1994

Applegate, Bixby, Cook, and Chvatal

7,397

1998

Applegate, Bixby, Cook, and Chvatal

13,509

2001

Applegate, Bixby, Cook, and Chvatal

15,112

2005

Cook and others

33,810

five thirty-seven

five thirty-seven

Some Landmark TSP Results

N-Cities
49
64

Lecture 21: Permutations and Backtracking

Slide 17

Lecture 21: Permutations and Backtracking

Slide 19

five thirty-seven

five thirty-seven

Slide 18

Solving the Puzzle

Consider a floating mobile with weightless and perfectly rigid

pin-connected rods. The object of the puzzle is to move the weights
and balloons to the empty frame so that it maintains equilibrium.
3

Specially for MIP, we developed a new branching rule called strong

branching, which is particularly useful in the solution of difficult
integer programs. This branching rule is now available in several
commercial MIP solvers. -Bixby

Lecture 21: Permutations and Backtracking

Balloon Balance Puzzle

13,509 U.S. cities with populations of more than 500

people connected optimally:

To maintain overal equilibrium, the vertical forces

must collectively sum to zero. In addition, the
moments about any position on a rod must sum to
zero:
moment = force moment arm
If there is no need to satisfy equilibrium requirements,
how many configurations of balloons and weights on
the empty frame are possible?
Would an exhaustive search find the solution?
Could a backtracking algorithm be devised?

Lecture 21: Permutations and Backtracking

Slide 20