CSC 580 Quicksort Lecture

CSC 580 Computer Algorithms
LECTURE 4. Quicksort
SEN-920 Computer Algorithms
Agenda
Quicksort
Divide and conquer
Partitioning
Worst-case analysis
Intuition
Randomized quicksort
Analysis
Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Quicksort
Proposed by C.A.R. Hoare in 1962.

Divide-and-conquer algorithm.
Sorts in place (like insertion sort, but not
like merge sort).
Very practical (with tuning).

Divide and Conquer
Quicksort an n-element array:

1. Divide: Partition the array into two subarrays
around a pivot x such that elements in lower
subarray x elements in upper subarray.
xx xx xx
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
Key: Linear-time partitioning subroutine.

Partitioning Subroutine
PARTITION(A, p, q) A[ p . . q]
x A[ p] pivot = A[ p] Running time
ip = O(n) for n
for j p + 1 to q elements.
do if A[ j] x
then i i + 1
exchange A[i] A[ j]
exchange A[ p] A[i]
return i
Invariant: xx xx xx ??
p i j q
Example of Partitioning
66 10
10 13
13 55 88 33 22 11
11
i j

66 10
10 13
13 55 88 33 22 11
11
i j

66 10
10 13
13 55 88 33 22 11
11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
6 5 3 2 8 13 10 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
6 5 3 2 8 13 10 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
6 5 3 2 8 13 10 11
i j

66 10
10 13
13 55 88 33 22 11
11
6 5 13 10 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
6 5 3 2 8 13 10 11
2 5 3 6 8 13 10 11
2 5 3 6 8 13 10 11
i

Pseudocode for Quicksort
QUICKSORT(A, p, r)
if p < r
then q PARTITION(A, p, r)
QUICKSORT(A, p, q1)
QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)

Analysis of Quicksort
Assume all input elements are distinct.

In practice, there are better partitioning
algorithms for when duplicate input
elements may exist.
Let T(n) = worst-case running time on
an array of n elements.

Worst-Case of Quicksort
Input sorted or reverse sorted.

Partition around min or max element.
One side of partition always has no elements.
T (n) T(0) T (n 1) (n)
(1) T (n 1) (n)
T(n 1) (n)
(n 2 ) (arithmetic series)

Worst-Case Recursion Tree
T(n) = T(0) + T(n1) + cn

T(n)

T(n) = T(0) + T(n1) + cn

cn
T(0) T(n1)

T(n) = T(0) + T(n1) + cn

cn
T(0) c(n1)
T(0) T(n2)

T(n) = T(0) + T(n1) + cn

cn
k n 2
n
k 1
T(0) c(n1)
T(0) c(n2)
T(0)
(1)

T(n) = T(0) + T(n1) + cn

cn
k n 2
n
(1) c(n1)
k 1
(1) c(n2)
h=n T(n) = (n) + (n2)
(1) = (n2)
(1)

Best-Case Analysis
(For intuition only!)

If were lucky, PARTITION splits the array evenly:
T(n) = 2T(n/2) + (n)
= (n lg n) (same as merge sort)
1 9
What if the split is always :
10 10
?
T () T T
1 9
10
10
) ()
What is the solution to this recurrence?

Analysis of Almost-Best Case
T (n)

cn
T T
1 9

10 10

cn
1 9
10
cn 10
cn
T 100 T100 n T 100 T100 n

1 9 9 81
n n

cn cn
1
10
cn 9
10
cn cn
log10n log 10/9 n
1 9 9
100 cn 100 cn 100 cn 81
100 cn
cn

(1) O(n) leaves
(n lg n) (1)
Lucky! cn log10n T(n) cn log10/9n + O(n)
More Intuition
Suppose we alternate lucky, unlucky,

lucky, unlucky, lucky, .
L(n) = 2U(n/2) + (n) lucky
U(n) = L(n 1) + (n) unlucky
Solving:
L(n) = 2(L(n/2 1) + (n/2)) + (n)
= 2L(n/2 1) + (n) 2L(n/2) + (n)
= (n lg n) Lucky!
How can we make sure we are usually lucky?
Randomized Quicksort
IDEA: Partition around a random element.
RANDOMIZED-PARTITION(A, p, r)
i = RANDOM(p, r)
Exchange A[p] with A[i]
return PARTITION(A, p, r)
RANDOMIZED-QUICKSORT(A, p, r)
if p < r
then q RANDOMIZED-PARTITION(A, p, r)
RANDOMIZED-QUICKSORT(A, p, q1)
RANDOMIZED-QUICKSORT(A, q+1, r)
Properties of Randomized Quicksort
Running time is independent of the input order.

No assumptions need to be made about the
input distribution.
No specific input elicits the worst-case
behavior.
The worst case is determined only by the output
of a random-number generator.

Randomized Quicksort Analysis

Let T(n) = the random variable for the running
time of randomized quicksort on an input of size
n, assuming random numbers are independent.
For k = 0, 1, , n1, define the indicator
random variable
1 if PARTITION generates a k : nk1 split,
Xk = i.e., A[1] is the (k+1)th smallest number
0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are
equally likely, assuming elements are distinct.
Analysis (continued)
T(0) + T(n1) + (n) if 0 : n1 split,

T(1) + T(n2) + (n) if 1 : n2 split,
T(n) =
T(n1) + T(0) + (n) if n1 : 0 split,
n1
X k T (k) T (n k 1) (n)
k 0

Calculating Expectation
Take expectations of both sides.
E[T(n)] =
n1
E X k T (k) T (n k 1) (n)
k 0

Linearity of expectation.
E[T(n)]
= E + 1 +
=0
1
= [ + 1 + ]
=0

E[T(n)]
E + 1 +

=0
n1
EX k T(k) T (n k 1) (n)
k 0
n1
EX k ET (k) T (n k 1) (n)
k 0
Independence of Xk from other random choices.

E[T(n)]
E + 1 +

=0
n1
EX k T(k) T (n k 1) (n)
k 0
n1
k 0
n1 n1 n 1
1n ET (k) 1n ET (n k 1) 1n (n)
k 0 k 0 k 0
Linearity of expectation; E[Xk] = 1/n .

E[T(n)]
E + 1 +

=0
n1
EX k T(k) T (n k 1) (n)
k 0
n1
k 0
n1 n1 n 1
1n ET (k) 1n ET (n k 1) 1n (n)
k 0 k 0 k 0
n1
2n ET (k) (n) Summations have
k 0 identical terms.
Hairy Recurrence
n1
E[T (n)] 2 ET(k ) (n)
n k =0
(The k = 0 term can be absorbed in the (n).)
Prove by induction: E[T(n)] an lg n for
constant a > 0 .
Choose a large enough so that an lg n
dominates E[T(n)] for sufficiently large n 2.
n1
Use fact: k lg k 1 n 2 lg n 1n 2
2 8
k1
Upper Bound

Substitution Method
n1
ET (n) 2 ak lg k (n)
n k1
Substitute inductive hypothesis.

Substitution Method
n1
n k1
2 1 2 1 2
( n lg n n ) (n)
2 8
Use the upper bound.

Substitution Method
n1
n k1
2 1 2 1 2
( n lg n n ) (n)
2 8

anlgn ( 4 (n))
Express as desired residual.

Substitution Method
n1
n k1
2 1 2 1 2
( n lg n n ) (n)
2 8

anlgn ( 4 (n))
anlgn,
if a is chosen large enough so that

4
dominates the (n).

Quicksort in Practice
Quicksort is a great general-purpose

sorting algorithm.
Quicksort is typically over twice as fast
as merge sort.
Quicksort can benefit substantially from
code tuning.
Quicksort behaves well even with
caching and virtual memory.

CSC 580 Quicksort Lecture

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

CSC 580 Quicksort Lecture

Caricato da

Copyright:

Formati disponibili

CSC 580 Computer Algorithms

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Proposed by C.A.R. Hoare in 1962.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Divide and Conquer

Quicksort an n-element array:

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Pseudocode for Quicksort

Initial call: QUICKSORT(A, 1, n)

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Assume all input elements are distinct.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Input sorted or reverse sorted.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Worst-Case Recursion Tree

T(n) = T(0) + T(n1) + cn

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Worst-Case Recursion Tree

T(n) = T(0) + T(n1) + cn

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Worst-Case Recursion Tree

T(n) = T(0) + T(n1) + cn

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Worst-Case Recursion Tree

T(n) = T(0) + T(n1) + cn

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Worst-Case Recursion Tree

T(n) = T(0) + T(n1) + cn

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

(For intuition only!)

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Analysis of Almost-Best Case

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Analysis of Almost-Best Case

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Analysis of Almost-Best Case

T 100 T100 n T 100 T100 n

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Analysis of Almost-Best Case

Suppose we alternate lucky, unlucky,

Properties of Randomized Quicksort

Running time is independent of the input order.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Randomized Quicksort Analysis

T(0) + T(n1) + (n) if 0 : n1 split,

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Take expectations of both sides.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017

Independence of Xk from other random choices.

Based on slides by Erik Demaine and Charles Leiserson 2/7/2017