CS F364: D ESIGN & A NALYSIS OF A LGORITHMS

Lecture-kt05: Randomized Quick Sort

Dr. Kamlesh Tiwari,

Assistant Professor,
Department of Computer Science and Information Systems,
BITS Pilani, Rajasthan-333031 INDIA

Jan 31, 2017 (Campus @ BITS-Pilani Jan-May 2017)

Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Recurrence Solution with Master Method

T (n) = 9T (n/3) + n Case-1: T (n) = Θ(n2 )

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Recurrence Solution with Master Method

T (n) = 9T (n/3) + n Case-1: T (n) = Θ(n2 )
T (n) = T (2n/3) + 1 Case-2: T (n) = Θ(log n)

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Recurrence Solution with Master Method

T (n) = 9T (n/3) + n Case-1: T (n) = Θ(n2 )
T (n) = T (2n/3) + 1 Case-2: T (n) = Θ(log n)
T (n) = 3T (n/4) + n log n Case-3: T (n) = Θ(n log n)

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Recurrence Solution with Master Method

T (n) = 9T (n/3) + n Case-1: T (n) = Θ(n2 )
T (n) = T (2n/3) + 1 Case-2: T (n) = Θ(log n)
T (n) = 3T (n/4) + n log n Case-3: T (n) = Θ(n log n)
T (n) = 2T (n/2) + n log n Case-?: Falls in gap of case 2 & 3

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Master method
When T (n) = aT (n/b) + f (n) a ≥ 1, b > 1

Let  > 0 be a constant

1 If f (n) = O(nlogb a− ) then T (n) = Θ(nlogb a )
2 If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a log n)
3 If f (n) = Ω(nlogb a+ ) then T (n) = Θ(f (n))
provided if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n. Regularity condition must be checked in case-3.

Recurrence Solution with Master Method

T (n) = 9T (n/3) + n Case-1: T (n) = Θ(n2 )
T (n) = T (2n/3) + 1 Case-2: T (n) = Θ(log n)
T (n) = 3T (n/4) + n log n Case-3: T (n) = Θ(n log n)
T (n) = 2T (n/2) + n log n Case-?: Falls in gap of case 2 & 3

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 2/8
Recap: Quick Sort
Which algorithm is better ?
Best Case Worst Case Average Case
Algo-01 n log n n log n n log n
Algo-02 n log n n(n − 1) n log n

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 3/8
Recap: Quick Sort
Which algorithm is better ?
Best Case Worst Case Average Case
Algo-01 n log n n log n n log n
Algo-02 n log n n(n − 1) n log n
If I tell you Algo-01 is merge sort and Algo-02 is quick sort then?

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 3/8
Recap: Quick Sort
Which algorithm is better ?
Best Case Worst Case Average Case
Algo-01 n log n n log n n log n
Algo-02 n log n n(n − 1) n log n
If I tell you Algo-01 is merge sort and Algo-02 is quick sort then?
Why quick sort is popular? it always behaves like average case as
the number of items to be sorted sort increases

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 3/8
Recap: Quick Sort
Which algorithm is better ?
Best Case Worst Case Average Case
Algo-01 n log n n log n n log n
Algo-02 n log n n(n − 1) n log n
If I tell you Algo-01 is merge sort and Algo-02 is quick sort then?
Why quick sort is popular? it always behaves like average case as
the number of items to be sorted sort increases
1000 time execution of randomized quick sort on randomly
selected items
Number of items n =
Number of times the runtime 102 103 104 105 106
exceed average behavior
10% 190 49 22 10 3
20% 28 17 12 3 0
50% 2 1 1 0 0
100% 0 0 0 0 0

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 3/8
Execution of Quick Sort

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 4/8
Randomized Quick Sort

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 5/8
Analysis of Randomized Quick Sort

We need to estimate number of comparisons performed during its

execution

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 6/8
Analysis of Randomized Quick Sort

We need to estimate number of comparisons performed during its

execution
Let the sorted list of items be < S1 , S2 , S3 , ..., Sn > with Si being
i th smallest element

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 6/8
Analysis of Randomized Quick Sort

We need to estimate number of comparisons performed during its

execution
Let the sorted list of items be < S1 , S2 , S3 , ..., Sn > with Si being
i th smallest element
Define a random variable Xij to be the number of comparisons
between Si and Sj

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 6/8
Analysis of Randomized Quick Sort

We need to estimate number of comparisons performed during its

execution
Let the sorted list of items be < S1 , S2 , S3 , ..., Sn > with Si being
i th smallest element
Define a random variable Xij to be the number of comparisons
between Si and Sj
Xij can either take a value 0 or 1

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 6/8
Analysis of Randomized Quick Sort

We need to estimate number of comparisons performed during its

execution
Let the sorted list of items be < S1 , S2 , S3 , ..., Sn > with Si being
i th smallest element
Define a random variable Xij to be the number of comparisons
between Si and Sj
Xij can either take a value 0 or 1
Expected number of comparison is

Xn X n X
X
E[ Xij ] = E[Xij ]
i=1 j>i i=1 j>i

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 6/8
Randomized Quick Sort

Let pij denote the probability

of comparison between Si
and Sj . Then,

E[Xij ] = pij ×1+(1−pij )×0 = pij

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

Let pij denote the probability

of comparison between Si
and Sj . Then,

E[Xij ] = pij ×1+(1−pij )×0 = pij

Pivot element can either be

1 Si or Sj : its probability is
2
j−i+1

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

Let pij denote the probability

of comparison between Si
and Sj . Then,

E[Xij ] = pij ×1+(1−pij )×0 = pij

Pivot element can either be

1 Si or Sj : its probability is
2
j−i+1
2 Sq from i < q < j

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

Let pij denote the probability

of comparison between Si
and Sj . Then,

E[Xij ] = pij ×1+(1−pij )×0 = pij

Pivot element can either be

1 Si or Sj : its probability is
2
j−i+1
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si
and Sj . Then,

E[Xij ] = pij ×1+(1−pij )×0 = pij

Pivot element can either be

1 Si or Sj : its probability is
2
j−i+1
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si n X
X 2
and Sj . Then, =
j −i +1
i=1 j>i
E[Xij ] = pij ×1+(1−pij )×0 = pij

Pivot element can either be

1 Si or Sj : its probability is
2
j−i+1
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si n X
X 2
and Sj . Then, =
j −i +1
i=1 j>i
E[Xij ] = pij ×1+(1−pij )×0 = pij n n−i+1
X X 1
= 2
k
i=1 k=1
Pivot element can either be
1 Si or Sj : its probability is
2
j−i+1
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si n X
X 2
and Sj . Then, =
j −i +1
i=1 j>i
E[Xij ] = pij ×1+(1−pij )×0 = pij n n−i+1
X X 1
= 2
k
i=1 k=1
n X
n
Pivot element can either be X 1
≤ 2
1 Si or Sj : its probability is k
2 i=1 k=1
j−i+1
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si n X
X 2
and Sj . Then, =
j −i +1
i=1 j>i
E[Xij ] = pij ×1+(1−pij )×0 = pij n n−i+1
X X 1
= 2
k
i=1 k=1
n X
n
Pivot element can either be X 1
≤ 2
1 Si or Sj : its probability is k
2 i=1 k=1
j−i+1
= 2nHn
2 Sq from i < q < j
3 Sr from r < i or j < r

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Randomized Quick Sort

n X
X n X
X
E[Xij ] = pij
Let pij denote the probability i=1 j>i i=1 j>i
of comparison between Si n X
X 2
and Sj . Then, =
j −i +1
i=1 j>i
E[Xij ] = pij ×1+(1−pij )×0 = pij n n−i+1
X X 1
= 2
k
i=1 k=1
n X
n
Pivot element can either be X 1
≤ 2
1 Si or Sj : its probability is k
2 i=1 k=1
j−i+1
= 2nHn
2 Sq from i < q < j
3 Sr from r < i or j < r = O(n ln n)

as Hn ∼ ln n + Θ(1)

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 7/8
Thank You!

Thank you very much for your attention!

Queries ?

Design & Analysis of Algo. (CS F364) T Th S (12-1PM) 6164@BITS-Pilani Lecture-kt05 (Jan 31, 2017) 8/8