Design and Analysis of Algorithms

Lecture 9:
Basic Sorting Algorithms

Tauseef Iftikhar

Department of Computer Science

Government College University, Lahore.
Todays Agenda

Sorting Problem

Insertion Sort Analysis

Bubble Sort Analysis

Selection Sort Analysis

Merge Analysis
Sorting

I Sorting the sequence is to arrange its elements such that they

are in increasing (or decreasing) order
I If the sequence is stored in 1-dimensional array then a sorted
array is: a1 a2 . . . an or a1 a2 . . . an
I Types of sorting:
I Internal sort: the sequence is stored in a random access
memory
I External sort: the sequence could be stored in a secondary
storage
We will study only internal sorting algorithms
Properties of Sorting Algorithms

I Efficiency: use minimum resources

I In place: use no extra space in memory (at most for single
element)
I Stability: order of same elements in unsorted array must be
preserved in sorted array.
I Naturalness: at most as much operations should be
performed as much elements are out of order.
Insertion sort: Running time analysis

Algorithm 1 Insertion sort Line no. Iterations

1: for j 2, n do 1 2n
2: key A[j] 2 n1
3: i j 1 3 Pn 1
n
4: while i > 0 AND A[i] > key do 4 f (j)
Pj=2
n
5: A[i + 1] A[i] 5 f (j) 1
Pj=2
n
6: i i 1 6 j=2 f (j) 1
7: end while 7
8: A[i + 1] key 8 n1
9: end for 9
Pn Pn Pn
T (n) = 5n 3 + j=2 f (j) + j=2 (f (j) 1) + j=2 (f (j) 1)
Insertion sort: Running time analysis

Best-case: When array is already sorted

f (j) = 1
T (n) = 5n 3 + nj=2 1 + nj=2 1 1 + nj=2 1 1
P P P
T (n) = 5n 3 + n 1 + n 2 + n 2
T (n) = 8n 8
T (n) = (n)
Worst-case: When array is in reverse order
f (j) = j
T (n) = 5n 3 + nj=2 j + nj=2 j 1 + nj=2 j 1
P P P

since nj=2 j = n(n+1) n(n1)

P Pn
2 1 and j=2 j 1 = 2
T (n) = 5n 3 + n(n+1)
2 1+ n(n1)
2 + n(n1)
2
T (n) = 5n 3 + O(n )2

T (n) = O(n2 )
Bubble sort: Running time analysis

Algorithm 2 Bubble sort Line no. Iterations

1: for i 1, n do 1 2(n + 1)
n
for j n, i + 1 do
P
2: 2 Pi=1 f (i)
n
3: if A[j] < A[j 1] then 3 i=1 f (i) 1
4: A[j] A[j 1] 4 g (n)
5: end if 5
6: end for 6
7: end for

Pn Pn
T (n) = 2n + 2 + i=1 f (i) + i=1 f (i) 1 + g (n)
Bubble sort: Running time analysis

We see that both loop will run mandatory. Inner loop depends on
outer loop.
f (i) = n i + 1
P n
i=1 (n i +P1)
= Pni=1 n Pni=1 i + Pni=1 1
P P
= ni=1 n ni=1 i + ni=1 1
= n2 n(n+12 +n
2 n2 +n
=n 2 +n
2
= n2 n2 n2 + n
2
= n2 + n2 = n(n+1)
2
Running time:
T (n) = 2n + 2 + n(n+1)
2 + n(n+1)
2 1 + g (n)
Bubble sort: Running time analysis

Best-case: When array is already sorted

g (n) = 0
T (n) = 2n + 2 + n(n+1)
2 + n(n+1)
2 1+1
n(n+1) n(n+1)
T (n) = 2n + 3 + 2 + 2 1
T (n) = (n2 )

Worst-case: When array is in reverse order

g (n) = n(n+1)
2 1
T (n) = 2n + 2 + n(n+1)
2 + 2 n(n+1)
2 1
T (n) = O(n )2

Hence T (n) = (n2 )

Selection sort: Running time analysis

Algorithm 3 Selection sort

1: for j 1ton 1 do
Line no. Iterations
2: smallest j
1 2n
3: for i j + 1, n do
2 n1
4: if A[i] < A[smallest] then Pn1
3 f (j)
5: smallest i Pj=1
n1
6: end if
4 j=1 f (j) 1
7: end for 5 g (n)
8: A[j] A[smallest] 8 n1
9: end for

Pn Pn
T (n) = 4n 2 + j=1 f (j) + j=1 f (j) 1 + g (n)
Selection sort: Running time analysis

We see that both loop will run mandatory. Inner loop depends on
outer loop.
f (j) = n j + 1
P n
P (n j +
i=1 P1)
= nj=1 n nj=1 i + nj=1 1
P

= n2 n(n+1
2 +n
2 n2 +n
=n 2 +n
2
= n2 n2 n2 + n
2
= n2 + n2 = n(n+1)
2
Running time:
T (n) = 4n 2 + n(n+1)
2 + n(n+1)
2 1 + g (n)
Selection sort: Running time analysis

Best-case: When array is already sorted

g (n) = 0
T (n) = 4n 2 + n(n+1)
2 + n(n+1)
2 1
T (n) = (n2 )

Worst-case: When array is in reverse order

g (n) = n(n+1)
2 1
T (n) = 4n 2 + n(n+1)
2 + n(n+1)
2 1+ n(n+1)
2 1
T (n) = O(n )2

Hence T (n) = (n2 )