Data Structures and

Algorithms
Lecture 7. Basic sorting algorithms
 Sorting problem

Input
A sequence of n numbers a1, a2, . . . , an .

Output
A permutation (reordering) a1, a2, . . . , an of the
input sequence such that a1 a2 . . . an

The numbers are referred to as keys.

The sequences are typically stored in arrays.
 Insertion sort

An efficient algorithm for sorting a small number of

elements.
In insertion sort, an array A[1..n] containing a
sequence of n that is to be sorted is given as a
parameter.
Insertion sort works the way many people sort a
bridge or gin rummy hand.
We start with an empty left hand and the cards face
down on the table.
We then remove one card at a time from the table
and insert it into the correct position in the left hand.
To find the correct position for a card, we compare it
with each of the cards already in the hand, from
right to left,
Insertion-sort procedure
 Example

The operation of INSERTION-SORT on the array A = [5, 2, 4, 6,1, 3].

Array indices appear above the rectangles.
Values stored in the array positions appear within the rectangles.

(a)-(e) The iterations of the for loop of lines 1-8. In each iteration, the
black rectangle holds the key taken from A[j], which is compared with
the values in shaded rectangles to its left in the test of line 5. Shaded
arrows show array values moved one position to the right in line 6, and
black arrows indicate where the key is moved to line 8.
(f) The final sorted array.
 Running time

The time cost of each statement and the number of times each
statement is executed.

The running time of the algorithm is the sum of running times

for each statement executed.
To compute T(n), the running time of INSERTION-SORT, we
sum the products of the cost and times columns.
Running time
 Insertion Sort Analysis Best case

Best case: The array is already sorted.

Always find that A[i ] key upon the first time the
while loop test is run (when i = j 1).
All tj are 1.
Running time is

T(n) = c1n + c2(n 1) + c4(n 1) + c5(n 1) + c8(n 1)

= (c1 + c2 + c4 + c5 + c8)n (c2 + c4 + c5 + c8) .

Can express T(n) as an + b for constants a and b

(that depend on the statement costs ci ).
T(n) is a linear function of n.
Insertion Sort Analysis Worst case

Worst case: The array is in reverse sorted order.

 Insertion Sort Analysis

Best Case O(n) Array is already sorted

Average Case O(n2)
Worst Case O(n2) The array is in reverse order
 Selection Sort

Selection sort orders a list of values by repetitively

putting a particular value into its final position

More specifically:
find the smallest value in the list
switch it with the value in the first position
find the next smallest value in the list
switch it with the value in the second position
repeat until all values are in their proper places
Example
 Selection Sort in Pseudocode

SelectionSort(A)
// GOAL: place the elements of A in ascending order
1 n := length[A]
2 for i := 1 to n
3 // GOAL: place the correct number in A[i]
4 j := FindIndexOfSmallest( A, i, n )
5 swap A[i] with A[j]
// L.I. A[1..i] the i smallest numbers sorted
6 end-for
7 end-procedure

FindIndexOfSmallest( A, i, n )
// GOAL: return j in the range [i,n] such
// that A[j]<=A[k] for all k in range [i,n]
1 smallestAt := i ;
2 for j := (i+1) to n
3 if ( A[j] < A[smallestAt] ) smallestAt := j
// L.I. A[smallestAt] smallest among A[i..j]
4 end-for
5 return smallestAt
6 end-procedure
 Running time

Selecting the lowest element requires scanning all

n elements (this takes n - 1 comparisons) and
then swapping it into the first position.
Finding the next lowest element requires scanning
the remaining n - 1 elements and so on, for (n - 1)
+ (n - 2) + ... + 2 + 1 = n(n - 1) / 2 = (n2)
comparisons.
Each of these scans requires one swap for n - 1
swaps (the final element is already in place).
Thus, the comparisons dominate the running time,
which is (n2).
 Bubble Sort

Suppose we have an array of data which is

unsorted:
starting at the front, traverse the array, find the
largest item, and move (or bubble) it to the top
with each subsequent iteration, find the next largest
item and bubble it up towards the top of the array
 Bubble Sort

Starting with the first item, assume that it is the

largest
Compare it with the second item:
if the first is larger, swap the two,
otherwise, assume that the second item is the
largest
Continue up the array, either swapping or
redefining the largest item
 Bubble Sort

After one pass, the largest item must be the last in

the list
Start at the front again:
the second pass will bring the second largest
element into the second last position
Repeat n 1 times, after which, all entries will be
in place
 Bubble Sort: Example

Consider the unsorted

array to the right
We start with the element
in the first location, and
move forward:
if the current and next
items are in order,
continue with the next
item, otherwise
swap the two entries
 Bubble Sort: Example

After one loop, the

largest element is in the
last location
Repeat the procedure
 Bubble Sort: Example

Now the two largest

elements are at the end
Repeat again
 Bubble Sort: Example

With this loop, 12 is

brought to its appropriate
location
 Bubble Sort: Example

Finally, we swap the last

two entries to order them
At this point, we have a
sorted array
 Pseudocode

procedure bubbleSort( A : list of sortable

items ) defined as:
n := length( A )
do
swapped := false
n := n - 1
for each i in 0 to n do:
if A[ i ] > A[ i + 1 ] then
swap( A[ i ], A[ i + 1 ] )
swapped := true
end if
end for
while swapped
end procedure
 Worst-case Run-time Analysis

With the first loop, we must make n 1

comparisons
This equals the maximum number of swaps, thus
we need only consider this case
The total number of comparisons is:

n 1
n 1 n

k 1
n k n n 1
2
 Worst-case Run-time Analysis

Consequently, the worst-case run-time of bubble

sort is O(n2)
The average case is no different: the form of the
algorithm requires O(n2) comparisons
 Observation

Any algorithm which relies on pair-wise swaps

must consequently have an average run time of
O(n2)