Design and Analysis of Algorithms

Lecture 6:
Efficiency Analysis

Tauseef Iftikhar

Department of Computer Science

Government College University, Lahore.
Todays Agenda

What is efficiency analysis?

Measuring time efficiency

Best case, worst case and average case analysis

Efficiency Analysis

I The amount of computing resources needed to execute the

algorithm
I it is also known as complexity analysis
Efficiency Analysis

I The amount of computing resources needed to execute the

algorithm
I it is also known as complexity analysis
I Computing resources:
I Memory space: space required to store the data processed by
algorithm
I CPU time: also known as running time. It is time needed to
execute the operations of the algorithm.
I An efficient algorithm uses a minimum amount of computing
resources.
Efficiency Analysis

I There are two types of efficiency analysis

I Space analysis: how much space an algorithm required to store
the data in memory?
I Running time analysis: it refers to how fast an algorithm runs
I The key strategy in efficiency analysis is the amount of
computing resources depends on the input size (problem size)
I Input size: the number of elements belonging to the input
data.
I Hence the main question to be answered by efficiency analysis
is how depends the time and/or space needed by algorithm on
the input size?
Space and time trade off

I Often we have to make a compromise between space

efficiency and time efficiency
I Example: adjacency matrix vs adjacency list for graph
representation.
I A sparse graph can be represented by an adjacency matrix
which would be time efficient for traversing an edge but it will
be at the cost of space
I A sparse graph can be represented by an adjacency list which
would be space efficient and but it would take longer time for
traversing the edges.
How can be time efficiency measured?

Our efficiency measure for running time must be independent of

I machine
I programming language
I programmer
How can be time efficiency measured?

Our efficiency measure for running time must be independent of

I machine
I programming language
I programmer
Running time is a function of input size denoted by T (n) where n
is input size.
RAM: Computational Model
However to estimate running time we must use some
computational model.
RAM: Computational Model
However to estimate running time we must use some
computational model.
Computational model is an abstract machine having some
properties.
RAM: Computational Model
However to estimate running time we must use some
computational model.
Computational model is an abstract machine having some
properties.
For this purpose we will use RAM computational model(Random
Access Machine) having the following properties
1. All processing steps are sequentially executed (there is no
parallelism in the execution of the algorithm)
2. The time of executing the basic operations does not depend
on the values of the operands (there is no time difference
between computing 1+2 and computing 12433+4567)
3. The time to access data does not depend on their address
(there are no differences between processing the first element
of an array and processing the last element)
4. All basic operations, assignment, arithmetic, logical, relational
take unit time.
Example1: running time

Now we see how running time expresses the dependence of the

number of executed operations on the input size

Example: swapping iterations cost

aux x 1 c1
x y 1 c1
y x 1 c1
T (n) = 3 c1

where c1 is some constant and 3c1 is also some constant. We

conclude that running time of this algorithm is some constant.
Hence we can understand that the running time of above algorithm
is independent of the input size.
Example2: running time

Example 2: Compute sum of the series s = 1 + 2 + . . . + n

precondition: n 1 postcondition: s = 1 + 2 + . . . + n

input: n
line no. cost iterations
1 c1 1
1: s0 2 c1 1
2: i 1 3 c2 n+1
3: while i n 4 c3 n.1
do 5 c3 n.1
4: s s +i
5: i i +1
6: end while
Example2: running time

Example 2: Compute sum of the series s = 1 + 2 + . . . + n

precondition: n 1 postcondition: s = 1 + 2 + . . . + n

input: n
line no. cost iterations
1 c1 1
1: s0 2 c1 1
2: i 1 3 c2 n+1
3: while i n 4 c3 n.1
do 5 c3 n.1
4: s s +i
T (n) = 2c1 + c2(n + 1) + 2c3n
5: i i +1
6: end while
Example2: running time

Example 2: Compute sum of the series s = 1 + 2 + . . . + n

precondition: n 1 postcondition: s = 1 + 2 + . . . + n

input: n
line no. cost iterations
1 c1 1
1: s0 2 c1 1
2: i 1 3 c2 n+1
3: while i n 4 c3 n.1
do 5 c3 n.1
4: s s +i
T (n) = 2c1 + c2(n + 1) + 2c3n
5: i i +1
T (n) = 2c1 + c2.n + c2 + 2c3.n
6: end while
Example2: running time

Example 2: Compute sum of the series s = 1 + 2 + . . . + n

precondition: n 1 postcondition: s = 1 + 2 + . . . + n

input: n
line no. cost iterations
1 c1 1
1: s0 2 c1 1
2: i 1 3 c2 n+1
3: while i n 4 c3 n.1
do 5 c3 n.1
4: s s +i
T (n) = 2c1 + c2(n + 1) + 2c3n
5: i i +1
T (n) = 2c1 + c2.n + c2 + 2c3.n
6: end while
T (n) = a.n + b
Example2: running time

Example 2: Compute sum of the series s = 1 + 2 + . . . + n

precondition: n 1 postcondition: s = 1 + 2 + . . . + n

input: n
line no. cost iterations
1 c1 1
1: s0 2 c1 1
2: i 1 3 c2 n+1
3: while i n 4 c3 n.1
do 5 c3 n.1
4: s s +i
T (n) = 2c1 + c2(n + 1) + 2c3n
5: i i +1
T (n) = 2c1 + c2.n + c2 + 2c3.n
6: end while
T (n) = a.n + b

Running time is linear function of input size

Example3: running time

Example 3: Find the minimum in non-empty array x[1 . . . n]

P: n 1 Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]

1: m x[1] line no. cost iterations

2: for i 2, n do 1 1 1
3: if x[i] < m 2 1 2n
then 3 1 n1
4: m x[i] 4 1 h(n)
5: end if
6: end for
7: return m
Example3: running time

Example 3: Find the minimum in non-empty array x[1 . . . n]

P: n 1 Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]

1: m x[1] line no. cost iterations

2: for i 2, n do 1 1 1
3: if x[i] < m 2 1 2n
then 3 1 n1
4: m x[i] 4 1 h(n)
5: end if T (n) = 1 + 2n + n 1 + h(n)
6: end for
7: return m
Example3: running time

Example 3: Find the minimum in non-empty array x[1 . . . n]

P: n 1 Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]

1: m x[1] line no. cost iterations

2: for i 2, n do 1 1 1
3: if x[i] < m 2 1 2n
then 3 1 n1
4: m x[i] 4 1 h(n)
5: end if T (n) = 1 + 2n + n 1 + h(n)
6: end for T (n) = 3n + h(n)
7: return m
Example3: running time

Example 3: Find the minimum in non-empty array x[1 . . . n]

P: n 1 Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]

1: m x[1] line no. cost iterations

2: for i 2, n do 1 1 1
3: if x[i] < m 2 1 2n
then 3 1 n1
4: m x[i] 4 1 h(n)
5: end if T (n) = 1 + 2n + n 1 + h(n)
6: end for T (n) = 3n + h(n)
7: return m
The running time depends not only on n but also on the properties
of input data
Best-case analysis and worst-case analysis
Whenever analysis of an algorithm not only depends on the input
size but also on some property of input data then we have to
perform analysis in more details
I Worst-case analysis: gives the longest running time for any
input of size n.
I Best-case analysis: gives us the minimum running time for
any input of size n.
I worst-case running time of an algorithm gives us an upper
bound on the running time for any input.
I Knowing it provides a guarantee that the algorithm will never
take any longer.

I best-case running time of an algorithm gives us lower bound

on the running time for any input.
I Knowing it provides a guarantee that the algorithm will never
take more less time.
Example 4: sequential search

Preconditions: x[1..n], n >= 1, v a value

Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v

Algorithm 1 search
1: found true
2: i 1
3: while (found = false) and (i n)
do
4: if x[i] = v then
5: found true
6: else
7: i i +1
8: end if
9: end while
Example 4: sequential search

Preconditions: x[1..n], n >= 1, v a value

Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v

Algorithm 2 search
line no cost
1: found true
1 1
2: i 1
2 1
3: while (found = false) and (i n)
3 f (n) + 1
do
4 f (n)
4: if x[i] = v then
5 g (n)
5: found true
7 h(n)
6: else
7: i i +1
T(n)=
8: end if
3 + 2f (n) + g (n) + h(n)
9: end while
Example 4: sequential search

Preconditions: x[1..n], n >= 1, v a value

Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v

Algorithm 3 search
line no cost
1: found true
1 1
2: i 1
2 1
3: while (found = false) and (i n)
3 f (n) + 1
do
4 f (n)
4: if x[i] = v then
5 g (n)
5: found true
7 h(n)
6: else
7: i i +1
T(n)=
8: end if
3 + 2f (n) + g (n) + h(n)
9: end while
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
f (n) =
n, if v
/ x[1..n]
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
f (n) =
n, if v
/ x[1..n]
(
1, if v x[i..n]
g (n) =
0, if v
/ x[1..n]
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n] T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v
/ x[1..n]
(
1, if v x[i..n]
g (n) =
0, if v/ x[1..n]
(
k 1, if v x[i..n]
h(n) =
n, if v
/ x[1..n]
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n] T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v / x[1..n] Best case: x[1] = v
( f (n) = 1, g (n) = 1, h(n) = 0
1, if v x[i..n] T (n) = 6
g (n) =
0, if v / x[1..n]
(
k 1, if v x[i..n]
h(n) =
n, if v
/ x[1..n]
Example 4: sequential search

The running time depends on the properties of the array.

Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n] T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v / x[1..n] Best case: x[1] = v
( f (n) = 1, g (n) = 1, h(n) = 0
1, if v x[i..n] T (n) = 6
g (n) = Worst case: v / x[1..n]
0, if v / x[1..n]
( f (n) = n, g (n) = 0, h(n) = n
k 1, if v x[i..n] T (n) = 3 + 2n + n = 3 + 3n
h(n) =
n, if v
/ x[1..n]