Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
What is an Algorithm?
Algorithms are the ideas behind computer programs.
An algorithm is the thing that stays the same whether the
program is in C++ running on a Cray in New York or is in
BASIC running on a Macintosh in Alaska!
To be interesting, an algorithm has to solve a general, specified
problem.
What is a problem?
Definition
A mapping/relation between a set of input instances
(domain) and an output set (range)
Problem Specification
Specify what a typical input instance is
Specify what the output should be in terms of the input
instance
Example: Sorting
Input: A sequence of N numbers a1an
Output: the permutation (reordering) of the input
sequence such that a1 a2 an .
Types of Problems
Search: find X in the input satisfying property Y
Structuring: Transform input X to satisfy property Y
Construction: Build X satisfying Y
Optimization: Find the best X satisfying property Y
Decision: Does X satisfy Y?
Adaptive: Maintain property Y over time.
Efficiency
Computes correct output quickly given input
Correctness
Example: Traveling Salesperson Problem (TSP)
Input: A sequence of N cities with the distances dij
between each pair of cities
Output: a permutation (ordering) of the cities <c1, , cn>
that minimizes the expression
j =1 to n-1 dj,j+1 + dn,1
Efficiency
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
Course Objectives
1.
2.
3.
4.
5.
Classic Algorithms
Algorithm correctness
Loop invariants
Analyzing algorithms
Clear Writing
Methods for Expressing Algorithms
Implementations
Pseudo-code
English
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
Asymptotic analysis
Asymptotic notation
Input Size
Asymptotic analysis
Asymptotic notation
Measuring Complexity
F(1) = 3
F(2) = 5
F(3) = 7
F(n) = 2n+1
3 different analyses
The worst case running time of an algorithm is the function
defined by the maximum number of steps taken on any
instance of size n.
The best case running time of an algorithm is the function
defined by the minimum number of steps taken on any
instance of size n.
The average-case running time of an algorithm is the function
defined by an average number of steps taken on any instance
of size n.
Which of these is the best to use?
Drawbacks
Quicksort
simplex method for linear programming
Asymptotic analysis
Asymptotic notation
Simplifications
Ignore constants
4n2 - 3n log n + 17.5 n - 43 n + 75 becomes
n2 n log n + n - n + 1
Asymptotic Efficiency
n2 n log n + n - n + 1 becomes n2
Simplification of analysis
Much easier to analyze if we focus only on n2 rather
than worrying about 3.7 n2 or 3.9 n2
Asymptotic Analysis
We focus on the infinite set of large n
ignoring small values of n
Usually, an algorithm that is asymptotically
more efficient will be the best choice for all
but very small inputs.
0
Big Oh Notation
O(g(n)) =
{f(n) : there exist positive constants c and n0 such
that nn0, 0 f(n) c g(n) }
What are the roles of the two constants?
n0:
c:
0
n0
f(n) c g(n)
O(g(n))
(g(n))
(g(n))
1
100
( g (n))
1
25
n2
Example Function
f(n) = 3n - 100n + 6
2
Quick Questions
c
3n2 - 100n + 6 = O(n2)
3n2 - 100n + 6 = O(n3)
3n2 - 100n + 6 O(n)
3n2 - 100n + 6 = (n2)
3n2 - 100n + 6 (n3)
3n2 - 100n + 6 = (n)
3n2 - 100n + 6 = (n2)?
3n2 - 100n + 6 = (n3)?
3n2 - 100n + 6 = (n)?
n0
10
20
110-5 sec
n2
0.025 sec
0.036 sec
n3
0.001 sec
0.008 sec
0.027 sec
0.064 sec
0.125 sec
0.216 sec
n5
0.1 sec
3.2 sec
24.3 sec
1.7 min
5.2 min
13.0 min
2n
0.001sec
1.0 sec
17.9 min
12.7 days
35.7 years
366 cent
3n
0.59sec
58 min
6.5 years
3855 cent
2108cent
1.31013cent
log2 n
310-6 sec
510-6 sec
610-6 sec
610-6 sec
30
40
50
410-5 sec
60
Example Problems
1. What does it mean if:
f(n) O(g(n)) and g(n) O(f(n)) ?
2. Is 2n+1 = O(2n) ?
Is 22n = O(2n) ?
3. Does f(n) = O(f(n)) ?
4. If f(n) = O(g(n)) and g(n) = O(h(n)),
can we say f(n) = O(h(n)) ?
Extra Slides
Slides illustrating TSP algorithms
Case study with Insertion sort for Best,
Average, Worst case analysis
Possible Algorithm:
Nearest neighbor
Not Correct!
A Correct Algorithm
Try all possible orderings of the points selecting the
ordering that minimizes the total length:
d=
For each of the n! permutations, Pi of the n points,
if cost(Pi) < d then
d = cost(Pi)
Pmin = Pi
return Pmin