Sei sulla pagina 1di 25

Design and Analysis of Algorithm

The following are the topics to be discussed under


Limitations of algorithm power:

Definition of lower-bound arguments


Different algorithms using decision tree
Application of P, NP and NP-complete
problems

Limitations of algorithm Power

* Property of STI
Page 1 of 25

Design and Analysis of Algorithm

A reasonable evaluation of algorithms as problemsolving tools is inevitable.


These algorithms are considered as powerful
instruments, particularly when these are
processed by modern computers.
Sometimes there are problems that cannot be
solved by an algorithm or the problems seem not
to have efficient solutions.

Limitations of algorithm Power

* Property of STI
Page 2 of 25

Design and Analysis of Algorithm

Lower bound :is a comparison-based algorithm


where the behavior of the algorithm is based only
on the comparison between elements
[www.cs.toronto.edu].
The lower bound deals with the complexity of a
problem instead of the algorithm.
Basically we have to prove that no algorithm, no
matter how complex, can do better than our
bound

Limitations of algorithm Power

* Property of STI
Page 3 of 25

Design and Analysis of Algorithm

Examples:
number of comparisons needed to find the largest
element in a set of n numbers
number of comparisons needed to sort an array of
size n
number of comparisons necessary for searching in
a sorted array
number of multiplications needed to multiply two
n-by-n matrices

Limitations of algorithm Power

* Property of STI
Page 4 of 25

Design and Analysis of Algorithm

Two ways to examine the algorithm efficiency:


Establish the asymptotic efficiency class similar to
the worst case
Single out where this class stands with respect to
the hierarchy of efficiency class

Limitations of algorithm Power

* Property of STI
Page 5 of 25

Design and Analysis of Algorithm

Trivial lower bound: is based on counting the


number of items that must be processed in input
and generated as output [LEV07].
Examples:
finding max element -- n steps or n/2
comparisons
polynomial evaluation
sorting
element uniqueness
Hamiltonian circuit existence
Conclusions:
may and may not be useful
be careful in deciding how many elements
must be processed

Limitations of algorithm Power

* Property of STI
Page 6 of 25

Design and Analysis of Algorithm

Information-theoretic arguments (decision trees) :


seek to establish a lower bound based on the
amount of information it has to produce [LEV07].
Example:
Deducing a positive integer between 1 and n
selected by somebody by asking that person
questions with yes/no answers.
Conclusions:
The approach is related to information theory
because it has proved to be useful for finding
the information-theoretic bounds in different
problem types concerning comparisons, such
as sorting and searching.
Its core concept can be apprehended more
precisely through decision trees.

Limitations of algorithm Power

* Property of STI
Page 7 of 25

Design and Analysis of Algorithm

Adversary arguments: Malevolent simply means


that it will change a problem to force algorithm
down worst path and we can say that it is fair
because we must remain consistent with the work
that is already done for selection problems.
Examples:
Three-Card Monte
n-Card Monte
Finding Patterns in Bit Strings
Evasive Graph Properties
Conclusions:
The adversary should be thought of as a very
powerful, clever being that is trying to make
your algorithm run as slowly as possible.
The adversary cannot "read the algorithm's
mind", but it can try to be prepared for
anything the algorithm does.

Limitations of algorithm Power

* Property of STI
Page 8 of 25

Design and Analysis of Algorithm

Problem reduction: proves a lower bound by


transforming a problem that already has an
established lower bound into the new problem.
Examples:
Reduction to linear programming
Algorithmic problem solving
Reduction to graph problems
Evasive Graph Properties

Limitations of algorithm Power

* Property of STI
Page 9 of 25

Design and Analysis of Algorithm

Conclusion:
We must prove that an arbitrary instance of
problem Q can be transformed in a logically
efficient manner to an instance of problem P.
As a result, any algorithm solving P would solve Q
as well. Therefore, the lower bound for Q will also
be the lower bound for P.
The figure below are the list of problems used for
establishing lower bounds by problem reduction:

Limitations of algorithm Power

* Property of STI
Page 10 of 25

Design and Analysis of Algorithm

Decision tree : is a convenient model of algorithms


involving comparisons in which internal nodes
represent comparisons and leaves represent
outcomes (or input cases).
The figure below represents the decision tree of
an algorithm for finding a minimum value of three
numbers:

Limitations of algorithm Power

* Property of STI
Page 11 of 25

Design and Analysis of Algorithm

By studying the properties of decision trees for


comparison-based sorting algorithms, we can
derive important lower bounds on time efficiencies
of such algorithms.
Consider the following figure:

Limitations of algorithm Power

* Property of STI
Page 12 of 25

Design and Analysis of Algorithm

Inequality implies that the height of a binary


decision tree for any comparison-based sorting
algorithm and the worst-case number of
comparisons made such an algorithm cannot be
less than [log2n!]:

Cworst (n) [log2n!]


We can also use the decision trees for analyzing
the average-case behavior of a comparison-based
sorting algorithm.

Limitations of algorithm Power

* Property of STI
Page 13 of 25

Design and Analysis of Algorithm

We can compute the average number of


comparisons for a particular algorithm as the
average depth of its decision trees leaves like the
average path length from the root to the leaves.
The figure shows an example of a decision tree for
the three element in the insertion sort :

Limitations of algorithm Power

* Property of STI
Page 14 of 25

Design and Analysis of Algorithm

An algorithm solves a problem in polynomial time


if its worst-case time efficiency belongs to O(p(n))
time, where p(n) is a polynomial of problems input
size n [LEV07].
Tractable : problems that can be solved in
polynomial time.
Intractable : problems that cannot be solved in
polynomial time.

Limitations of algorithm Power

* Property of STI
Page 15 of 25

Design and Analysis of Algorithm

Reasons for drawing the intractability line :


We cannot solve arbitrary instances of
intractable problems in a reasonable amount
of time unless such instances are very small.
Although there might be a huge difference
between the running time in O(p(n)) for
polynomials of drastically different degrees,
there are very few useful polynomial time
algorithm with the degree of polynomial higher
than three.
Polynomial functions posses many convenient
properties, in particular both the sum and
composition of two polynomials are always
polynomials, too.
The choice of this class led to a development
of an extensive theory called computational
complexity.

Limitations of algorithm Power

* Property of STI
Page 16 of 25

Design and Analysis of Algorithm

Class P: is a class of decision problems that can


be solved in polynomial time by deterministic
algorithms. This class of problems is called
polynomial [LEV07].
The restriction of P to decision problems can be
justified by the following:
It is sensible to exclude problems not solvable
in polynomial time because of their
exponentially large output.
Different important problems that are not
decision problems that are easier to study.

Limitations of algorithm Power

* Property of STI
Page 17 of 25

Design and Analysis of Algorithm

There are many important problems showing the


polynomial time algorithm:
Hamilton circuit
Traveling salesman
Knapsack problem
Partition problem
Bin packing
Graph coloring
Integer linear programming

Limitations of algorithm Power

* Property of STI
Page 18 of 25

Design and Analysis of Algorithm

NP (nondeterministic polynomial): the class of


decision problems whose proposed solutions can
be verified in polynomial time, which is solvable by
a nondeterministic polynomial algorithm.
A nondeterministic polynomial algorithm is an
abstract two-stage procedure that:
generates a solution of the problem (on some
input) by guessing
checks whether this solution is correct in
polynomial time

Limitations of algorithm Power

* Property of STI
Page 19 of 25

Design and Analysis of Algorithm

NP (nondeterministic polynomial): the class of


decision problems whose proposed solutions can
be verified in polynomial time, which is solvable by
a nondeterministic polynomial algorithm.
A nondeterministic polynomial algorithm is an
abstract two-stage procedure that:
generates a solution of the problem (on some
input) by guessing
checks whether this solution is correct in
polynomial time
Sample NP or Nondeterministic algorithm
problems:
Guess a truth assignment
Substitute the values into CNF formula to see
if it evaluates to true

Limitations of algorithm Power

* Property of STI
Page 20 of 25

Design and Analysis of Algorithm

Other Problems in NP
Class NP :is the class of decision problems that
can be solved by nondeterministic polynomial
algorithms. This class of problems is called
nondeterministic polynomial [LEV07].
All the problems in P can also be solved in this
manner (but no guessing is necessary), so we
have:

NP

Limitations of algorithm Power

* Property of STI
Page 21 of 25

Design and Analysis of Algorithm

NP sample problems also includes:


Hamiltonian circuit existence
Partition problem: Is it possible to partition a
set of n integers into two disjoint subsets with
the same sum?
Decision versions of TSP, knapsack problem,
graph coloring, and many other combinatorial
optimization problems.

Limitations of algorithm Power

* Property of STI
Page 22 of 25

Design and Analysis of Algorithm

NP-complete problem: is a problem in NP that is


as difficult as any other problem in this class,
because by definition, any problem in NP can be
reduced to it in polynomial time.
A decision problem D1 is said to be polynomially
reducible to a decision problem D2 if there exists a
function t that transforms instances of D1 to
instances of D2 such that:
t maps all yes instances of D1 to yes instances
of D2 and all no instances of D1 to all no
instances of D2
t is computable by a polynomial time
algorithm.
A decision problem D is NP-complete if it is as
hard as any problem in NP, i.e.,
D is in NP
every problem in NP is polynomial-time reducible
to D

Limitations of algorithm Power

* Property of STI
Page 23 of 25

Design and Analysis of Algorithm

This definition immediately implies that if a


problem D is polynomially reducible to some
problem D that can be solved in polynomial time,
then problem D.

NP problems

know n
NP-complete
problem
candidate
for NP completeness

Limitations of algorithm Power

* Property of STI
Page 24 of 25

Design and Analysis of Algorithm

Other NP-complete problems obtained through


polynomial-time reductions from a known NPcomplete problem, as illustrated in the figure
below.

NP problems

know n
NP-complete
problem
candidate
for NP completeness

Limitations of algorithm Power

* Property of STI
Page 25 of 25