CH9.

Theory of NP
I cant find an efficient algorithm,
I guess Im just too dumb.

Serious damage to your position within the company !!!
I cant find an efficient algorithm,
because no such algorithm is possible!

Unfortunately, proving intractability of prob can be just
as hard/difficult as finding efficient algorithms !!!

No hope !!!
I cant find an efficient algorithm,
but, neither can all these famous people
Anyway, we have to solve this problem.
Can we satisfy with a reasonable algo ?
Theory of NP
Solving a problem:
Design an efficient algorithm
Analysis it with Lower bound

If cannot find efficient/good algo?
NP-Complete Prob?
If NP_ Complete Prob, approximate(reasonable) algo

Theory of NP
Classification

Turings Halting Prob
Most people believe P = NP !!!
NP
P ? NP-Complete !!!
9.1. Intractability
Intractable difficult to treat or work
(Def) polynomial-time algorithm (worst-case)







Tractable(easy) Problem : Good/efficient algorithm exist
Intractable(difficult) Problem : No good algorithm
(O(p(n))) ) n ( O W(n)
) 1 ( o
e
n! , 2 , 2 , 2 , n
n nlogn, , n n 5 n, 4 n 3 n, 2
n n 0.01 n n
100 10 3
+ +
)
`

e
e
O(p(n)) W(n)
O(p(n)) W(n)
: algorithm "bad"
: orithm "Good" alg
Intractability
Intractable Problem
No polynomial-time(good) algo
I cannot find a polynomial time algo

an exp. time(bad) algo

Eg. 0/1-Knapsack, n-queens, TSP, HP,
Eg. Chained Matrix Mult. Prob
Recursion exp. time algo
Dynamic Programming p-time algo O(n
3
)
n = 100, 10
-9
sec./op.
W(n) = n
3
: 1 millisecond
W(n) = 2
n
: 10
90
years
Intractability
3 categories
1) Problems proven to have P-time algo (tractable)
2) Problems proven not to have P-time algo (intractable)
3) Problems neither 1) nor 2)
9.3. Three Categories of Problems
(1) problems in P polynomial-time algorithms found
sorting, searching, matrix mult.,
(2) problems not in P proven to be intractable
() non-polynomial amount of output
Determine all Hamiltonian Circuits (n-1) ! circuits
() Halting problem - partially solvable

relatively few such problems

(3) Neither proven to be tractable, Nor proven to be intractable
TSP, 0/1-Knapsack, HC, m-coloring(m3),
9.4. Theory of NP
Optimization problems
Decision problems : yes or no

Eg. TSP optimization problem : optimal tour
TSP decision problem :

a tour with cost d ?

0/1 Knapsack O.P. : max total profit
0/1 Knapsack D.P. :

set with profit p ?

Graph coloring O.P. : chromatic #
Graph coloring D.P. : colorable with m ?
Theory of NP
Clique problem:
Clique : complete subgraph


Maximal Clique : maximal size
clique O.P. : maximal size
clique D.P. : k ?



Theory of NP
An solution for O.P.
A solution for D.P. Easy!
But, sol. for D.P. sol. for O.P. ? No!

Polynomial-time algo for O.P.
P-time algo for D.P.

Theory of NP
The Sets P and NP

(Def) P : set of poly. time solvable D.P.s

Eg. Sorting, Searching, MM, CMM, MST,

TSP D.P ? 0/1 Knapsack D.P. ?
Graph coloring D.P. ? Clique D.P. ?

Nobody knows whether in P or not in P !!!
Almost probably not in P
Theory of NP
Polynomial-time verification
Possibly difficult to solve a D.P.,
but easy to verify it
(TSP, 0/1-K.S., Graph C., Clique, )

Eg. TSP
Given a tour with cost d,
the tour can be verified in p-time.
Theory of NP
Non-deterministic Algorithm
1. Guessing (Non-deterministic) stage :
Given an instance,
guess a correct solution in unit time if exists
2. Verification (Deterministic) stage:
Verify the guessed solution

Eg. TSP: Digraph & d ( tour d ? )
Guessing Stage : A tour d
Verifying Stage : verify it is a tour & d
Theory of NP
(Def) Poly. time non-deterministic algo:
Non-det. algo s.t. verification in p-time

(Def) NP : the set of all problems
solvable by p-time non-det. algo
solvable in p-time by non-det. Turing machine

Eg. TSP, 0/1 K.S., G.C., Clique NP
Easy to verify, but possibly hard to solve
Easy verification Easy solution

P NP : O
P = NP ? Most probably No!
:

Theory of NP
Why easy verification easy sol?
Exponential # candidates
Unit time guessing is not realistic
(useful only for problem classification)
Almost all important problems are in NP
Why most researcher believe in P NP ?
1. Too many problems cannot be proven to be in P
2. Only one of them is in P
All of them are in P, i.e, P = NP
c
=
NP-Complete Problems
TSP 0/1-Knapsack
D.P. (n
2
2
n
) (min(2
n
, nW))
B & B (n-1)! leaves 2
n

TSP seems more difficult(intractable)
But, polynomially equivalent

Actually, only one of NP-C probs is p-time solvable,
so are all of them
NP-Complete Problems
CNF-satisfiability prob
Logical variables : x, y, x
1
, x
2
,
Literal : x or
Clause : v (or)

CNF(Conjunctive Normal Form)

) (
3 2 1
x x x v v
) ( ) ( ) (
4 3 1 4 1 3 2 1
x x x x x x x x v v . v . v v
x
NP-Complete Problems
CNF-satisfiability decision prob

a satisfying truth assignment?

Eg.
yes since x
1
=T, x
2
= x
3
= F.


CNF NP
CNF P ? Most probably Not!

Cook(1971) : If CNF P, then P = NP
polynomial-time reducibility
2 1 2 1
) ( x x x x . . v
2 3 2 2 1
) ( ) ( x x x x x . v . v
e
e
no
NP-Complete Problems
Transformation algorithm :
Want to solve prob A
Have an algo to solve prob B

an algo to transform an instance x of A

to an instance y of B s.t. x is yes iff y is yes
To solve A,
1. Transform x y
2. Apply algo for B
x
y=trans(x)
Algo. for
Prob. B
trans
Yes
or
No
Algorithm for problem A
NP-Complete Problems
(Def) A is p-time (many-one) reducible to B,
if p-time trans. algo from A to B
(A B)

Theorem 9.1
If prob B P and A B
A e P

NP-Complete Problems
(Def) A problem B is NP-complete if
1. B e NP, and
2.

A e NP, A B

Thm 9.2: (Cooks Theorem) CNF is NP-C

Thm 9.3: A problem C is NP-C if
1. CeNP, and
2. for a NP-C problem B, B C
NP-Complete Problems
Eg. Clique decision prob is NP-complete
(Proof) CNFClique

Given k, verify whether it is a clique

) ( ) ( ) (
3 2 3 2 1 2 1
X X X X X X X B v . v v . v =
) 2 , (
1
X
) 2 , (
2
X ) 2 , (
3
X
) 1 , (
1
X
) 1 , (
2
X
) 3 , (
3
X
) 3 , (
2
X
9.5. Handling NP-Hard Problems
Dynamic Programming
Backtracking
Branch & Bound

exponential-time(bad) in W.C.

approximation(Heuristic/reasonable) algo
ex. Greedy