Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
ROAD MAP
Greedy Technique
Knapsack Problem Minimum Spanning Tree Problem
Prims Algorithm Kruskals Algorithm
Greedy Technique
Greedy technique is for solving optimization problems
such as engineering problems
Greedy approach suggests constructing a solution through a sequence of steps called decision steps
Each expanding a partially constructed solution Until a complete solution to the problem is reached
Greedy Technique
On each decision step the choice should be
Feasible has to satisfy the problems constraints Locally optimal has to be the best local choice among all feasible choices available on that step Irrevocable once made, it can not be changed
4
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
5
Greedy Technique
In each step, greedy technique suggests a greedy grasp of the best alternative available
Locally optimal decision Hope to yield a globally optimal solution
Greedy technique does not give the optimal solution for all problems There are problems for which a sequence of locally optimal choices does not yield an optimal solution
EX: TSP, Graph coloring Produces approximate solution
6
Knapsack Problem
Given :
wi : weight of object i m : capacity of knapsack pi : profit of all of i is taken
Find:
xi : fraction of i taken
Feasibility:
n
x w
i i =1
m
n
Optimality: maximize
x p
i i =1
Knapsack Problem
Algorithm Knapsack (m,n) { U = m for i = 1 to n x(i) = 0 for i = 1 to n if (w[i] > U) break x[i] = 1.0 U = U w[i] if (i n) x[i] = U/w[i]
Proof of Optimality G is greedy solution G = x1, x2, , xn let xj 1 j : least index xi = 1 1ij 0 xj 1 xi = 0 j i n O is optimal solution O = y1, y2, , yn let xk yk yk < xk k is the least index
yj yj ys
xj yj + xj yj ys
profit profit
O O'
O O O G
M
O
10
ROAD MAP
Greedy Technique
Knapsack Problem Minimum Spanning Tree Problem
Prims Algorithm Kruskals Algorithm
11
Definition:
A spanning tree of a connected graph is its connected acyclic subgraph A minimum spanning tree of a weighted connected graph is its spanning tree of the smallest weight weight of a tree is defined as the sum of the weights on all its edges
Feasible Solution:
A spanning tree, a subgraph G of G
G ' = (V , E ' )
E' E
12
C (G ' ) =
C (e)
eG '
Optimum Solution :
Minimum cost spanning tree
13
14
Prims Algorithm
Prims algorithm constructs a MST through a sequence of expanding subtrees Greedy choice :
Choose minimum cost edge add it to the subgraph
Approach :
1. Each vertex j keeps near[j] T where cost(j,near[j]) is minimum 2. near[j] = 0 if j T = if there is no egde between j and T 3. Using a heap to select minimum of all edges
15
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
16
Prims Algorithm
17
18
19
20
Assume that Ti-1 is a part of MST We need to prove that Ti, generated by Ti-1 by Prims algorithm is a part of a MST We prove it by contradiction
Assume that no MST of the graph can contain Ti
21
Let ei=(u,v) be minimum weight edge from a vertex in Ti-1 to a vertex not in Ti-1 used by Prims algorithm to expand Ti-1 to Ti By our assumption, if we add ei to T, a cycle must be formed In addition to ei, cycle must contain another edge (v, u) If we delete the edge ek (v, u), we obtain another spanning tree wei wek So weight of new spanning tree is les or equal to T New spanning tree is a minimum spanning tree which contradicts the 22 assumption that no minimum spanning tree contains Ti
Prims Algorithm
Analysis :
How efficient is Prims algorithm ?
It depends on the data structure chosen If a graph is represented by its weight matrix and the priority queue is implemented as an unordered array, algorithms running time will be in (|V2|) If a graph is represented by adjacency linked lists and priority queue is implemented as a min-heap, running time of the algorith will be in O(|E|log|V|)
23
Kruskals Algorithm
It is another algorithm to construct MST It always yields an optimal solution The algorithm constructs a MST as an expanding sequence of subgraphs which are always acyclic but are not necessarily connected on intermediate stages of the algorithm
24
Kruskals Algorithm
Greedy choice :
Choose minimum cost edge
Connecting two disconnected subgraphs
Approach :
1. Sort edges in increasing order of costs 2. Start with empty tree 3. Consider edge one by one in order if T U {e} does not contain a cycle T = T U {e}
25
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
26
Kruskals Algorithm
27
28
29
30
Proof of Optimality
Algorithm T not optimal e1, e2, , en-1 c(e1) < c(e2) < < c(en-1)
T is optimal
e T
31
2. quick union
optimizes the union operation
35
36
Linked list representation of subsets {1, 4, 5 ,2} and {3, 6} obtained by quick-find 37 after performing union (1,4), union (5,2), union (4,5) and union (3,6)
39
a ) forest representation of subsets {1, 4, 5, 2} & {3, 6} used by quick union b) result of union (5,6)
41
The efficiency of a sequence of at most n-1 unions and m finds to only slightly worse than linear
42
ROAD MAP
Greedy Technique
Knapsack Problem Job Sequencing With Deadlines Minimum Spanning Tree Problem
Prims Algorithm Kruskals Algorithm
Single Source Shortest Paths Dijkstras Algorithm Job Sequencing With Deadlines
43
44
Dijkstras Algorithm
Idea :
Incrementally add nodes to an empty tree Each time add a node that has the smallest path length
Approach :
1. S = { } 2. Initialize dist [v] for all v
Dijkstras Algorithm
46
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
47
48
49
50
Dijkstras Algorithm
Analysis :
Time efficiency depends on the data structure used for priority queue and for representing an input graph itself For graphs represented by their weight matrix and priority queue implemented as an unordered array, efficiency is in (|V|2) For graphs represented by their adjacency list and priority queue implemented as a min-heap efficiency is in O(|E|log|V|) A better upper bound for both Prim and Dijkstras algorithm can be achieved, if Fibonacci heap is used
51
ROAD MAP
Greedy Technique
Knapsack Problem Minimum Spanning Tree Problem
Prims Algorithm Kruskals Algorithm
52
Find
J {1,2,..., N }
Feasibility:
The jobs in J can be completed before their deadlines
Optimality:
maximize
P
i i j
53
p p p
55
56
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
57
To check feasibility
If the jobs are scheduled as follows i1, i2, , ik check whether dij, j
k jobs requires at least k! time What about using a greedy strategy to check feasibility?
58
j = i1 , i2 ,...ik
di 1 di 2 K di k
We only need to check this permutation
The subset is feasible if and only if this permutation is feasible
59
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
60
Checking feasibility
Each check takes linear time in the worst case O(n2) in total
Encoding Text
Suppose we have to encode a text that comprises characters from some n-character alphabet by assigning to each of the texts characters some sequence of bits called codeword We can use a fixed-encoding that assigns to each character
Good if each character has same frequency What if some characters are more frequent than others
62
Encoding Text
EX: The number of bits in the encoding of 100 characters long text
a 45 000
b 13 ... 101
c 12 100
d 16 111
e 9
f 5 101 = 300
variable word 0
63
Prefix Codes
A codeword is not prefix of another codeword
Otherwise decoding is not easy and may not be possible
Encoding
Change each character with its codeword
Decoding
Start with the first bit Find the codeword A unique codeword can be found prefix code Continue with the bits following the codeword
Prefix Codes
EX: Trees for the following codewords
b ... 101
c 100
f 101
65
Huffman Codes
Given: The characters and their frequencies Find: The coding tree Cost : Minimize the cost
Huffman Codes
What is the greedy strategy?
67
Huffman Codes
Approach :
1. Q = forest of one-node trees // initialize n one-node trees; // label the nodes with the characters // label the trees with the frequencies of the chars 2. for i=1 to n-1 3. x = select the least freq tree in Q & delete 4. y = select the least freq tree in Q & delete 5. z = new tree 6. z left = x and z right = y 7. f(z) = f(x) + f(y) 8. Insert z into Q
68
Greedy Technique
Greedy Algorithm ( a [ 1 .. N ] ) { solution = for i = 1 to n x = select (a) is feasible ( solution, x ) solution = solution U {x} return solution }
69
70
71
Huffman Codes
Optimality Proof :
Tree should be full Two least frequent chars x and y must be two deepest nodes Induction argument
After merge operation new character set
Characters in roots with new frequencies
72
Huffman Codes
Analysis :
Use priority queues for forest
O (| c | log | c |)
Buildheap + ( 2 | c | - 2 ) DeleteMin + ( | c | -1 ) Insert
73