Analysis of the Greedy Algorithm on Graph Theory Problems

Jerick Mari A. Tancinco

July 23, 2012

Introduction

In 1935, Whitney introduced the notion of matroids in studying planar graphs. He dened concepts like independence and rank in graphs and these were found to be connected also to linear independence and dimension in linear algebra. Eventually, Rado and Edmonds discovered its relation to greedy algorithms and the problems that these algorithms can solve. Korte and Lovasz had generalized matroids to greedoids in 1982 by relaxing the denition of matroids.[1, 6] In this paper, the mathematical structures behind greedy algorithms on some famous graph theory problems will be analyzed.

Denitions

A set system is a pair (E, F), where E is a nite set called the ground set and F is a non-empty collection of subsets of E. Set F is called the family of feasible sets. The following are some of the properties a given set system may have. 1. (accessibility) For every non-empty feasible set X there exists x X such that X\ {x} is also feasible. 2. (exchange) For any feasible sets X and Y such that |Y | = |X| + 1, there exists y Y \X such that X {y} is also feasible. 1

3. (hereditary) If X Y and Y is a feasible set, then X is also feasible. The Greedy Algorithm [1] Let (E, F) be a set system with an associated function W : F R. Let X be an empty set. Choose x E\X such that 1. X {x} is feasible, and 2. W (X {x}) W (X {y}) for all y such that X {y} is feasible. Let X = X {x} and repeat until X can no longer be augmented. By replacing condition (2) with 2. W (x) W (y) for all y such that X {y} is feasible, a more specic greedy algorithm is formed, and will be referred to as Greedy. A greedoid is an accessible system (E, F) that satises the exchange property. Meanwhile, a matroid is a greedoid (E, F) that satises the hereditary property. A greedoid is an antimatroid if, for every feasible sets X and Y , X Y is also feasible. A greedoid is said to satisfy the intersection property if, for every feasible sets X and Y , X Y is also feasible. A greedoid has the local union property if, for every feasible sets X, Y , and Z with X and Y subsets of Z, X Y is a feasible set. If, instead, X Y is feasible, then it satises the local intersection property. Given a greedoid (E, F), the rank of set A E is dened as (A) = max{|X| | X F, X A}. Meanwhile, the rank closure of set A is dened as (A) = {x E | (A {x}) = (A)} Then, (E, F) is said to satisfy the union closure property if, for every feasible sets X and Y , (X) (Y ) (X Y ). Interval greedoids are greedoids satisfying the local union property while local poset greedoids are greedoids satisfying both the local union and the local intersection properties. 2

In a set system M = (E, F), consider a partition of E. Let X be feasible if and only if no two elements of X are in the same set in . Then, M is a matroid, called a partition matroid. In a hereditary system M = (E, F), the rank quotient of M is dened as lr(A) :AE , rq(M ) = min ur(A) where lr(A) and ur(A) are the cardinalities of the smallest and the largest maximal feasible subsets of A, respectively. Terms 0 may occur but will be 0 considered to have value 1. Moreover, M is said to be k-extendible if for all C F and x C where C + x F and for every D C where D F, / there exists a subset Y D\C with |Y | k such that (D\Y ) + x F. Lemma 1: Let M = (E, F) be a hereditary system with weight function w : E R+ . Moreover, let Tg be a solution of a problem found by Greedy, 0 and To an optimal solution. Then, a) rq(M ) w(Tg ) 1. [8] w(To )

b) rq(M ) = 1 if and only if M is a matroid. [13] An optimization problem will be referred to as greedily solvable if a greedy algorithm can solve all instances of the problem, otherwise, it will be referred to as greedily unsolvable. A graph consists of an ordered pair (V, E) where V is a non-empty nite set whose elements are called vertices, and E is a list of unordered pairs of these vertices, called edges. If an edge has identical ends, it is called a loop. Otherwise, it is a link. An edge is said to be incident to its ends, and vice versa. If a set of edges are incident to a common vertex, they are said to be adjacent to each other. The degree of a vertex is the number of edges incident to it, counting a loop as two edges. A leaf is a vertex with degree 1. A path Pn is a graph consisting of distinct vertices v1 , v2 , . . . , vn and edges v1 v2 , v2 v3 , . . . , vn1 vn . Pn can also be denoted by the sequence < v1 , v2 , . . . , vn >. A cycle is also a graph consisting of vertices v1 , v2 , . . . , vn and edges v1 v2 , v2 v3 , . . . , vn1 vn , having v1 and vn the only identical vertices. A graph G is connected if for every pair of vertices u and v, a path from u to v is contained in G. 3

A subgraph H of a graph G = (V, E) is a graph (VH , EH ) where VH V and EH E. Graph H becomes a spanning subgraph of G if VH = V . Given a non-empty subset E of E, (V , E ) is said to be the subgraph of G induced by E where V is the set of end vertices of the edges in E . A connected subgraph C is a component of a graph G if G has no other connected subgraphs containing C. Simple graphs are loopless graphs and contain no links joining the same pair of vertices. Graphs without edges are called empty graphs. If each pair of vertices in a simple graph G has an edge joining them, then G is a complete graph. Acyclic graphs are called forests, and connected forests are called trees. Trees may have a designated vertex, called the root. If the vertices of a graph G can be partitioned in to two non-empty sets X and Y such that one end of each edge belongs to X and the other to Y , then that graph is a bipartite graph. The edges of G can also be partitioned into X = {E(x) | x X} or Y = {E(y) | y Y } where E(u) contains all the edges incident to u. Moreover, a star is a bipartite graph with one of its partition containing exactly one vertex. A matching M of a graph G is a collection of pair-wise non-adjacent edges in G. Two vertices u and v are matched together if and only if uv M . Graphs G = (VG , EG ) and H = (VH , EH ) are graph isomorphic if there exists a bijection f : VG VH such that uv EG if and only if f (u)f (v) EH . Let (E, F) be a set system with an associated weight function w : C + R . The value function v : C 2C R+ is dened as w(y) : x X F, X A , if such X exists min yX v(x, A) = 1 + w(y) , otherwise.
yC

In other words, v(x, A) is the minimum weight of a feasible set that is contained in A that must also contain x. The shortest path function W : 2C R+ is dened as W (A) =
xA

v(x, A).

The maximum path function W : 2C R+ is dened as W (A) = max v(x, A).

xA

Theorem 1: The Pigeonhole Principle states that if n + 1 objects are put into n boxes, then at least one box contains two or more objects.

Objectives
1. determine underlying structures behind the greedy algorithm on some optimization problems, 2. nd greedily solvable cases of greedily unsolvable problems, and 3. rate the performance of greedy algorithms based on their optimality.

This paper aims to:

4
4.1

Review of Literature
The Minimum Spanning Tree Problem

Problem 1: Consider a connected graph G = (V, E) with weight function w : E R+ . The minimum spanning tree of G is the spanning tree with the minimum sum of its edge weights. There are 3 greedy algorithms that can optimally solve the MST problem [5, 14]. Algorithm 1: Kruskals algorithm starts with an empty set and recursively adds an edge of minimum weight that does not form a cycle with the previously chosen edges. The algorithm has to terminate after obtaining a spanning tree. Many papers have observed the structure behind Kruskals algorithm [3, 7, 8, 9, 11, 14]. Theorem 2: Given a graph G = (V, E), consider M = (E, I) where I is the set of all E E such that (V, E ) is a spanning forest of G. Then, M is a matroid, also called the graphic matroid of G. 5

Proof. Let M = (E, I) where I is the set containing all subsets E E such that (V, E ) is a spanning forest of G = (V, E). Thus, M is hereditary since the subgraphs of a forest are also forests. Let X and Y be feasible sets where |Y | = |X|+1. In the forest F induced by X, the number of edges from some forest H that may form a cycle to F is at most |X|, otherwise H would contain a cycle. Since the graph induced by Y is also a forest and |Y | = |X| + 1, there must exist an edge e Y \X such that it will not form a cycle in X, that is, X {e} is feasible. Thus, M satises the exchange property. Therefore, M is a matroid. The optimality of Kruskals algorithm may follow from Lemma 1.a and 1.b. Algorithm 2: Prims algorithm works by rst assigning a vertex as the root. Then, it chooses the edge with the least weight such that it does not form a cycle with the already chosen edges and it must be connected to the root. In Klappeneckers papers [9, 10], the structure behind Prims algorithm was established. Theorem 3: Given a graph G = (V, E), consider N = (E, J ) where J is the set of all E E such that (V, E ) is a tree in G rooted at a xed vertex r. Then, N is a greedoid, also called the branching greedoid of G. Proof. In a graph G = (V, E), let N = (E, J ) where J is the set containing all subsets E E such that the graph induced by E is a tree rooted at a xed vertex r. Given a tree H, the edge sets of its subgraphs do not necessarily induce trees since removing an edge of H not incident with a leaf makes it disconnected. Then, N at least satises the accessibility property. Let X and Y be feasible sets where |Y | = |X| + 1. In the tree T induced by X, the number of edges from some tree H that may form a cycle to T is at most |X|, otherwise H would contain a cycle. Since the graph induced by Y is also a tree and |Y | = |X| + 1, there must exist an edge e Y \X such that it will not form a cycle in X, that is, X {e} is feasible. Thus, M satises the exchange property. Therefore, N is a greedoid. 6

Algorithm 3: Boruvkas algortihm begins with a partial solution T , where T is the empty graph on V . Then, for each component in T , it adds the shortest edge connecting that component to another component provided it does not create a cycle. This algorithm ends when T only has one component.

4.2

The Shortest Path and Minmax Path Problems

SHORTEST PATH PROBLEM

Problem 2: [1] Let G = (V, E) be a graph. Given a set system (E, F) with the shortest path function W , the shortest path problem (SP) looks for Xk F with |Xk | = k such that W (Xk ) = min {W (Y ) : Y F, |Y | = k} . Theorem 4: [1] If, in a set system (E, F) with an arbitrary weight function w : E R+ , a greedy algorithm solves the shortest path problem on (E, F), then (E, F) must be a greedoid. The greedy algorithm below is used to solve SP. Algorithm 4: Dijkstras algorithm [2] works by having a nested sequence of trees rooted at a vertex r. The trees are constructed by adding the shortest edge that does not form a cycle and must be connected to r. Theorem 5: [1] The greedy algorithm solves the shortest path problem on a set system (E, F) if any of the following is satised: (i) (E, F) is an antimatroid. (ii) (E, F) is an interval greedoid with the intersection property. (iii) (E, F) is a local poset greedoid with the union closure property. The structure behind Dijkstras algorithm was said to be a local poset greedoid with the union closure property. MINMAX PATH PROBLEM

Problem 3: [1] Let G = (V, E) be a graph. Given a set system (E, F) with the maximum path function W , the minmax path problem (MP) looks for Xk F with |Xk | = k such that W (Xk ) = min {W (Y ) : Y F, |Y | = k} . Theorem 6: [1] If the greedy algorithm solves the minmax path problem on an interval greedoid, then it also solves the shortest path problem.

4.3

Maximum Matching Problem

(UNWEIGHTED) MAXIMUM MATCHING PROBLEM

Problem 4: Given a graph G = (V, E), the maximum matching (MM) problem looks for a matching in G with the highest cardinality. The theorem below does not directly involve the greedy algorithm but it describes a structure that can be used for the MM problem. Theorem 7: [14] Let G = (V, E) be a bipartite graph with bipartition {X, Y }. Let (E, I) and (E, J ) be the partition matroids associated with the partitions X and Y , respectively. Then M is a maximum matching in G if M is maximum in I J . There are greedy heuristics formed for the MM problem. Algorithm 5: Simply collect pairwise non-adjacent edges from the graph. Terminate when no more edges can be collected that are non-adjacent to the already chosen edges. This is the Greedy algorithm for the MM problem. Theorem 8: If S is the set system associated with Greedy, then S is a matroid if and only if G does not contain P4 . Proof. Let S = (E, I) be a set system where I is the set of all matchings of G = (V, E). Then, S is hereditary since the subsets of a matching are also matchings. Suppose S is a matroid. Then, S satises the exchange property. Furthermore, suppose G contains P4 =< v1 , v2 , v3 , v4 >. When X = {v2 v3 } and Y = {v1 v2 , v3 v4 }, we have a contradiction with the exchange property. Thus, G must not contain P4 . 8

Suppose S is not a matroid. Since S is already hereditary, therefore it must not satisfy the exchange property. Thus, there exist matchings X and Y such that |Y | = |X| + 1 and X {e} is not a matching of G for all e Y . That is, each edge in Y is adjacent or equal to some edge in X. By the Pigeonhole Principle, there must exist p X adjacent to two edges q1 and q2 in Y , forming a path of length 3. Thus, G contains P4 . Therefore, S is a matroid if and only if G does not contain P4 . The Greedys performance ratio and a greedily solvable case of the MM problem are presented below. Theorem 9: [8, 15] In a graph G and set system M = (E, S) where S is the 1 set of all matchings of G, rq(M ) = 2 . Theorem 10: [8] Let G = (V, E) be a graph whose components are either isomorphic to a complete graph or to a star. Then, rq(M ) = 1. Algorithm 6: Choose the edge rst that has the least number of adjacent edges to one of its endvertices, since otherwise that endvertex will have a less possibility of being matched.

References
[1] E. A. Boyd. A combinatorial abstraction of the shortest path problem and its relationship to greedoids. Technical Report 887, Rice University, May 1988. Retrieved August 19, 2011, from http://www.caam.rice.edu/caam/trs/88/TR88-07.pdf. [2] E. W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269271, 1959. [3] M. Grotschel. Perspectives in Mathematics, chapter Developments in combinatorial optimization, pages 249294. Birkhauser Verlag, 1984. Electronic Version. [4] M. Grotschel and L. Lovasz. Handbook of Combinatorics, chapter Combinatorial optimization, pages 15411598. Elsevier Science, 1995. Electronic Version. [5] M. L. Hetland. Python algorithms: Mastering basic algorithms in the Python language. Apress, 2010. Electronic Version. [6] W. Johnson. Matroids. Retrieved October 26, 2011, from http://www.math.washington.edu/morrow/336 09/papers/Will.pdf, 2009. [7] C. Jones and R. L. Hadas. Matroids: The theory and practice of greed. Retrieved February 12, 2012, from http://www.cs.hmc.edu/courses/2002/spring/cs181b/matroids.pdf, 2002. [8] D. Jungnickel. Graphs, networks and algorithms. Springer, 3rd edition, 2008. Electronic Version. [9] A. Klappenecker. Theory of greedy algorithms. Retrieved October 29, 2011, from https://parasol.tamu.edu/welch/teaching/411.f08/greedy.pdf, 2008. [10] A. Klappenecker. Accessible set systems, 2009. Retrieved from http://faculty.cs.tamu.edu/klappi/cpsc411s09/cpsc411-set4.ppt. [11] A. Klappenecker. Greedy algorithms for matroids, 2009. Retrieved from http://faculty.cs.tamu.edu/klappi/cpsc411-f09/cpsc411-set3.ppt. 10

[12] B. Korte and D. Hausmann. An analysis of the greedy algorithm for independence systems. Annals of Discrete Mathematics, 2:6574, 1978. [13] J. Mestre. Greedy in approximation algorithms. Retrieved February 11, 2012, from http://rp-www.cs.usyd.edu.au/mestre/papers/greedy.pdf. [14] C. H. Papadimitriou and K. Steiglitz. Combinatorial optimization: Algorithms and complexity. Dover Publications, Inc., 1998. Electronic Version. [15] A. Rajaraman and J. D. Ullman. Mining of massive data sets, chapter Advertising on the Web, pages 261286. 2011. Electronic Version.

11