Sei sulla pagina 1di 6

2015 4th International Conference on Advanced Computer Science Applications and Technologies

2015 4th International Conference on Advanced Computer Science Applications and Technologies

An Efficient Object Extraction with Graph-Based Image Segmentation

Ali Saglam

Department of Computer Engineering Faculty of Engineering Selçuk University, Konya, Turkey

Nurdan Akhan Baykan

Department of Computer Engineering Faculty of Engineering Selçuk University, Konya, Turkey

Abstract— Object extraction process is a closely related issue with image segmentation process. To separate an image to several segments formed similar pixels, many methods are proposed in the area of image processing. Graph-based image segmentation is also one of the segmentation methods. Because of their representation convenience and ease of use, graphs are used as important tools in many image processing applications. While an image segmentation process runs, the processes splitting a graph to sub graphs and merging sub graphs are carried out in the meanwhile. To fulfill these processes, the method uses some local features such as differences between vertices in the graph, which represent pixels, or global features of the image and its segments. To extract an object from an image, we first segmented the entire image, because of to save global features, or to obtain more accurate segmentation. Finally, we extract the intended object from the image by merging the segments that are inside the area drawn before by us. We test the method on some images in the Segmentation Evaluating Database from Weizmann Institute of Science and evaluate the segmentation results. Our F-measure score values show that it seems noticeable good segmentation.



Minimum Spanning Tree






In many image processing application such as object extraction, medical image processing, satellite image processing, video processing, object recognition, image reconstruction and more, image segmentation is an important stage [1,2,3]. In the segmentation stage, an image is partitioned into regions according to similarity of its pixels [4,5,6]. Image segmentation makes an image and its perception easier and more meaningful to analyze [7]. It typically focuses on the object in the image and borders. Image segmentation techniques try to cluster homogeneous pixels within the same regions by using color, pattern (texture) and density differences [8]. Many pattern recognition and computer vision applications reduce image information for more efficient segmentation by taking advantage of global features of images such as eigenvectors, standard deviation, energy function and so on [9]. However, in this way, the processing times exceed the desired time period for many computer visions application when the process is applied to especially high-resolution images. Due to the computational complexity of spectral clustering methods, intuitive approaches such as greedy algorithms, which only make a decision according to

the current condition, or MST based algorithms are preferred more in real-time applications [8]. Many segmentation algorithms have been developed over the years. Among the existing image segmentation methods, many successful ones make use of mapping the image elements onto a graph so that they solve the segmentation problem in a spatially discrete space by the from graph theory tools [10]. These methods are called graph-based image segmentation methods. In a graph, each vertex (node) indicates a point, and each edge joins the points by a line. In a weighted graph, each edge is associated with a real number called weight [11]. Minimum Spanning Tree (MST) based methods are also one of the graph-based methods. To reach all vertices in a graph, the structure of spanning tree is generally used, because a spanning tree does not contain any cycle. If we navigate through the nodes in a spanning tree without return to back and we meet again the node that we met with before, there is a cycle. In a graph, MST is the shortest spanning tree that has the smallest total weight among spanning trees that cover all vertices in the graph. In 1960’s, pioneering methods of graph theory emphasize the basis of gestalt principles in perception meaningful clusters according to similarities and differences between vertices [10]. In this way, the graph is divided into sections according to the local features. In fact, every part of the image is considered to be an object. At this stage of the graph-based approach, the process of image segmentation has been an implicit transaction based on clustering and MST [1]. These methods generally have been used for local computations on the segmentation results. They do not guarantee a well segmentation over all images, because of taking into account only local features. To overcome this problem, then, some methods that are recursive and also take into account the characteristics of sub clusters are integrated to MST-based methods [7,10,12,13,14,15]. In this paper, we use Boruvka’s MST algorithm for MST based clustering and use Felzenszwalb and Huttenlocher’s equation (Eq. 3) as comparison criterion to merge vertices. Then, we apply our graph-based object extraction method upon this.


MST-based segmentation methods are associated with a graph-based clustering basis. Common works on graph clustering can be dated back to old times.

978-1-5090-0424-9/16 $31.00 © 2016 IEEE

978-1-5090-0424-9/16 $31.00 © 2016 IEEE

DOI 10.1109/ACSAT.2015.29

DOI 10.1109/ACSAT.2015.29

86 86

© 2016 IEEE 978-1-5090-0424-9/16 $31.00 © 2016 IEEE DOI 10.1109/ACSAT.2015.29 DOI 10.1109/ACSAT.2015.29 86 86
© 2016 IEEE 978-1-5090-0424-9/16 $31.00 © 2016 IEEE DOI 10.1109/ACSAT.2015.29 DOI 10.1109/ACSAT.2015.29 86 86
© 2016 IEEE 978-1-5090-0424-9/16 $31.00 © 2016 IEEE DOI 10.1109/ACSAT.2015.29 DOI 10.1109/ACSAT.2015.29 86 86

In the beginning of 1970’s, Zahn [16] applied a method that removes the longest edges from the MST formed to obtain clusters for predefined constant . At the same time, to overcome very small cluster pieces, he proposes a deletion criterion that compares the standard deviations on either sides of the edge which will be deleted and weight of the edge [17,18]. Lopresti and Zhou [19] used the Zahn’s method on color image segmentation using Euclidean distance measurement in 2000. In 1980’s, Morris et al. [20] used MST to partition images hierarchically. The developed method carries a segmentation process, which is in different scales, based on a principle which requires the most similar pixels grouped together and dissimilar pixels separated. It breaks the highest weight of the MST, and the graph pieces are formed by maximum differences between the neighboring sub graphs. He applied a recursive MST algorithm as another method. According to this algorithm, after each merging, the weights of the differences of the neighboring clusters are updated and the edge list is sorted every iteration. This form is thought to be inefficient in terms of an algorithm speed. Kwok et al. [14,15], then, proposed a fast recursive MST algorithm. According to this algorithm, to speed up the process weights of the edges are transformed a discrete structure, and instead of the stack structure, heap structure is used. In subsequent years, Bayramoglu applied this method on distributed systems and showed comparison results [7]. Back to 1998, Felzenszwalb and Huttenlocher proposed an efficient graph-based image segmentation method based on the structure of Kruskal’s MST algorithm [12,21]. It uses both global differences between neighboring two sub-graphs and local differences inside each sub-graph. Moreover, it nearly runs in linear time with the number of nodes in the graph, and this provides an advantage for real-time video segmentation which requires fast processing. Another MST- based algorithm is developed by Haxhimusa and Kropatsch, which makes up use the Boruvka’s MST principles [22]. As E refers the number of edges and N refers the number of nodes which represent the pixels in image (Fig. 1), computational complexity of Kruskal's algorithm is and the complexity of Boruvka’s algorithm is


is and the complexity of Boruvka’s algorithm is [8]. Figure 1. Graph representation of pixels according

Figure 1.

Graph representation of pixels according to 8-neighborhood

In this study, the calculation formula of the number of edge in an image is Eq. 1 for 8-connected neighborhood, as and that refer to lengths of the dimension of the image.


In 2009, Wassenberg et al. [23] developed an algorithm

that compute the independent MSTs and merge them. In 2010, Grundmann et al. [4] proposed a Graph-based Video Segmentation technique based on Graph-based Image Segmentation of Felzenszwalb et al. [12]. And, Rother et al. [24] proposed an interactive graph-based method that uses some global features of images iteratively. There are many studies in image segmentation using graph based algorithms.


We used the methods and the algorithm explained below and tested our method on the color images from Single Object Segmentation Evaluation Database [25]. Before we explain our method, we should review some background information about the materials and methods.

A. Graph-based Image Segmentation

In the constructed graph from in an image, the vertices or nodes set represent the

pixels and weighted undirected edges, which have a positive distance value and have no certain direction, represent the similarities between adjacent pixels as shown Fig. 1 and Fig. 2. The weight of each edge is computed with some different measures. Euclidean distance (Eq. 2) is one of the distance measures used for the weight of the edges [26].

A graph built from pixels of an image can be represented

different pixel neighborhoods such as 4-neighborhood and 8- neighborhood. Neighborhood-based clustering algorithm proposed by Zhou et al. performs good data clustering, because it can discover cluster of similar data and borders

using neighboring relationship between data points [18]. In our method, we used 8-neighborhood mapping. We used Euclidean distance for measurement of edges weights in RGB (Red, Green and Blue) color space as in Fig. 2. To find the Euclidean distance between two adjacent pixels, we applied the Eq. 2 [27].



Figure 2. Graph representation of the color pixels w ith the edge weights between their

Figure 2.

Graph representation of the color pixels w ith the edge weights between their neighboring pixels

B. Boruvka’s Minimum Spanning Tree Alg orithm

A spanning tree is a sub graph of a gr aph . contains every vertex and does not contain any cy cles in the graph.

A graph might have more than one spa nning tree. The

have also weight weight is called

Minimum Spanning Tree (MST). Some g reedy algorithms such as Prim’s Algorithm [28], Kruskal’s al gorithm [29] and Boruvka’s algorithm [30] are popular to fi nd the MST in a graph. In these algorithms, Boruvka’s a lgorithm is also unconcerned with any sequential proc ess and orders [8,31,32]. Each node or sub tree act ind ependently from

undirected edges in a spanning tree can values. The tree that has the least total

others. In this study, Boruvka’s algorithm i s used because it creates not much redundant small segmen ts and need not much preprocessing such as blurring filt er when looked segmentation results according to [33]. Bor uvka’s algorithm

is also known as Sollin’s algorithm. It fo rms a minimum

spanning tree by running the following steps . Step 1: Each vertex is considered to be subset and has a subset number different from others. Step 2: Each subset selects the minimu m-weighted edge

according to Euclidean distance, which con nects it to one of

the neighboring subsets as shown in Fig. 3.

Step 3: If subsets at both ends of the sel ected edges don’t have the same subset number, these subset s are merged and have the same subset number in any sequen ce. Step 4: If there is only one subset , the process is

terminated. Otherwise it continues from step


process is terminated. Otherwise it continues from ste p 2. Figure 3. Representation of Boruvka’s algorithm

Figure 3.

Representation of Boruvka’s algorithm o f the graph in Fig. 2

C. Felzenszwalb and Huttenlocher ’s Segmentation Criterion Felzenszwalb and Huttenlocher [21] presented a merging

criterion in Eq. 3 that make use o f some local and global differences of the two sub trees and the differences inside the both sub trees that are adjace nt each other [10,12, 17,33,34,35]. The internal differenc e of the sub tree is the

highest edge weight in the

is the weight value of t he smallest edge linking

the two adjacent sub trees and . refers to the size of the sub tree , to explain, total vert ices numbers in it, and k refers to a constant. If satisfies the merging criterion seen in Eq. 3, the two s ub trees will be merged.

Otherwise, it will pass without any

sub tree. The differences



D. Implementation of the Methods Firstly, we draw a rectangle

seen in the Fig. 4 or

rectangles which we use to take t he intended object to be entirely in. To look at the overall process, the steps of the segmentation method can be summa rized in Fig. 5.

The method can be explained a s continuing. We got the image into the 3D array image[M] [N][3] in which 3 is the color number of each pixel in RG B color space. And then, we prepared the 2D array nodes [M][N] that consists of objects of the Node class. We progr am the method by taking advantage of Object Oriented Progr amming (OOP) and using the C++ programming language.

Progr amming (OOP) and using the C++ programming language. Figure 4. Taking the intended object into

Figure 4.

Taking the intended object into an area

A Node object derived from th e Node class. It includes some information about the pixel re presented. It includes the

cluster number that indicates which cluster it belongs to, the RGB color values, the pointer roo t that addresses its root

node, the information whether the

or not, the set of neighboring nodes that keeps the pointers of the neighboring nodes accor ding to 8-connected

neighborhood, and the set of conn ected nodes. In addition

these, root nodes keep the cluster

node outside the cluster, against n ode of the nearest node

ixel is in the drawn area

size , , the nearest


inside the cluster, and the distance value between these nodes in other words between the clusters and which each of the two nodes belongs to. Initially, all the nodes have different cluster number, namely, each node is a cluster at the same time, and each node is a root node. Moreover, Node objects have some functions that run recursively by linking the connected nodes and they are doing in the same way too. By the way, we can reach whole region and execute the intended processes with a node. To give an example these functions, there are functions of adding neighbor, connecting to a node, changing the cluster number and the pointer linking the root node, getting the nearest node of the nearest region, and so on.

getting the nearest node of the nearest region, and so on. Figure 5. Flowing chart of

Figure 5.

Flowing chart of the segmentation and the object extraction steps

After initializing the array nodes[M][N], we set the edges between the neighboring nodes and their weight according to Euclidean distance by using the RGB color values of the pixels. After then, we used a self-recursive function that composes trees by using Boruvka’s algorithms with Felzenszwalb and Huttenlocher’s comparison criterion. With this criterion, it appears several trees.

Through the recursive function, some sequential operations are performed. Firstly, it finds the root nodes for each cluster and computes that refer the number of roots in whole present graph so that the further loops in the function will be cycled until this number. At the following step in the function, the process of finding the nearest region of each region is executed. The information about that is kept in the root nodes. And, it started to merge regions according to the criterion through the cycles. So, the process of the function is completed. If there is at least one merging at the last process, the function will be recalled by itself. Otherwise, the segmentation will be done. Finally, to extract the intended object, all the neighboring segments inside the bounded area are merged with each other, and other neighboring segments are also merged in the same way as seen Fig. 6. Showing the final clusters and the object on the screen can be performed by using the root nodes. For deletion small segments inside the area, the last process can be performed by the way that the list of the nearest neighboring region pairs is sorted according to the

distance in ascending order

and allowed to

merge region those are lower than the given parameter of the limit value of the least cluster size that we have determined.

value of the least cluster size that we have determined. Figure 6. Representation of a merging

Figure 6.

Representation of a merging according to case of selected area

F-score (F 1 score or F-measure) is one of the measurements of an image segmentation accuracy [36]. It takes into account both the precision value and the recall value to compute the score. The precision is the ratio of the number of correct positive results of the extracted region to the number of all intended region (human segmentation), and the recall is the ratio of the number of correct positive results of the extracted region to the number of all extracted region. F-score reaches its best value at 1 and worst at 0 [37]. In Eq. 4, it is shown how to compute the F-score traditionally. The formula is called traditional F-measure or balanced F-score and is achieved by calculating the harmonic mean of the values of precision and recall.



We applied the method to several images from Single Object Segmentation Evaluation Database [25]. The inputs and the outputs of some of them are shown in Fig. 7. We used the value 300 for the constant for these examples.







(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the
(a) (b) (c) (d) (e) Figure 7. Examples of segmentation results and the names of the

Figure 7.

Examples of segmentation results and the

names of the images


are (a) “100_0109”, (b) “beltaine_4_bg_05 “yokohm060409_dyjsn191”, (d) “tendrils” and (e)

0502”, (c)

We tested the images with the evalua score. The results for the images in Fig. 7 Table I.

ting function F- are as shown in




Image Name






























The results can vary because of the cha images. For example, the test result of the i worse than the others. Because, the object colors are very similar to each other.

racteristics of the mage “tendrils” is and background

Finally, to compare our method methods those have been perform perform our method overall sin database (100 images). We obtain methods from the source of the data

with other segmentation ed on the database, we gle object color image the scores of the other base [25].









Our method

0.88 ± 0.0



Unified approach [36]


± 0.0



Probabilistic bottom-up [38]


± 0.0



Texture segmentation [39]


± 0.0



Normalized cuts [40]


± 0.0



Mean shift [41]


± 0.0



Because of the pre-intervention assume our method as interactive Table II, it seems that our score is scores. We coded the program with C+ To import and illustrate the images, Source Computer Vision) library [4



Image segmentation process is a level of computer vision processe many field. Object extraction is an of these fields. In this paper we app segmentation method by benefiting to show how to extract an object fro using our object extraction method our method is how efficient. Ano method is that it is fast becaus algorithm and using only color dif As the operations that we showed understood that the object extract used upon another graph-based ima This shows that the success of the the segmentation method to be used


This study was supported by Coordination (Project No. 2015-OY

to draw intended area, we

. To look the scores in the best among the other

+ programming language. we used OpenCV (Open



n important intermediate

s that is used widely in important stage of some lied a graph-based image

from literature. We tried

m a segmented image by

and we demonstrated that ther characteristic of the

e of making a greedy ferences and cluster size. are examined, it can be ion method also can be ge segmentation method. process also depends on



Selcuk University OYP






G. Durusoy, “Graph Cuts And Re Medical Image Segmentation”, The P University, 2014

gion Growing Algorithms in olytechnic Institute, New York


W. Tao, H. Jin and Y. Zhang, “Color Mean Shift and Normalized Cuts”, I Man, And Cybernetics—Part B: Cyber

Image Segmentation Based on EEE Transactions on Systems, netics, Vol:37 (5), 2007


Q. Huang, X. Bai, Y. Li, L. Jin and X

. Li,

“Optimized graph-based

segmentation for ultrasound images”, 224, 2014

Neurocomputing 129, pp.216–



M. Grundmann, V. Kwatra, M. Han and I. Essa, “Efficient


Hierarchical Graph-Based Video Segmentation”, IEEE CVPR, pp. 2141-2148, 2010 D. F. Fitzgerald, D. S. Wills and L. M. Wills, “Real-time, parallel


segmentation of high-resolution images on multi-core platforms”, J Real-Time Image Proc, DOI 10.1007/s11554-014-0432-z, 2014 Y. Deng, B. S. Manjunath and H. Shin, “Color image segmentation”, Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR'99, Fort Collins, CO, volume: 2, pp. 446- 51, 1999

[7] N. Y. Bayramoglu and C. F. Bazlamacci, “Recursive Shortest Spanning Tree Algorithms for Image Segmentation”, EUROCON 2005 Belgrade, 2005


A. N. Skurikhin,

“Hierarchical Image Feature Extraction by an


Irregular Pyramid of Polygonal Partitions”, ASPRS 2009 Annual Conference, Baltimore, Maryland, 2009 J. Ding, R. Ma and S. Chen, “A Scale-Based Connected Coherence

Tree Algorithm for Image Segmentation”, IEEE Transactions on Image Processing, Vol:17 (2), 2008 [10] B. Peng, L. Zhang and D. Zhang, “A survey of graph theoretical approaches to image segmentation”, Pattern Recognition 46(2013)1020–1038, 2012


interactive foreground extraction using iterated graph cuts”, ACM

Transactions on Graphics (TOG), Volume 23 Issue 3, 2004

[25] x.html (Accessed in 2015) [26] F. Malmberg, “Graph-based Methods for Interactive Image Segmentation”, Acta Universitatis Upsaliensis Uppsala ISSN: 1651-

[24] Carsten Rother, Vladimir Kolmogorov, Andrew Blake,

6214 ISBN: 978-91-554-8037-0, 2011

[27] H.D.






2259-2281, 2001













[28] R. Prim,

“Shortest connection networks and some generalization”,

Bell Systems Technical Journal, Vol:36, pp.1389–1401, 1957

[29] J.B. Kruskal, “On the shortest spanning subtree of a graph and the traveling salesman problem”, Proceedings of the American Mathematical Society 7 (1): 48–50, JSTOR 2033241, 1956

[30] O.

minimal problem)”, Prace mor. prirodoved. spol. v Brne III (in Czech, German summary) 3: 37–58, 1926

[31] Y. Haxhimusa, A. Ion, W. G. Kropatsch and T. Illetschko, “Evaluating Minimum Spanning Tree Based Segmentation Algorithms”, Computer Analysis of Images and Patterns Volume

(About a certain


“O jistem problému minimalnim

[11] J.A. Bondy, U.S.R. Murthy, “Graph Theory with Applications”,


of the series Lecture Notes in Computer Science, pp.579-586,

American Elsevier Publishing Co., U.S.A., 1976


[12] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation”, International Journal of Computer Vision59 (2), pp.167–181, 2004

[13] Y. Xu, V. Olman and D. Xu.,

expression data clustering”, Genome Informatics, Vol:12, pp.24–33,

“Minimum spanning trees for gene


[14] S.H. Kwok and A.G. Constantinides, “A fast recursive shortest spanning tree for image segmentation and edge detection”, IEEE Transactionson Image Processing, Vol:6 (2), pp.328–332, 1997

[15] S.H. Kwok, A.G. Constantinides and W.C. Siu, “An Efficient Recursive Shortest Spanning Tree Algorithm using Linking Properties”, IEEE Transactions on Circuits And Systems for Video Technology, Vol: 14 (6), June 2004 [16] C. T. Zahn, “Graph-Theoretical Methods for Detecting and Describing Gestalt Clusters”, IEEE Transactions On Computers, Vol:20 (1), pp.68-86, 1971

[17] O. Grygorash, Y. Zhou and Z. Jorgensen, “Minimum Spanning Tree Based Clustering AlgorithmsProceedings of the 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'06) 0-7695-2728-0/06 $20.00, 2006 [18] S. J. Peter, “Minimum Spanning Tree-based Structural Similarity Clustering for Image Mining with Local Region Outliers”, International Journal of Computer Applications (0975 – 8887) Vol:8 (6), 2010

[19] D. Lopresti, J. Zhou,

“Locating and recognizing text in www

images”, Information Retrieval, pp.177–206, 2000 [20] O.J. Morris, M. de J. Lee, A.G. Constantinides, “Graph theory for image analysis: an approach based on the shortest spanning tree”, IEE Proceeding F, Communications, Radar and Signal Processing Vol:133, pp.146–152, 1986

[21] P. F. Felzenszwalb and D. P. Huttenlocher, “Image Segmentation Using Local Variation”, In Proceedings of IEEE Conference on CVPR, p:98–104, 1998

[22] Y. Haxhimusa and W. Kropatsch, “Segmentation Graph Hierarchies”, Proceedings of Joint Workshops on Structural, Syntactic, and Statistical Pattern Recognition S+SSPR. Vol:3138, Lecture Notes in Computer Science pp.343-351, 2004 [23] J. Wassenberg, W. Middelmann and P. Sanders, “An Efficient Parallel Algorithm for Graph-Based Image Segmentation”, Computer Analysis of Images and Patterns, LNCS Vol:5702 pp.1003–1010,


[32] S. Chung and A. Condon, “Parallel Implementation of Borikka’s Minimum Spanning Tree Algorithm”, IEEE Proceedings of IPPS ’96,


[33] M. Tepper, M. Mejail, P. Musé and A. Almansa,

Nearest Neighbors”, HAL Id: hal-00583120, 2011 [34] S. C. Turaga, “Learning image segmentation and hierarchies by learning ultrametric distances”, Massachubetts Institute of Technology, 2009

[35] A. Fahad and T. Morris, “A Faster Graph-Based Segmentation Algorithm with Statistical Region Merge”, ISVC'06 Proceedings of the Second international conference on Advances in Visual Computing - Volume Part II Pages 286-293 Springer-Verlag Berlin, Heidelberg, 2006

[36] S. Bagon, O. Boiman and M. Irani, “What is a Good Image Segment? A Unified Approach to Segment Extraction”, ECCV 2008 pp.30-44,

“Boruvka Meets


[37] C.

Butterworth-Heinemann Newton, MA, USA, 1979 [38] S. Alpert, M. Galun, R. Basri and A. Brandt, Image segmentation by probabilistic bottom–up aggregation and cue integration, in:

International Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2007 [39] M. Galun, E. Sharon, R. Basri and A. Brandt, Texture segmentation by multiscale aggregation of filter responses and shape Computer Vision, pp. 716–723, 2008 [40] J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (8) pp. 888–905, 2002 [41] D. Comaniciu and P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (5), pp. 603–619, 2002 [42] (Accessed in 2015)