Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Segmentation,
Mango
localization,
Introduction
202
activities through the help of computer vision techniques, as it is very helpful to the
formers, dealers and consumers. The mango fruits should be harvested at green
mature stage. Selective harvesting of green matured mangos is helpful for maturity
grading by distinguishing ripe and unripe mangos so that ripe mangos are used at the
earliest and unripe mangos can be stored for some time. Further, quality wise grading
of mangos can also be done during selective harvesting.
So, given an image of a mango tree, the problem of selective harvesting of mangos
can be defined as a problem of localization and segmentation of mango regions from
it. On the other hand it contributes the problem of spatial correlation since both leaf
and fruits are green in color. Mango region segmentation is a process of acquiring
knowledge about mango regions in an image. It is a very first task in a mango image
analysis process such as disease mapping, variable spraying and selective harvesting.
Quality of the subsequent tasks will depend on the success of the mango region
segmentation process. Color and texture are the two essential features in mango tree
image analysis. Texture is an efficient measure to estimate the structure, orientation,
roughness, smoothness, and regularity differences of mango regions in the mango tree
image. Two usual problems in mango region segmentation are over-segmentation:
where an image is segmented into a more number of regions than the actual mango
regions in the mango tree image and under-segmentation: where an image is
segmented into a less number of regions than the actual mango regions in the mango
tree image. The over-segmentation and the under-segmentation of mango regions
happen usually in case of images with spatially varying illumination. Also, the leaves
are occluded on mangoes in the mango tree image. Hence balancing the over and
under segmentation in unconstraint environment of the mango tree image is a
challenging issue in the area of color image segmentation.
The automation of agricultural practices for significant increase in food production
is emerging out as a new challenge for computer vision community. Ducournau et al.
[2] proposed a machine vision approach to count the number of emergent radical tips
on seed-lots, under the constrained environment and segmentation is accomplished
with the help of thresholding and morphological operators. Green vegetation region
segmentation [3] using the IHS (Intensity, Hue, Saturation) and RGB (Red, Green,
Blue) color space for color feature extraction and then apply mean shift and BPNN
(back propagation neural network) for segmentation. One of the central point of
precision agriculture is the selective treatment of weeds [4] and it is achieved in three
different stages at which each different agricultural elements is extracted. They are
segmentation of vegetation against non-vegetation, crop row elimination and weed
region extraction. The segmentation is done by thresholding on the basis of dominant
G-component, dominated B-component, minimum and maximum intensity in the
image. Segmentation of lesquerella flowers was proposed by Thorp et al. [5] based on
thresholding the images in HIS color space and boundary conditions using six
parameters, including the maximum and minimum hue, saturation and intensity in
image. The popular general unsupervised segmentation of color-texture region in
image [6] is presented and it is named as J-seg which consists of two independent
steps: color quantization and spatial segmentation.
All the segmentation problems which are related to a precision agriculture are
considered to be two class problems (the pixel belongs to vegetation/non-vegetation).
203
Proposed Framework
In this section, we propose a new framework for mango region segmentation, which
consists of two phases namely, mango localization and edge mapping. In first phase,
we apply thresholding to each color band separately then combined them. The
combined image (T-image) is said to have j-number of regions. These regions are
considered as mango regions. We perform smoothing to eliminate the non-mango
regions in the thresholded image. Then apply binarization to the resultant image to get
B-image. The B-image has say k-number of regions assumed to be the mango regions.
Finally we achieve mango localization through template matching which eliminates
the non-mango regions in the B-image and results with an image (L-image) having mnumber of mango regions where j k m. In second phase, the edge information
from the original image is superimposed on to the regions of the L-image for
segmentation. The block diagram of mango region segmentation is shown Fig .1.
2.1
Image Thresholding
The reflectance on the surface of mango regions is higher than that of non-mango
regions, so higher intensity will be preserved by the regions belonging to mangos.
This is an important clue for us to distinguish mango regions from non mango
regions. Based on this clue we apply a simple thresholding to segment the mango
regions from its tree image. We separately threshold each R, G & B components in
the mango tree image and then combine the resultant images back (T-image). The
middle value of the intensity range in a given image intensity distribution will be
taken as threshold. The thresholding process can be formulated as shown in Eq. (1)
and (2).
(1)
Where
Threshold =
204
Fig. 2. (a) A given mango tree image (RGB). (b) The R-component of the input image. (c) The
G-component of the input image (d) The B-component of the input image.
205
Fig. 3. (a) Thresholded image of R-component. (b) Thresholded image of G-component. (c)
Thresholded image of B-component. (d) Thresholded image of given input image Fig.2a.
2.2
The small regions and sharp transitions in the thresholded image are non-mango regions,
because the mango regions are relatively larger regions. The sharp transitions in intensity
levels can be seen only when there is random noise in the image. So, we perform
smoothing to remove small regions and sharp transitions (non-mango regions) in the
thresholded image of each of the R,G and B components separately and combined them.
We use a 3x3 mask for smoothing because it will not affect lager mango regions and also
it results in less blurring effect. The mango region segmentation is a two class problem, we
convert each smoothed R, G and B component into binary images where, white pixels
represent mango regions and black will represent non-mango regions. Then combine the
resultant images into a single binary image (B-image) as shown in the Fig.4. The
binarization process will fixes up a contrast break-point between pixels belonging to
mango regions and that of non-mango regions.
206
2.3
Template Matching
Here we use template matching technique to eliminate the non-mango regions in the
B-image. For this purpose, we extract the texture properties of the original image
from the regions located in the B-image and match with the templates stored in the
database. We retain only those regions whose texture properties are similar to
mangoes template. The template of mango is as shown in the Fig.5.
Texture features are the very fundamental and invariant properties of images. Each
image has its own texture properties which describe different image regions present in
it. In the image classification and image segmentation literatures texture properties of
images have been efficiently used. Statistical texture features proposed by Haralick et
al. [7] are used for template matching to eliminate the non-mango region in the
binarized image (B-image). Initially Gray Level Co-occurrence matrix (GLCM) is
computed for the gray image using the pair-wise occurrences of image resolutions.
And the various texture properties are calculated using the GLCM obtained. In our
frame work three different texture features are used. Let us assume that P is the gray
level co-occurrence matrix obtained from the image region rx, expressions for
different texture features which we have used are as follows.
(1). Contrast:
f 1=
N g 1
n =0
Ng Ng
n P(i, j)
i=1 i j = n j=1
(2). Correlation:
f 2=
i j (ij)p(i, j) xy
xy
(3). Entropy:
207
Notation:
p(i, j) is the (i, j) th entry in a normalized gray-tone spatial dependence matrix.
x , y , x and y are the means and standard deviation of px and py.
208
3.1
Dataset
Since, we could not find any dataset with mango tree image in the literature, we have
captured 44 natural mango tree images in a natural lighting condition using Kodak
digital camera (8.2-megopixel) with resolution 480x480dpi from different directions
and with the distance of less than five feet because, the image intensity at pixel
depends on the optical properties of the surface material, the surface shape and spatial
distribution of the incident illumination. The images we captured consists mango
leaves and branches in addition to mangos. We captured images under different
illumination conditions and occlusion of leaves on the mangoes. Sample images of
our own dataset is show in the Fig.7
(a)
(c)
(b)
(d)
3.2
Results
The quantitative results of mango region segmentation are computed on our own data
set using precision, recall and F-measures as given below,
Pr ecision(P) =
MRS
MRS+NMRS
(3)
Re call(R) =
MRS
AMR
(4)
Fmeasure =
2*P*R
P +R
(5)
Here MRS is the mango region segmented correctly, NMRS is the non mango
region segmented as a mango region and AMR is actual mango regions present in the
image (ground truth) and human segmented image considered as a ground truth. The
more segmented results are shown in Fig.8.
Input Image
209
Output image
Image 1
Image 2
Image 3
Image 4
Fig. 8. Segmented mango tree images with original input images
In the image (3) we exactly segment the mango region, but in the case of image
(1), image (2), and image (4) there is a over segmentation due to the occlusions of leaf
on the mango which is still a challenging issue in mango region segmentation. The
graphical representation of precision, recall and f-measures obtained for our data set is
as shown below Fig.9, Fig.10 and Fig.11 respectively.
210
211
In this paper, we have proposed a novel approach for mango region segmentation
from the mango tree image. The intensity of the pixel location depends on the optical
property and shape roughness of the objects present. Texture is the only feature to
achieve discrimination between objects in an image. Through this framework we have
exploited the intensity distribution and texture features of the objects in images for the
purpose of segmentation. The segmentation of mango regions from its tree image
taken in an unconstrained environment is a very challenging task. Presently, there are
no mango tree datasets available in the literature and hence we have created our own
dataset consisting of 44 mango tree images taken in an unconstrained environment.
The quantitative performance of the proposed method is analyzed and tabulated using
Precision, Recall and F-measures. We have achieved 72.77 % of Precision, 71.43 %
of Recall and 66.70% of F-measure. The poor results (image-20) are evidence in the
Fig.11 due to the problem of illumination, leaves shadows occluded on the mango
regions. In future, we would like to enhance the efficiency and accuracy of the
proposed method by making better use of texture features and with the addition of
features such as shape, spatial geometry and solidity etc.
References
1. Bramley, R.G.V., Proffit, A.P.B., Hinze, C.J., Pearse, B., Hamilton, R.P.: Generating
benefits from precision viticulture through selective harvesting. In: Proceedings of the 5Th
European Conference on Precision Agriculture, pp. 891898 (2005)
2. Ducournau, S., Feutry, A., Plainchault, P., Revollon, P., Vigouroux, B., Wagner, M.H.: An
image acquisition system for automated monitoring of the germination rate of sunflower
seeds. Computers and Electronics in Agriculture 44, 189202 (2004)
3. Zheng., L., Zhang., J., Wang, Q.: Mean-shift-based color segmentation of images containing
green. Vegetation Computers and Electronics in Agriculture 65, 9398 (2009)
4. Burgos-Artizzu., X.P., Ribeiro., A., Tellaeche., A., Pajares., G., Fernndez-Quintanilla, C.:
Analysis of natural images processing for the extraction of agricultural elements. Image and
Vision Computing 28, 138149 (2010)
5. Thorp, K.R., Dierig, D.A.: Color image segmentation approach to monitor flowering in
lesquerella. Industrial Crops and Products 34, 11501159 (2011)
6. Deng, Y., Manjunath, B.S.: Unsupervised Segmentation of Color-Texture Regions in
Images and Video. Pattern Analysis and Machine Intelligence 23, 800810 (2001)
7. Haralick, R.M., Shanmugam, K., Dinstein, I.: Texture Feature For Image Classification.
IEEE Transactions on Systems, Man and Cybernetics SMC-3, 610621 (1973)