IEEE Sponsored 2nd International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)‟15
Local shape and texture feature based moving object
classification in traffic surveillance scenario
B.Yogameena
Dept. of Electronics and Communication
Thiagarajar College of Engineering
Madurai, India
b.yogameena@gmail.com
Abstract — Computer vision techniques for traffic
surveillance have proved to be non invasive, automated,
and cost effective solutions to detect, count, and classify
vehicles, finding immense application in intelligent
transportation system. Since vehicles have high intra class
variation and relatively low interclass variation,
classification poses a great challenge, when the video
resolution is low, under varying illumination, occlusion etc.
To overcome these problems, a novel method for detection and
classification of moving objects in traffic surveillance scenes is
proposed. From the side view, the detected objects are classified
into pedestrians, two wheelers and cars. The moving object
segmentation is done using background subtraction. Features are
extracted from the segmented foreground using haar wavelets
and oriented centre symmetric local binary pattern (OCS LBP).
Hierarchical SVM classifier, trained using the extracted features,
classifies the detected objects. The proposed algorithm is able to
classify up to 97%.
Index Terms — Haar wavelets, OCS LBP. Contrast ratio,
Support Vector Machine
I. INTRODUCTION
In recent years computer vision technologies have found
wide applications in our day to day life for convenience
improvement. Real time moving object classification is an
important issue in video surveillance with a great potential
for use in various applications such as motion capture
techniques, Intelligent Transportation System (ITS), and
video interpretation. Intelligent transportation systems can
be used for monitoring traffic conditions, reducing
congestion, enhancing mobility, optimizing the use of
transport infrastructure and improving security. The
detailed classification of moving objects can draw traffic
forecasts and advise the best departure times in different
regions, and alternative routes for major roads. Object
classification in surveillance scenes is challenging for the
following aspects. Primarily, the region of interest in
surveillance scenes is of reduced size and low resolution
due to the capacity of surveillance cameras. The intra class
variance for each class is very large than inter class
variation. Object appearance is highly varied for different
view angles and illuminations. Therefore, real time
performance is limited by such complexities.
1305
G. Seethalakshmi
Dept. of Electronics and Communication
Thiagarajar College of Engineering
Madurai, India
sgseethalakshmi@gmail.com
Various studies have been done in this field using different
features. Viola Jones algorithm made use of haar like
features for rapid object detection [1]. It suffers from high
false positive rate. The HOG features have been used to
capture local shape characteristics [2]. But, the HOG
features involve huge computations. LBP is a robust texture
descriptor, which is computationally efficient [3].
Combination of HOG and LBP takes advantage of each of
the descriptor, but this approach increases the feature
dimension [4]. Centre symmetric LBP reduces the
computational load by half that of LBP [5]. CS LBP
provides a measure of intensity difference of each given
pixel. Since orientation of each pixel is lost, a new lower
dimentional feature oriented CS LBP, known as Oriented
Centre Symmetric Local Binary Pattern (OCS LBP) was
devised [6]. The proposed work combines the shape
features obtained from haar wavelets and texture features
extracted by OCS LBP to obtain a descriptor that provides
better performance and reduced features..
II. METHODOLOGY
The primary step in this work is the segmentation of the
moving objects from the video frames. For this a simple
background subtraction technique is used. Subsequently,
shadow removal is done using contrast enhancement, to extract
only actual foreground blobs. Shape feature and texture
features are extracted using Haar DWT sub bands and
OCSLBP. Classification is done using hierarchical binary
SVM classifier. Fig. 1 shows the proposed methodology.
INPUT VIDEO
MOVING OBJECT
SEGMENTATION
SHAPE FEATURE EXTRACTION
USING HAAR DWT
TEXTURE FEATURE
EXTRACTION USING OCSLBP
CLASSIFICATION USING
BINARY SVM CLASSIFIER
Fig. 1 Proposed methodology
 IEEE Sponsored 2nd International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)‟15
A. Moving object segmentation
First, the foreground is to be found for further processing
like feature extraction and classification. Assuming that
surveillance video is captured by a fixed camera, the
moving objects are segmented using background
subtraction technique [8]. The difference between current
frame 𝐼𝑛 and the background frame 𝐵𝑛 is found to detect the
moving objects.
0, 𝑖𝑓 𝐼𝑛 𝑥, 𝑦 − 𝐵𝑛 𝑥, 𝑦 < 𝑇ℎ
𝐷𝑛 𝑥, 𝑦 =
1, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
where Th is a predefined threshold and chosen as the
average of the difference image Dn. After background
subtraction, simple morphological operations are performed
for noise removal.
B. Shadow Removal
A cast shadow is a semi-transparent mask that has the same
texture of part of background and it changes the color
appearance. So, the three RGB colour components,
decreases and becomes less dispersed on a shadowed
region. This variation depends on the time of day and the
scene geometry. By making these two distributions
(shadow and light regions) similar, shadow removal is
done. A contrast ratio measure is used to make the
distribution of shadow region similar to the background in
terms of the dispersion and darkness [11]. Contrast is a
distinctive measure of the luminance distribution of an
image. The contrast is formulated as
1 𝑁−1 𝑀−1
𝐶= 𝑖=0 𝑗 =0 (𝐼 𝑖, 𝑗 − 𝐼𝑚 )2
𝑁𝑋𝑀
where N and M are the dimensions of image, is I(i,j) the
intensity value at the pixel (i,j) and Im is the intensity mean of
image. A candidate shadow region is determined and the
contrast of the region without shadow Ck , the contrast of the
region with shadow C sh is found from the above equation.
The contrast ratio C r k is then found for each color component
(R, G, B) as
𝐶
𝐶𝑟 𝑘 = 𝑘 𝐶
𝑠ℎ
By multiplying this ratio to the extracted foreground in the
respective color components, the shadow regions become
nearly similar to the background. Foreground segmentation is
again performed and thus shadow is removed. Figure 2 shows
the effect of shadow removal.
Fig. 2 Shadow removal using contrast ratio method
C. Feature extraction
Haar DWT (Discrete Wavelet Transform) features are used
in order to extract shape information. The mother wavelet
function is given by
1 0 ≤ t < 1/2
Ψ( t ) = −1 1/2 ≤ t < 1 (4)
0 otherwise
The scaling function is given by
1 0 ≤ t < 1
φ(t) = (5)
0 otherwise
(1)
The above scaling and wavelet basis functions decomposes an
image into four square sub-bands, which includes the average
sub-band(LL1), horizontal sub-band (LH1), vertical sub-band
(HL1) and diagonal sub-band (HH1). The average sub-band
LL1 is further decomposed into the sub bands LL2, LH2,
HL2, and HH2 as shown in Figure 3. The sub bands provide
information regarding the vertical, horizontal and diagonal
edges and are thus used to extract shape features. The sub
bands in level two are used for obtaining a better low
dimensional descriptor [9]. The LSF(local shape feature)
vector is given by Eq. 6
𝑖
𝐹𝑠ℎ𝑎𝑝𝑒 𝑥, 𝑦 = 𝐿𝐻2𝑖 𝑥, 𝑦 + 𝐻𝐿𝑖2 𝑥, 𝑦 + 𝐻𝐻2𝑖 𝑥, 𝑦
1≤ x≤4,1≤y≤4 (6)
Where (x,y) denotes pixel position. For each block the feature
size is 16.
In order to make use of texture features, Oriented centre
symmetric Local Binary Pattern is chosen as the descriptor
[6]. Local Binary Pattern is a robust texture descriptor, but it
(2) produces long histograms. In order to reduce the computations
CSLBP (Centre symmetric local binary pattern) was
developed. CSLBP reduced the computations by half that of
LBP, but orientation of pixel is lost. Therefore, OCSLBP
scheme is chosen as texture descriptor [7]. To extract an
oriented histogram of OCS-LBP from a sub block, the
gradient orientations are calculated at every pixel and
histogram of each kth orientation in the neighborhood is
binned using Eq. 7 and Eq. 8
(3)
𝑠 𝑥 = 𝑥 𝑖𝑓 𝑥 >𝑇 (7)
𝑘
𝑂𝐶𝑆 𝐿𝐵𝑃𝑅,𝑁 𝑥, 𝑦 = 𝑠(𝑛𝑖 − 𝑛𝑖+𝑁 ) (8)
2
0 < i < (N/2)-1
K= 0, 1, 2, 3, 4, 5, 6, 7
where ni and ni+(N/2) are the intensity values of the center-
symmetric pairs of pixels, for N equally spaced pixels in a
circle with radius R and k is the bin number of gradient
orientation. Fig. 4 shows the extraction of OCS LBP
descriptor. When the difference between centre symmetric
1306
 IEEE Sponsored 2nd International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)‟15
pixels is greater than a given threshold, the gradient
orientation is confirmed. To reduce the computation time,
integral histogram is used to preserve the local features [10].
Fig. 3 Sub bands of Haar DWT
The integral histogram at a given point is obtained by copying
the previous values and then increasing the corresponding bin
with respect to the current value of the point, instead of
generating all of the histograms for every given pixel. Thus
OCS LBP descriptor with dimension 8 is obtained.
The new feature that combines the Local shape features
(extracted from haar wavelet transform) and the texture
features (from OCSLBP) by concatenation is proposed to
classify the moving object. As a result, the total number of
features in each block is 16 (from the LSF) + 8 (from the OCS
LBP) = 24. These features are fed to the SVM classifier for
classification.
D. Classification
Using the extracted features, classification of moving
objects in to pedestrians, two wheelers and cars is done by a
linear hierarchical SVM (Support Vector Machine) classifier.
The proposed classification scheme is shown in Figure 5.
Three linear binary SVMs are used. The output of each linear
SVM is is given as
𝑖
𝑦𝑖 𝑥 = 𝑁 𝑘=1 𝑤𝑘 𝑥𝑘 + 𝑏𝑘 (9)
where xk= [x1, ..., xN] is the feature vector, N is the feature
dimension and wki and bk are the coefficients trained through
the SVM.
In training phase, N samples for each of the class, like
pedestrians, cars and two wheelers are taken which totals to
3N training samples.
Fig.4 Extraction of OCS LBP descriptor
1307
While training, for the first linear SVM, the desired output is
“1”for the N training samples of the pedestrian class, and “−1”
for the remaining samples of other two classes. In a similar
way, other two SVMs are trained.
In the testing phase, the first linear SVM tests whether
detected object is pedestrian. If the output is -1, then the
second SVM in level two checks for two wheeler. If it is not
classified as two wheeler, then the third SVM classifies as car
or other vehicle.
III. RESULTS AND DISCUSSIONS
The performance of the proposed algorithm is tested with
the benchmark dataset collected on national highway,
consisting of 1000 frames. The proposed approach was
implemented using MATLAB software.
The moving objects are segmented using background
subtraction. For the resulting binary image bounding box is
drawn and the rectangular coordinates for the moving objects
are obtained. Shadow removal is done by multiplying the
foreground with the contrast ratio. Although this step
illuminates the objects of interest, this does not affect the
features.
After foreground segmentation and shadow elimination,
feature extraction is done, taking in to account the shape and
texture of an object. In order to extract the shape features, haar
wavelets in sub band two are utilized since edge information
can be obtained with reduced dimensions. Texture features are
obtained using OCS LBP descriptor, from which gradient
magnitude and orientation can be inferred. A hierarchical SVM
classifier is trained using samples. Linear SVMs are used to
reduce the computational load. The detected objects are finally
classified as shown in the Fig. 7
Performance is analyzed using overall accuracy rate in Eq. 10.
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
Accuracy rate =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑒𝑠𝑡 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
(10)
 IEEE Sponsored 2nd International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)‟15

PEDESTRIAN
OBJECT FEATURE TWO
SVM 1
IMAGE EXTRACTION WHEELER
NON
SVM2 CAR
PEDESTRIAN
NOT A TWO
SVM3
WHEELER
OTHER
VEHICLE

Fig. 5 Classification using SVM classifier

The overall accuracy rate of the above method is 97%. A

comparison of different methods making use of various
descriptors is shown in Table 1.Feature dimensions are
reduced, to ease the computational load, without compromising
the classification performance. Table 2 shows the feature
dimensions for various descriptors.The number of false
positive occurrences have been reduced as shown in Fig. 6.

IV. CONCLUSION
In this paper, a combination of the new feature descriptor
(LSF+OCS LBP) with the hierarchical linear SVM classifier is
proposed for the classification of moving object. The proposed
feature descriptor provides better classification with reduced
dimensionality. Additionally, the shadow removal technique
and hierarchical SVM classifier configuration have reduced the
number of false classifications. Further improvement on this
work can be done on view independent vehicle detection in
occluded areas.

600
500
400
300
200
100
0

Fig. 7 Classified objects using hierarchical SVM classifier

Fig. 6 Number of false classifications

1308
 IEEE Sponsored 2nd International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)‟15

TABLE I. PERFORMANCE COMPARISON OF VARIOUS

[7] Mi Ra Jeong, Jun-Yong Kwak, Jung Eun Son, ByoungChul Ko,
CLASSIFICATION SCHEMES
Jae-Yeal Nam, “Fast pedestrian detection using a night vision system
Proposed for safety driving”, IEEE International Conference on Computer
HOG method(Haar Graphics, Imaging and Visualization, pp 69-72, August 2014.
Method HOG Wavelet LBP [8] Jun-Wei Hsieh, Shih-Hao Yu, Yung-Sheng Chen, Wen-Fong Hu,
LBP DWT +
OCSLBP) “Automatic traffic surveillance system for vehicle tracking and
classification”, IEEE Transactions on Intelligent transportation
Accuracy rate 94.3% 92% 94% 95% 97% systems, Vol. 7 no 2, pp 175-187, June 2006.
[9] Chung-Wei Liang, Chia-Feng Juang, “Moving object
No of false classification using local shape and HOG features in wavelet-
classifications
215 252 223 192 98 transformed space with hierarchical SVM classifiers”, Elsevier,
Applied Soft Computing, Vol 28, pp 483-97, March 2015, in press.
[10] ByoungChulKo, Ji-HunJung, Jae-YealNam,“View-independent
object detection using shared local features”, Elsevier, Journal of
TABLE II. FEATURE DIMENSIONS OF VARIOUS DESCRIPTORS Visual Languages & ComputingVol 28 pp 56-70, June 2015, in press.
[11] H. Asaidi, A. Aarab, M. Bellouki, “Shadow elimination and
Method Feature dimension per block vehicles classification approaches in traffic video surveillance
context”, Elsevier, Journal of Visual Languages & Computing, Vol
HOG 4X9 25, Issue 4, pp 333-345, August 2014.

Wavelet 3

LBP 59

HOG LBP 36(HOG) + 59(LBP)

Proposed method (Haar DWT +
16(LSF)+ 8(OCS LBP)
OCSLBP)

ACKNOWLEDGMENT
This work has been supported by UGC under Major Research
Project Scheme entitled, „Intelligent Video Surveillance
System for Human Action Analysis‟ with reference F.No.41-
592/2012 (SR).

REFERENCES

[1] P. Viola, M. Jones, “Rapid object detection using a boosted

cascade of simple features”, IEEE Trans. Comput. Vis. Pattern
Recognit. 1 (2001) 511–518.
[2] N. Dalal, B. Triggs, “Histogram of oriented gradients for human
detection”, Proc.IEEE Int. Conf. Comput. Vis. Pattern Recognit.
(CVPR) (2005) 886–893.
[3] T. Ojala, M. Pietikainen, D. Harwood, “A comparative study of
texture measures with classification based on featured distributions”,
Pattern Recognition, Vol 29 (1) , pp 51–59, 1996.
[4] X. Wang, T.X. Han, S. Yan, An HOG-LBP human detector with
partial occlusion handling, in: Proc. IEEE Int. Conf. Comput. Vis.
(ICCV), pp. 32–39, 2009.
[5] M.Heikkilä, M.Pietikäinen, C.Schmid, “Description of interest
regions with center-symmetric local binary patterns”, Pattern
Recognition, 42(3)(2009)425–436.
[6] Byoung Chul Ko, Joon-Young Kwak, Jae-Yeal Nam, “Human
tracking in thermal images using adaptive particle filters with online
random forest learning”, Proceedings of SPIE, Optical engineering,
Vol. 52(11), November 2013.

1309