See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/254819945

CAMERA OBJECT TRACKING FAST ALGORITHM

Article

CITATION READS

1 44

3 authors, including:

Nikola Teslic
RT-RK Computer Based Systems
121 PUBLICATIONS   662 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

hysteresis phenomenon View project

Human-Device Interface Research View project

All content following this page was uploaded by Nikola Teslic on 05 June 2014.

The user has requested enhancement of the downloaded file.

 CAMERA OBJECT TRACKING FAST ALGORITHM
1
Nikola Teslic, 2Vladimir Radenkovic, 3Snezana Crnogorac
1
Novi Sad Institute of Information Technologies MicronasNIIT, Fruskogorska 11a, 21000
Novi Sad, Serbia and Montenegro, Nikola.Teslic@MicronasNIT.com
2
Novi Sad Institute of Information Technologies MicronasNIIT, Fruskogorska 11a, 21000
Novi Sad, Serbia and Montenegro, Vladimir.Radenkovic@MicronasNIT.com
3
Novi Sad Institute of Information Technologies MicronasNIIT, Fruskogorska 11a, 21000
Novi Sad, Serbia and Montenegro, Snezana.Crnogorac@MicronasNIT.com

ABSTRACT
This paper addresses camera real-time object tracking
problems. Here we present one camera object tracking
solution based on block-matching technique. In our solution
object is marked in the first frame, and afterwards its
movement is tracked using the block-matching technique in
limited area of the picture. The size of limited area depends
on predicted object movement between two frames. We are
using block-matching technique in reduced number of pixels,
searching for the best matched square blocks in two
successive frames. In the first frame, square block is
centered to the object that we are tracking. Square block
center difference in two neighboring frames determines
object movement between these two frames. Our algorithm
does not depend on position between object and camera and
can be used in wide range of light brightness. Object
tracking is possible even when other objects are moving near
object of interest. Algorithm is implemented on PC platform
with minimal additional hardware resources. Stepper motor
is used for camera movement.
1. INTRODUCTION
Object or human tracking is a complex problem in
field of image processing. Firstly, object of interest
needs to be separated from other objects in the picture,
and after that it needs to be tracked. This paper will
present some object tracking techniques that we were
considering, and some problems involved.
The object tracking technique based on color is
widely used technique [1] [2] [5]. The main problem is
determining human skin color, if tracking object is
human, or determining the exact color of any other
object in general. HSI color space is used more often
than RGB color space, because color is represented
only with H in HSI color space [3] [5]. If tracking
object is human, than usually the face is being tracked.
H-value of the skin color is in the range from 0 to 50,
or ranging from red to yellow color [2]. During that,
certain problems arise with other parts of the body,
such as arms, parts of the clothes or the surrounding
area that have the same color as the human face. On
the other hand, the face of the person being tracked
must be turn toward the camera during the process.
The other factor that influences the effectiveness of
application is the fact that the color depends on the
general level of light brightness of the area in which
the object is moving.
Object could also be tracked based on its shape
[4]. Human body is not a rigid object, so the body
shape during movement changes from frame to frame,
and that also makes the tracking more difficult.
Commonly used object tracking technique is
tracking based on the object motion between two
frames. The tracking based on the motion between two
frames is just the calculation of differences between
pixel characteristics of two successive frames [3]. If
human face is being tracked, it is difficult to separate
the movements of the face from the movements of the
rest of the body. On the other hand, in the observed
area there are many objects that are not of interest, and
that are difficult to separate from the object of interest.
With this technique there is one additional problem,
during the tracking, the camera moves and that creates
the movement of the still area that must be
compensated for [6].
Our solution uses the block-matching technique
that eliminates all aforementioned problems and
ensures real-time tracking with minimal hardware
resources. It was developed based on PII (700MHz)
platform. The camera used was Logitech QuickCam
Express USB color web camera with 15 frames per
second. Stepper motor is used for camera movement.
2. COLOR-BASED OBJECT TRACKING
TECHNIQUE
This chapter will present our solution of object
tracking based on color.
In the first frame object was marked, and R, G and
B values of object pixel were read. In the second
frame, all pixels that are in the range [+20, -20] of R,
G and B values of pixel from the first frame were At first, each of two successive frames, with even
separated. Considering that the object color lines from i-frame and odd lines from (i+1)-frame, are
significantly changes in the real situations because of combined. With this, parts of the picture that have
different light brightness of the area in which the changed due to the object movement are separated.
object is moving; it was necessary to update the color After that, pixels from newly combined picture are
from frame to frame. Updating is realized by separated using following expression:
calculating average R, G and B values of separated
pixels. In order to speed up the processing and to 2 ⋅ Y2 − DIL − ER > 50 , (1)
decrease the possibility of separation of similar
colored objects, the object was separated in the limited where:
area of the picture. The size of the limited area
depends on predicted object movement between two
DIL = max(Y1 , Y2 , Y3 ) , (2)
frames.
After object was separated, its motion between ER = min(Y1 , Y2 , Y3 ) . (3)
two frames is calculated as a difference between
Y1, Y2 and Y3 are the values of pixel’s luminance
centers of object’s circumscribed rectangle.
component for every 3 successive pixels in vertical
The use of HSI color space in the color based
line of the picture. Expression (1) is obtained
object tracking technique makes tracking more
experimentally.
efficient then when using RGB color space.
Even when we have applied aforementioned
limitations we could not perfectly separate object of
interest from other similar moving objects. Figure 1
shows good object separation when there are no other
unwanted similar objects in its vicinity, and Figure 2
shows bad object separation when there is unwanted
object in object’s vicinity. These two figures show
results gained by using HSI color space. a) b)

c) d)
Figure 1. Good object separation (there are no Figure 3. a) Frame i, b) frame (i+1), c)
unwanted objects in its vicinity). combined picture, d) separated area that
represents area of object movement.
Figure 3 shows the separation of object’s
movement area between two frames.
Object movement is determined like brunt’s
difference of separated pixels between two combined
pictures. Brunt is calculated upon general brunt
expression:
Figure 2. Bad object separation (there is
W 
unwanted object in its vicinity).
T X =  ∑ NWi ⋅ i  N , (4)
 i =1 
3. OBJECT TRACKING TECHNIQUE
BASED ON OBJECT MOTION  H 
BETWEEN TWO FRAMES TY =  ∑ NH i ⋅ i  N . (5)
 i =1 
This chapter will present our solution of object W and H are picture width and height, NWi and NHi
tracking based on object motion between two frames. are numbers of picture’s separated pixels in
column and row i, and N is overall number of picture’s
separated pixels.
We were encountered two problems using this
technique. The first problem was separating object of
interest from other objects in its vicinity that are not of
interest and which are also moving. We have marked
the object in the first frame and after that we have
Figure 4. The position of the block, in relation to the tracked object in five frames.
Figure 5. The position of the block, in relation to the tracked object in five frames, when there are similar objects
in its vicinity
4. OBJECT TRACKING BASED ON
BLOCK-MATCHING TECHNIQUE
With object tracking based on block-matching
technique we have succeeded in eliminating all
aforementioned problems of other techniques.
Block-matching technique is widely used in image
processing [7] [8]. Square block of specific size which
center is reference pixel, is compared with square
blocks of same size in subsequent frame in the vicinity
of reference pixel from previously frame. Purpose of
block comparison is to find best matched blocks in two
successive frames.
In our solution we have used 40x40 pixel square
block that we have compared in 80x80 pixel area.
The area in which matching takes place is
predicted area of object movement between two
frames.
In the first frame, block is centered to the object
that is being tracked, and in the second frame the most
similar block to the block from the first frame is
searched and matched by block-matching technique. In
the third frame, the most similar block to the block
from the second frame is searched and matched by
block-matching technique and etc. Block’s updating
separated it in limited area of other frames, and in that
way we have partially solved this problem. The size of
limited area depends on predicted object movement
between two frames. Second problem arises due to
camera movement. During the tracking, the camera
moves and that creates the movement of the still area
that must be compensated for.
from frame to frame solves problem about human
shape that changes during movement.
Block’s comparison between successive frames is
done via summed absolute difference of pixel
characteristics within blocks. As every pixel is
represented with three characteristics: blue (B), green
(G) and red (R), absolute difference is average
difference of R, G and B values. The block that has the
lowest summed absolute difference is the best-
matched block. Also, the summed differences in
subsequent frames must be lower then summed
differences for no motion case by given threshold. In
that way we have avoided cases when there is a
uniform area of the picture to miscalculate motion
direction of the object.
When the best matched blocks are determined,
motion direction of object is the difference between
centers of those two blocks.
Figures 4 and 5 show positions of the blocks in
relation to the tracked object, with fixed camera,
during five frames. Figure 5 shows that other similar
unwanted objects moving in the vicinity of the tracked
object do not interfere with the tracking.
In order to speed-up the image processing the
number of pixels inside the block, which we have used
to calculate the summed difference, was reduced. The
number of referent pixels in the area (80x80 pixel) in
which the object moves was reduced 4 times.
 Figure 6. Object tracking using block-matching technique while the camera moves in 10 frames.

The autonomous robot caddy is designed to be a

5. CAMERA CONTROL
For the camera motion we have used the standard
stepper motor (1.8º per step, unipolar) that moves the
camera on the horizontal axis from -180º to +180º.
Based on the information of the object movement
obtained by block-matching technique, PC generates
control signals to stepper motor driver that controls the
camera. One signal controls the camera direction
movement (right or left), and the other signal controls
the number of steps that stepper motor makes. The
adjustment of the impulse and the pause of the second
signal influences the speed of the motor. We have
achieved the smallest possible delay (the biggest
replacement for a human golf caddy. Robot caddy
main functions are: to carry golf clubs, to follow its
master after receiving command to move and to be still
while its master plays.
In order to apply our algorithm on robot caddy, it
is necessary that the robot is equipped with camera.
Robot follows its master using movement information
obtained from our algorithm, supplied to robot’s
moving control algorithm.
Figure 7 shows robot caddy.
7. EXPERIMENT
Color-based object tracking technique

possible number of frames per second) by adjusting 300 True object movement -------
the impulse and the pause (pulse duty factor) of the Object tracking .......

second signal. 250

Figure 6 shows the object tracking using block-

matching technique while the camera moves. 200
x [pixel]

150
6. USAGE
100

50

0
0 100 200 300 400 500 600 700 800
Number of frames

(a)

Figure 7. Robot caddy.

Algorithm presented in this paper, among other
applications, can be applied on autonomous robot
caddy.
 300
Error of color-based object tracking technique Figure 9. (a) Comparison of the true object
movement and object tracking based on object
250
motion between two frames, on the horizontal
axis, in the 850-frame sequence, (b) mean-
square error of object tracking technique based
Mean-square error [pixel]

200
on object motion between two frames, in the
150
850-frame sequence.
Object tracking based on block-matching technique

100
300 True object movement -------
Object tracking .......

50
250

0 200
0 100 200 300 400 500 600 700 800
Number of frames

x [pixel]
150
(b)
100
Figure 8. (a) Comparison of the true object
movement and color-based object tracking, on 50
the horizontal axis, in the 850-frame sequence,
(b) mean-square error of color-based object 0
tracking technique, in the 850-frame sequence. 0 100 200 300 400 500
Number of frames
600 700 800

Object tracking based on object motion between two frames (a)

Error of object tracking based on block-matching technique
300 True object movement -------
300
Object trcking .......

250
250

200
Mean-square error [pixel]

200
x [pixel]

150
150

100
100

50
50

0
0 100 200 300 400 500 600 700 800
Number of frames 0
0 100 200 300 400 500 600 700 800
(a) Nuber of frames

Error of object tracking technique based on motion between two frames (b)
300

Figure 10. (a) Comparison of the true object

250 movement and object tracking based on block-
matching, on the horizontal axis, in the 850-
Mean-square error [piksel]

200 frame sequence, (b) mean-square error of

object tracking based on block-matching
150 technique, in the 850-frame sequence.

100
All three techniques were tested in the same 850-
50
frame sequence, in order to show validity of them and
to compare them. In the frame sequence from 0 to 600
0
object is moving, and in its vicinity other similar
0 100 200 300 400 500
Nuber of frames
600 700 800
objects are also moving. In the frame sequence from
600 to 850 object is rotating.
(b)
Figures 8a, 9a and 10a show the curve of true
object movement and the curve of object tracking
obtained by certain technique, on the horizontal axis,
 in the 850-frame sequence. As curves are more
matched tracking technique is better. Figures 8b, 9b
and 10b show mean-square error of object tracking
when object is being tracked with certain technique.
Mean-square error contains error of object movement
on both, horizontal and vertical axis.
From Figure 8 can be concluded that with
application of color-based tracking technique, object
can be tracked very efficiently when in object’s
vicinity there are no similar objects. Around 460th and
540th frame we have significant deviation between the
curve that represents true object movement and the
curve that represents color-based object tracking. The
reason for this deviation is that in those frames in the
vicinity of the object was unwanted object of similar
characteristics. From the graph it can be seen that
color-based tracking technique is not efficient during
the rotation of the object (the face is not turn toward
the camera).
Figure 9 shows that the tracking technique based
on object motion between two frames is much more
sensitive to movement of similar objects in its vicinity
in comparison to color-based object tracking
technique. The tracking technique based on object
motion between two frames handles the tracking of the
rotation of the object more efficiently.
Figures 10 shows that the block-matching
technique is very efficient when there are similar
objects in the tracked object’s vicinity, but also it is
very efficient if the tracked object is rotating. The
graph 10b shows that the mean-square error of the
block-matching technique is the smallest (mean-square
error contains error of object movement on both,
horizontal and vertical axis).
8. CONCLUSIONS AND FUTURE
THOUGTHS
This paper addresses camera object tracking
technique. Object is marked in the first frame, after
that it is separated and tracked by camera. Our
algorithm is able to track any moving object, and
because of that object must be marked in the first
frame.
Pixel’s characteristics that represent moving
object during tracking are updated from frame to
frame, and because of that our algorithm can be used
in wide brightness range of light in the area. In that
way, the tracking is possible both, in the daylight and
in the indoor light.
Due to the fact that the square block is updated
from frame to frame in tracking based on block-
matching technique, object that is being tracked can
change its shape (no need for object that is being
tracked to be rigid). In cases when object that is being
View publication stats
tracked is human, there is no need for object to be
face-turned toward camera.
Algorithm can be upgraded, for example some
preprocessing can be added in order to emphasize
certain distinguishable characteristics of the object,
and which would in turn make the tracking more
efficient.
If the camera is mounted on a moving robot, the
complete system could be autonomous.
9. LITERATURE
[1] S. Lenser, J. Bruce and M. Veloso, “CMPack: A
Complete Software System for Autonomous Legged
Soccer Robots,” in Proc. of the Fifth International
Conference on Autonomous Agents, Montreal, Canada,
May 2001.
[2] C. Garcia, G. Tziritas, “Face Detection Using Quantized
Skin Color Regions Merging and Wavelet Packet
Analysis,” IEEE Trans. On Multimedia, 1(3), September
1999, p.264-277.
[3] W. J. Hong, “Camera/Tracking Software
Documentation,” work done under the KOSEF/NSF
Summer Research Program at the Korea Advanced
Institute of Science and Technology (KAIST), August
1998.
[4] Yu Zhong, Anil K. Jain, Marie-Pierre Dubuisson-Jolly,
“Object Tracking Using Deformable Templates,” IEEE
Trans. on Pattern Analysis and Machine Intelligence,
22(5): 544-549, 2000.
[5] V. Krüger, R. Herpers, K. Daniilidis and G. Sommer,
“Teleconferencing using an attentive camera system, ”
Proc. of 2nd Int. Conf. on Audio- and Video-based
Biometric Person Authentication, University of
Maryland, USA, pp. 136-141, 1999.
[6] K. Daniilidis, Ch. Krauss, M. Hansen, and G. Sommer,
“Real-Time Tracking of Moving Objects with an Active
Camera,” Journal of Real Time Imaging, 4:3-20, 1998.
[7] J. Lu and M. L. Liou, “A simple and efficient search
algorithm for block-matching motion estimation,” IEEE
Trans. Circuits Syst. Video Technol, vol. 7, no. 2, pp.
429-432, April 1997.
[8] Jie-Bin Xu, Lai-Man Po and Chok-Kwan Cheung,
“Adaptive motion tracking block matching algorithms
for video coding,” IEEE Trans. on Circuits and Systems
for Video Technology, vol 97, pp. 1025-1029, October
1999.