Spatio Temporal Feature Detection - Fast 3D Detector

Pi19404
March 6, 2013

Contents

Contents
Spation Temporal Feature Detection - Fast 3D Detector 3

0.1 Introduction . . . . . . . . . . . . . . . . 0.2 Spatio-Temporal Features . . . . . . 0.2.1 Fast 3D Corner Detection . 0.2.2 Spatial Corners . . . . . . . . . . 0.2.3 Temporal Corners . . . . . . . . 0.2.4 Spatio Temporal Corner Test 0.2.5 Response Function . . . . . . . . 0.3 Code . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

3 3 4 4 5 6 7 7 8

2 | 8

Spation Temporal Feature Detection - Fast 3D Detector

Spation Temporal Feature Detection - Fast 3D Detector

0.1 Introduction
In this article we will look at the extension of fast accelerated segment test detector called fast 3D detector Koelstra and Patras, 2009 which can be used to detect robust spatio temporal corners quickly. The fast segmenet detector test enables us to discard non corners quickly and select possible corner locations quickly.For these corner locations a response function is computed and strongest corners are selected from this set of corners. The 3D Harris corner detector is used to detect spatial as well as temporal corners that can be detected over a sequence of frames.It finds spatial corners with velocity change which exhibit changes in space as well as time.However it is computationally expensive.

0.2 Spatio-Temporal Features

The aim of spatial feature detection is to identify point in the image that correspond to object in motion that can be tracked in adjacent image frames robustly. The aim of spatio-temporal feature detection is to identify points in the image where significant spatial and temporal intensity changes are observed.These locations provide a sparse representation of motion in the image and local neighborhood of these points can be used to represent motion occurring in the image.

3 | 8

Spation Temporal Feature Detection - Fast 3D Detector It is required that the feature be as sparse as possible to reduce the computational requirement and enable us to achieve real time tracking. Potential candidates for Spatial interest points are edge pixels,points with significant changes in the neighborhood and junction of corners. Potential candidates for Patio-Temporal interest points are points which exhibit large variation in spatial as well as temporal dimensions.

0.2.1 Fast 3D Corner Detection

For each pixel x; y given intensity I x; y and threshold  The Fast2D corner detector looks at circle of 16 surrounding pixels and detects a corner if 12 contiguous pixels have intensity all above I x; y  or all below I x; y  .

( )

( )

( )+

( )

For each pixel x; y; t with intensity I x; y; t and threshold  26 directional neighbors are considered in 3D spatial and temporal space. A corner is detected is proportion of surrounding pixels have intensities that are all above I x; y; t  or all below I x; y; t  .

)+

The contiguous pixels criteria is not suitable for 3D and is also computationally expensive. A corner is detected when atleast I x; y; t  or all below I x; y; t  .

)+

50%

of pixels are all above

The strength of the Fast corner detector was quickly rejecting invalid pixels based on the constraint that pixels are required to be contiguous. For Spatio Temporal features a constraint is imposed that pixel must encounter spatial as well as temporal change

4 | 8

Spation Temporal Feature Detection - Fast 3D Detector

0.2.2 Spatial Corners

If the point encounters temporal changes is checked by considering the pixels I x; y; t ; I x ; y; t ; I x ; y; t ; I x; y ; t ; I x; y ; t .If any of these intensity values are greater than I x; y; t  or less than I x; y; t  then it may be a possible corner candidate else reject the point as corner candidate.

) (

) ( +1

) (

+1 ) ( ( )+

1 )

I x

( + @x; y + @y; t) > I (x; y; t) +  orI (x + @x; y + @y; t) < I (x; y; t)  I (x + @x; y + @y; t) I (x; y; t) >  orI (x + @x; y + @y; t) I (x; y; t) <  I (x + @x; y + @y; t) I (x; y; t) >  or I (x + @x; y + @y; t) + I (x; y; t) >  jI (x + @x; y + @y; t) I (x; y; t)j >  P jI (x + @x; y + @y; t) I (x; y; t)j2 >  ( )+ 50%
(1)

The last criteria is 13 pixels must have intensities all greater than I x; y; t  or lower than I x; y; t 

Thus sobel tives then

we compute the derivatives along x and y direction using edge detection. Ans compute the sum of magnitude of derivain the x and y direction. If this exceeds a specified threshold point is a possible candidate for corner location

0.2.3 Temporal Corners

If the point encounters temporal changes is checked by considering the pixels I x; y; t I x; y; t I x; y; t

)(

1) (

+ 1)

5 | 8

Spation Temporal Feature Detection - Fast 3D Detector

I x; y; t

( + 1) or I (x; y; t 1) > I (x; y; t) +  I (x; y; t + 1) or I (x; y; t 1) < I (x; y; t)  I (x; y; t + 1) I (x; y; t) >  I (x; y; t 1) I (x; y; t) >  I (x; y; t 1) I (x; y; t) <  I (x; y; t + 1) I (x; y; t) <  I (x; y; t 1) + I (x; y; t) >  I (x; y; t + 1) + I (x; y; t) >  jI (x; y; t 1) I (x; y; t)j >  jI (x; y; t + 1) I (x; y; t)j >  P P1 1 jI (x; y; t + k) I (x; y; t)j > 
k ;

(2)

This can be checked by computing the temporal derivative using the kernel ; ; .

1 21

If the magnitude of temporal derivative is greater than threshold  then it is a possible candidate for corner location. The temporal filtering is only performed for points which satisfy the spatial corners criteria.

0.2.4 Spatio Temporal Corner Test

If the point satisfies both spatial and temporal corner test a set of 26 neighboring pixels is considered in the spatio temporal scale.A x spatio temporal block is considered. There are 26 pixels in the block leaving the center pixel.All the pixel are checked if their intensities are greater than I x; y; t  or less than I x; y; t  . A count is kept for both classes.

33

)+

6 | 8

Spation Temporal Feature Detection - Fast 3D Detector

0.2.5 Response Function

The above approach provides dense spatio temporal corners. from this local strong corners needs to be selected which are at specified minimum distance from each other.However corner strengths are not available. Hence corner strength needs to be evaluated using a suitable response function .Corner strength is evaluated using only spatial Harris corner constraint which is the response function.A local neighborhood around the corner point is chosen and the minimum eigen values of derivative matrix is evaluated which indicates the corner strength. Then corner filtering is performed to chose corners which are dominant and spaced at minimum distance from each other.The approach corner filtering is same used in earlier corner detector.

0.3 Code
we define a main feature_detector base class containing methods and data common to all feature detector. the fast_3d class is derived class containing specific implementations of algorithms. The code OpenCV code can be found in code repository https://github.com/pi19404/m19404/tree/master/FEATURE_DETECTOR or https://code.google.com/p/m19404/source/browse/FEATURE_DETECTOR/

7 | 8

Bibliography

Bibliography
[1] Chris Harris and Mike Stephens.  A combined corner and edge detector. In: In
Proc. of Fourth Alvey Vision Conference.

1988, pp. 147151.

[2]

S. Koelstra and I. Patras.  The FAST-3D Spatio-Temporal Interest Region Detector. In: Workshop on Image Analysis for Multimedia Interactive Services. 2009, pp. 242245.

[3]

Ivan Laptev.  On Space-Time Interest Points. In: Int. J. Comput. Vision 64.2-3 (Sept. 2005), pp. 107123.

7.
[4]

issn: 0920-5691. doi: 10.1007/s11263-005-1838url: http://dx.doi.org/10.1007/s11263-005-1838-7.

1994, pp. 593 600.

Jianbo Shi and C. Tomasi.  Good features to track. In: Computer Vision and
Pattern Recognition, 1994. Proceedings CVPR '94., 1994 IEEE Computer Society Conference on.

doi: 10.1109/CVPR.1994.323794.

8 | 8