Automated Target Recognition

Using Mathematical Morphology

Prof. Robert Haralick
Ilknur Icke
Jos Hanchi
Computer Science Dept.
The Graduate Center of CUNY
Outline
Gray Scale Morphology
Converting Images to Datasets
Decision Tree Classifier
Results / Conclusions
Outline
Gray Scale Morphology
Converting Images to Datasets
Decision Tree Classifier
Results / Conclusions
Mathematical Morphology
Given an image I I _ E
N
and a structuring element S S _E
N
define the morphological operation of
Dilation
} , , | { s i c S s I i E c S I
N
+ = n n =
} , | { ) ( x i c I i E c I
N
x
+ = n =
and set translation as
x x
S I S I ) ( ) ( =
Dilation is translation invariant
Mathematical Morphology
I
S
S I
Dilation
Mathematical Morphology
Define the morphological operation of
Erosion
} , | { I s x S s E x S I
N
+ V =
Erosion is translation invariant
x x
S I S I ) ( =
x x
S I S I

= ) (
Mathematical Morphology
If a structuring element can be decomposed as
Y X S =
then
Y X I Y X I S I = = ) ( ) (
Y X I Y X I S I = = ) ( ) (
Basic Morphology Operators
Opening
Closing
S S I S I = ) ( Q
S S I S I = - ) (
S I S S I Q Q Q = ) (
S I S S I - = - - ) (
Gray Scale Morphology
Dilation of f by k
)} ( ) ( {
max
) )( ( z k z x f x k f
F z x
K z
+ =

Erosion of f by k
)} ( ) ( {
min
) )( ( z k z x f x k f
K z
+ =

Gray Scale Morphology

Opening of f by k
k k f k f = ) ( Q
k k f k f = - ) (
Closing of f by k
We have used flat structuring elements
of size { 3,5,7,9,11,13,15,17,19,21 }
Structuring Elements Used
H
w
= Horizontal
V
h
=Vertical
B
wxh
= Box
h = 5
w = 5
an illustration
Dilation
9
H I
w = 9
Erosion
9
H I
w = 9
Opening
9
H I Q
w = 9
Closing
9
H I -
w = 9
The van-Herk-Gil-Werman (HGW)
Algorithmdilation
STAGE 1
Given the input signal stream and a flat structuring
element of size = 3
x
0
, x
1
, x
2
, x
3
, x
4
, x
5
, x
6
, x
7
, x
8
, x
9
, x
10
, x
11
, x
12
, x
13
,
9 4 5 14 21 18 12 7 8 3
center segments located at
x
0
, x
1
, x
2
, x
3
, x
4
, x
5
x
6
, x
7
, x
8
x
9
, x
10
, x
11

example:
take the first segment and find the max (i.e. dilation)
The van-Herk-Gil-Werman (HGW)
Algorithmdilation
STAGE 1.a
9 4 5 14 21
5
copy
9
max
R
0
5
R
1
5
R
2
9 4
max
preprocess the prefixes
x
0
x
4
x
2
x
1
x
3
The van-Herk-Gil-Werman (HGW)
Algorithmdilation
STAGE 1.b
9 4 5 14 21
5
copy
21
max
14
max
preprocess the suffixes
S
0
5
S
1
14
S
2
21
x
0
x
4
x
2
x
1
x
3
The van-Herk-Gil-Werman (HGW)
Algorithmdilation
STAGE 2
9 4 5 14 21
R
0
5
R
1
5
R
2
9
merging prefixes and suffixes
S
0
5
S
1
14
S
2
21
max
99
max
14 14
max
21 21
p
p
p
p p 4 3 ) 2 ( ) 1 ( 2
=
+
number of max operations
per window:
x
0
x
4
x
2
x
1
x
3
The van-Herk-Gil-Werman (HGW)
Algorithmdilation
Processing a given input signal for p=3 , segment size=5
x
0
, x
1
, x
2
, x
3
, x
4
, x
5
, x
6
, x
7
, x
8
, x
9
, x
10
,
9 14 21 21 21 18 12 8 12
9 4 5 14 21 18 12 7 8 3 12
Calculating Morphological
Features in 2-D
The HGW algorithm works on 1-D input
To apply it to 2-D images apply
Horizontal Structuring Elements
process the image line by line
Vertical Structuring Elements
transpose the image
process line by line
transpose again
Box shaped Structuring Elements
horizontal first, then vertical
Efficiency of Flat Structuring Elements
Given the flat structuring elements H and V
Dilation
Erosion
Opening
Closing
Since and given w = h
Dilation
Erosion
Opening
Closing
H I
H I
H I Q
H I -
V I
V I
V I Q
V I -
h w h w
V H B =
-
V H I B I = ) (
V H I B I = ) (
V H V H I B B I B I = = ) ) ) ((( ) ( Q
V H V H I B B I B I = = - ) ) ) ((( ) (
O
r
i
g
i
n
a
l

I
m
a
g
e
Dilation
with H
5
Erosion
With
H
5
Opening
With
H
5
Closing
With H
5
Using V
5
Structuring Element
erosion dilation opening closing
Using B
5x5
Structuring Element
dilation erosion opening closing
Outline
Gray Scale Morphology
Converting Images to Datasets
Decision Tree Classifier
Results / Conclusions
Using Morphological Operations As
Features for a Pixel
ground truth
image
I I
(3 structural elements) x (10 sizes) x (4morphological
operations) = 120 transformed images

5
H I
5
H I
5
H I -
5
V I
5
V I Q
5
V I -
5 5-
B I
5 5-
B I

5 5-
- B I
5 5-
B I Q
5
V I
5
H I Q
Using Morphological Operations As
Features for a Pixel
ground truth
image
I I

5
H I
5
H I Q
5
H I -
5
H I
5
V I
5
V I Q
5
V I -
5
V I
5 5-
B I
5 5-
B I Q
5 5-
- B I
5 5-
B I
class label
f f
13 13
f f
14 14
f f
15 15
f f
16 16
f f
17 17
f f
18 18
f f
19 19
f f
20 20
f f
21 21
f f
22 22
f f
23 23
f f
24 24
{t,c} {t,c}
={1,0} ={1,0}
Given a pixel
Ground Truth Image
of mxn mxn pixels
I I
(x (x
1,f1 1,f1
, x , x
1,f2 1,f2
, x , x
1, f3 1, f3
,... , x ,... , x
1, f119 1, f119
, x , x
1,f120 1,f120
, , tt) )
(x (x
2,f1 2,f1
, x , x
2,f2 2,f2
, x , x
2, f3 2, f3
,..., x ,..., x
2, f119 2, f119
, x , x
2,f120 2,f120
, , tt) )
(x (x
3,f1 3,f1
, x , x
3,f2 3,f2
, x , x
3, f3 3, f3
,..., x ,..., x
3, f119 3, f119
, x , x
3,f120 3,f120
, , cc) )
... ...
(x (x
N N- -1,f1 1,f1
,x ,x
N N- -1,f2 1,f2
, x , x
N N- -1, f3 1, f3
,..., x ,..., x
N N- -1, f119 1, f119
, x , x
N N- -1,f120 1,f120
, , cc) )
(x (x
N,f1 N,f1
, x , x
N,f2 N,f2
, x , x
N, f3 N, f3
,..., x ,..., x
N, f119 N, f119
, x , x
N,f120 N,f120
, , tt) )
data set
representation
of I I of size
N = mxn N = mxn
D D
Morphological Features Data Set From an
Image
5
H I
5
H I
5
H I Q
5
H I -
5
V I
5
V I
5
V I Q
5
V I -
5 5-
B I
5 5-
B I Q
5 5-
- B I
5 5-
B I
Preparation Of Data Sets to Train and
Test the Classifier
D D
11
D D
22

D D
kk
D D
k+1 k+1

D D
KK
Create
datasets
separate
vectors
I I
11
I I
22

I I
kk
I I
k+1 k+1

I I
KK
ground truth
images
target
dataset
clutter
dataset
training
dataset
test
dataset
Outline
Gray Scale Morphology
Converting Images to Datasets
Decision Tree Classifier
Results / Conclusions
Creating a Decision Tree Classifier
classify classified classified
dataset dataset
evaluate accuracy accuracy
create
decision
tree
decision tree decision tree
training
dataset
parameters parameters
test
dataset
decision tree decision tree
test
dataset
classified classified
dataset dataset
Creating a Decision Tree Classifier
1
1
1
1
1
1 1
1
1
0
0
0
0
0
0
U
11
U
22
f1 f1
f2 f2
D D
training training
f1 f1 > U
11
true
f2 f2 > U
22
class 0
true
0
0
0 0
0
0
0
0
U
33
U
44
f2 f2 > U
33
class 1
true
class 0
f1 f1 > U
44
class 1
true
class 0
threshold decision rule
max.entropy = 0.001
max. depth = 20
Outline
Gray Scale Morphology
Converting Images to Datasets
Decision Tree Classifier
Results / Conclusions
Decision Tree Classifier results
for test dataset derived from
images of resolution = 75mm
Clutter Target
Clutter
1,712,090 15,422
Target
1,419 54,503
train dataset size = 292,831 vectors
test dataset size = 1,783,434 vectors
t
r
u
e

c
l
a
s
s
assigned class
accuracy (% correct classification) = 99.046%
Decision Tree Classifier results
for images of resolution = 75mm
345 images of
clutter-only
44 images with
mostly target
accuracy # clutter images # target images
[0.91 , 0.92)
4 -
[0.92 , 0.93)
3 -
[0.93 , 0.94)
4 -
[0.94 , 0.95)
12 -
[0.95 , 0.96)
14 -
[0.96 , 0.97)
18 -
[0.97 , 0.98)
31 5
[0.98 , 0.99)
93 21
[0.99 , 1.00)
166 18
Total # images
345 44
Decision Tree Classifier results
for test dataset derived from
images of resolution = 200mm
Clutter Target
Clutter
231,495 1,702
Target
254 8,391
train dataset size = 64,127 vectors
test dataset size = 241,842 vectors
t
r
u
e

c
l
a
s
s
assigned class
accuracy (% correct classification) = 99.19%
Decision Tree Classifier for
images with resolution = 200mm
689 images with
mostly clutter
34 images with
mostly target
accuracy # clutter images # target images
< 0.90
4 -
[0.90 , 0.92)
1 -
[0.92 , 0.94)
1 -
[0.94 , 0.95)
2 1
[0.95 , 0.96)
3 -
[0.96 , 0.97)
8 1
[0.97 , 0.98)
12 1
[0.98 , 0.99)
76 10
[0.99 , 1.00]
582 21
Total # images
689 34
<end/>