Image Change Detection Algorithms

K. Lakshmi Sravya

Roopa Lahari Kalluri

14104055
Email : sravya@iitk.ac.in

14104136
Email : krlahari@iitk.ac.in

AbstractDetecting regions of change in multiple images of

the same scene taken at different times is of widespread interest
due to a large number of applications in remote sensing, medical
diagnosis and underwater sensing. This paper presents a systematic outline of the popular decision rules employed in change
detection algorithms. Image Differencing and Image Ratioing
are simplest techniques which employ an empirical threshold
to detect changes. A Significance Test on the difference image is
used to assess how well the null hypothesis H0 (corresponding
to no change) describes the observations, and this hypothesis is
correspondingly accepted or rejected. In Likelihood Ratio Tests,
we characterise the alternative hypothesis H1 (corresponding to
change) based on conditional pdfs against a suitable threshold
which associates cost and prior. In Probabilistic Mixture Models, instead of classifying the pixels as change/no change they
are softly classified into mixture components corresponding to
different generative models of change. An approach to detect
a particular hypothesis based on Minimum Description Length
(MDL) principle is also considered. For unsupervised change
detection minimization of mean square error (MMSE) approach
is proposed. Exploiting 2D histogram of difference image and further clustering based on Maximum Entropy Principle is visited.
Algorithms which result from exploiting the close relationships
between nearby pixels both in space (viz., Spatial Models) and
time (viz., Temporal Models) are also discussed.
Keywords : Probabilistic mixture of models, MDL principle,
MMSE approach, Maximum Entropy Principle, Spatial and
Temporal Models

I. I NTRODUCTION
DETECTING regions of change in images of the same
scene taken at different times is of widespread interest due to a
large number of applications in diverse disciplines. Important
applications of change detection include video surveillance,
remote sensing, medical diagnosis and treatment,civil infrastructure,underwater sensing, and driver assistance systems. Despite the diversity of applications, change detection researchers
employ many common processing steps and core algorithms.
The goal of this paper is to present a systematic survey of
these steps and algorithms.
The core problem discussed in this paper is as follows.We
are given a set of images of the same scene taken at several
different times. The goal is to identify the set of pixels that are
significantly different between the last image of the sequence
and the previous images; these pixels comprise the change
mask. The change mask may result from a combination of
underlying factors, including appearance or disappearance of
objects, motion of objects relative to the background, or shape
changes of objects. In addition, stationary objects can undergo
changes in brightness or color. A key issue is that the change

mask should not contain unimportant or nuisance forms of

change, such as those induced by camera motion, sensor noise,
illumination variation, nonuniform attenuation, or atmospheric
absorption. The notions of significantly different and unimportant vary by application, which sometimes makes it difficult
to directly compare algorithms.
Estimating the change mask is often a first step toward the
more ambitious goal of change understanding: segmenting and
classifying changes by semantic type, which usually requires
tools tailored to a particular application. The present survey
emphasizes the detection problem, which is largely application
independent. We do not discuss algorithms that are specialized to application-specific object classes, such as parts of
human bodies in surveillance imagery or buildings in overhead
imagery. Furthermore, our interest here is only in methods
that detect changes between raw images, as opposed to those
that detect changes between hand-labeled region classes. In
remote sensing, the latter approach is called post-classification
comparison or delta classification.1 Finally, we do not address
two other problems from different fields that are sometimes
called change detection: first, the estimation theory problem of
determining the point at which signal samples are drawn from
a new probability distribution and second, the video processing
problem of determining the frame at which an image sequence
switches between scenes.
Automated videosurveillance systems analyse the incoming
video stream in order to detect anomalous or dangerous situations and then trigger automatically an adequate counteraction,
such as for example sending an alarm to an operator and/or
recording the incoming images. Typically, these systems deploy one or more static cameras. Most systems currently
available in the videosurveillance market do not try to achieve
a high degree of scene understanding but restrict the analysis
of the incoming images to the detection of relevant colour
or gray-scale changes with respect to a reference image.
However, an impressive amount of ongoing research aims at
the development of advanced videosurveillance systems, that
should be able to trigger a counteraction on the basis of the
comprehension of the nature and behaviours of the objects
appearing in the scene. Typically, the latter systems include as
the very first image analysis step a change-detection algorithm
aimed at segmenting out the interesting regions from a static
background. Then, higher level processing modules, such as
tracking, classification and interpretation modules, process the
output yield by the change detection algorithm in order to
attain the required degree of scene understanding.

Most change-detection algorithms rely on the principle of

background subtraction: a background model, consisting of a
representation of the observed scene without moving objects,
is compared against the current image in order to mark as
foreground those pixels that exhibit a significant difference
with respect to the corresponding background pixels. The
main difficulty associated with change-detection is not the
background subtraction step, but instead the maintenance of
a background model that follows correctly the changes of the
reference scene. These changes can be grouped into two major
classes:
Illumination changes: sun-light intensity and inclination
vary slowly, continuously and regularly during the day, thus
varying the illumination conditions of outdoor scenes (and
in many circumstances, also of indoor scenes) and causing
slow image changes. Sudden image changes occur when the
illumination intensity varies much more rapidly, for example,
when a cloud appears in a sunny day or, in an indoor
environment, when lights are switched.
Changes due to objects: these changes occur when an object
is introduced into or removed from the reference scene. If a
foreground object stops a decision should be taken on whether
and when it is more appropriate to consider the object as part
of the reference scene and consequently include its appearance
into the background model. Similarly, if a background object
starts moving, its absence in the previously occupied image
region is detected by the subtraction step as a bogus blob
(usually known as ghost). In this case, the elimination of the
ghost is typically desirable and can be achieved by updating
the background in the image region previously occupied by
the object.
II. I MAGE D IFFERENCING AND R ATIOING
Early change detection methods were based on the signed
difference image D(x) = I2 (x) I1 (x),and such approaches
are still widespread. The most obvious algorithm is to simply
threshold the difference image. That is, the change mask is
generated according to the following decision rule:

1, if there is a significant change at pixel x of IM
B(x)=
0, otherwise.
(1)
We denote this algorithm as simple differencing. Often, the
threshold is chosen empirically.Once a difference image has
been obtained, then the problem of change detection reduces
to an image segmentation problem. The most popular solution
for image segmentation is thresholding. The output of the
thresholding process done on the difference image gives a
binary change map, in which one state indicates the changed
pixels and the other state represents the unchanged pixels.
Thresholding can be classified on the basis of parameters like:
shape of the histogram, entropy, object-attribute, clustering
mechanism etc. Some of the popular and efficient thresholding
techniques are discussed in this paper.

A. Thresholding based on shape of the histogram

In convex hull thresholding,h(g) represented by convex hull,
Hull(g) given by h(g) = Hull(g) p(g) where p(g) is the
input image distribution. In this method, after determining the
convex hull, the point having the deepest concavity is selected
as the thresholding level. If there is more than one such point,
the point having low busyness of the threshold image edges
can be taken as the thresholding level.
In peak and valley thresholding, the peak analysis of the
histogram is done. Peak analysis is done by the convolving
the image with a differencing and smoothing kernel. The
smoothing aperture of the kernel is adjusted to get a histogram
having two-lobe function. By performing the differencing
operation on the kernel, a triplet of incipient, maximum,
and terminating zero crossings of the histogram lobe S =
[(ei , mi , si ), i = 1, 2...]. And hence the thresholding level
should be between the first terminating and second initiating
zero-crossing that is:
= e1 + (1 )s2 , 0 1

(2)

In shape-modelling thresholding, a simple functional approximation to the PMF consisting of a two step function is
used. Thus we obtain the sum of squares between a bi-level
function and the histogram is minimized. So we can obtain
the thresholding solution, , by using iterative search.
B. Minimum error thresholding
Another commonly used thresholding approach is minimumerror thresholding. In minimum error thresholding, the
image is considered to be a mixture of background and
foreground pixels:
p(g) = P( ).pf (g) + [1 P( )].pb (g)

(3)

This can be performed in many ways. For example, Lloyd [1]

considered the equal variance Gaussian density function which
performs minimization of the total misclassification error iteratively. While Kittler and Illingworth [2] used a minimum error
Gaussian density-fitting functions. Cho et.al [3], proposed an
enhancement of this thresholding technique, by concluding
that the means and variances determined from the truncated
distributions result in a bias. But this bias is relevant only
when the two histogram modes are not distinguishable.
C. Thresholding based on attribute similarity
In this kind of thresholding algorithms, the thresholding
value is selected based on some attribute quality or similarity
measure between the original image and the binarized version of the image. The attributes taken can be either edge
matching,shape compactness, gray level moments,connectivity
or texture.In some other algorithms, the similarity between
the original gray-level image and binary image using fuzzy
measure [4],or resemblance of the cumulative probability
distributions [5].

D. Locally adaptive thresholding

In this class of thresholding, the thresholding value is
calculated at each pixel depending on some local statistics
like range, variance, or surface-fitting parameters of the pixel
neighbourhood. Thus the threshold (i, j) is indicated as the
function of the co-ordinates (i,j) of each pixel [6], or else
the determination of foreground or background are indicated
by a logical variable B(i,j). The parameter can be the local
variance,local contrast or center-surround scheme [7].
Miscellaneous approaches include like the one states in
[8].The algorithm is based on the idea of incorporating into the
background model a set of simple low-level features capable
of capturing effectively structural (i.e. robust with respect
to illumination variations) information. To overcome low resolution (leading to blobs) problem faced by structure based
methods a background model is adopted which is made out of
two separate parts: the first is the structure of the scene without
moving objects while the second is simply a colour image of
the reference scene. The first part of the model will be referred
to as background structure, and its components indicated as
BDxr , BDxg , BDxb , BDyr , BDyg , BDyb the second part of
the model will be referred to as background image, and its
components indicated as Br ,Bg ,Bb . The background image
provides the algorithm with the capability of detecting changes
at the highest possible resolution (i.e. the same resolution as
the original image). This enables the algorithm to yield blobs
following accurately the true objects silhouettes. Given the
described background model, at each new frame the detection
process operates at both the structure and image level; then,
the information provided by the two detection operations is
combined adequately to obtain the final output. Since structure
variations can be largely ascribed to objects, the updating
process of the background structure is focused on handling
this type of changes, with a simple mechanism allowing a
feedback from a higher-level processing module to control the
process. On the other hand, since the background image is
affected by all scene changes (illumination changes as well as
changes due to objects) it needs to be updated continuously.
Before starting the detection process, our algorithm activates
a simple bootstrap process aimed at estimating the initial data
to be included into the background model.
Initialization of the Background Model: This step consists
mainly in estimating the initial background structure and lasts
a user-selectable time period. When all the samples have been
collected, the background structure is initialised by taking for
each element of the six background structure components the
value observed most frequently. Once the background structure
is initialised, the background image is initialised by simply
copying the current colour image into Br , Bg ,Bb .
Structure-Level Detection:We compare the structure of the
current frame,I, with the background structure by building up
two delta-structure images associated respectively with the x
and y directions. Then, choosing a suitable threshold value
, if Sx > a foreground object occupies the image region
associated with structure element (x,y) or (x+1,y). Same is the

case for Sy .
Image-Level Detection In this case the detection step is
simpler and consists in computing the difference between I
and the background image. Hence, calling Ir ,Ig ,Ib the colour
channels of the current frame and p as a new threshold value,
we define a binary image Imask containing the image-level
detection.
Combination of the Detection Results: Following structure
and image level detection, the information contained in Smask
is used to decide whether or not each of the blobs detected in
Imask is valid. The result of the combination step, representing
the final output of the overall change-detection system, is a
binary image, Mask, that contains only the blobs associated
with objects changing the image structure.
Updating of the Background Image: Indicating the background image before and after the updating as Bt and B( t + 1)
update rule is implemented for red,blue and green channels.
Image ratioing is another related technique that uses the
ratio, instead of the difference, between the pixel intensities
of the two images. Different thresholding techniques similar
to above can be used for Image Ratioing also.
III. S IGNIFICANCE AND H YPOTHESIS T ESTS
The decision rule in many change detection algorithms is
cast as a statistical hypothesis test. The decision as to whether
or not a change has occurred at a given pixel x corresponds to
choosing one of two competing hypotheses: the null hypothesis
H0 or the alternative hypothesis H1 , corresponding to nochange and change decisions, respectively.
The image pair (I1 (x),I1 (y)) is viewed as a random vector.
Knowledge of the conditional joint probability density functions (pdfs) p(I1 (x), I2 (x)|H0 , and p(I1 (x), I2 (x)|H1 allows
us to choose the hypothesis that best describes the intensity
change at x using the classical framework of hypothesis
testing. Since interesting changes are often associated with
localized groups of pixels, it is common for the change
decision at a given pixel x to be based on a small block of
pixels in the neighborhood of x in each of the two images (such
approaches are also called geo-pixel methods). Alternately,
decisions can be made independently at each pixel and then
processed to enforce smooth regions in the change mask.
We denote a block of pixels centered at x by x . The pixel
values in the block are denoted

I(x)
= {I(y) | y x }

(4)

Note that I(x)

is an ordered set to ensure that corresponding
pixels in the image pair are matched correctly. We assume
that the block contains N pixels. There are two methods for
dealing with blocks, as illustrated in Fig. 1
One option is to apply the decision reached at x to all the
pixels in the block x in which case the blocks do not overlap,
the change mask is coarse, and block artifacts are likely. The
other option is to apply the decision only to pixel x, in which
case the blocks can overlap and there are fewer artifacts;
however, this option is computationally more expensive.

A. Significance Tests
Characterizing the null hypothesis H0 is usually straightforward, since in the absence of any change, the difference
between image intensities can be assumed to be due to noise
alone. A significance test on the difference image can be
performed to assess how well the null hypothesis describes the
observations, and this hypothesis is correspondingly accepted
or rejected. The test is carried out as below
H0

S(x) = p(D(x)) | H0 )

(5)

H1

The threshold can be computed to produce a desired false

alarm rate
Aach et al. [9],[10] modeled the observation D(x) under the
null hypothesis as a Gaussian random variable with zero mean
and variance o2 .The unknown parameter o2 can be estimated
offline from the imaging system or recursively from unchanged
regions in an image sequence[11]. In the Gaussian case, this
results in the conditional pdf


D2 (x)
1
exp

(6)
p(D(x) | H0 ) = p
202
202
They also considered a Laplacian noise model that is similar.
The extension to a block-based formulation is straightforward. Though there are obvious statistical dependencies within
a block, the observations for each pixel in a block are typically
assumed to be independent and identically distributed (iid). For
example, the block-based version of the significance test (5)
uses the test statistic
P 2

D (y)

1
yx

(7)
p(D(x) | H0 ) =
exp
N

202
(202 ) 2

which can be rewritten as:

p(D(x) | H0 ) =

1
N

(202 ) 2

exp

G(x)
2


(8)

Here, G(x) which has a 2 pdf with N degrees of freedom.

Tables for the 2 distribution can be used to compute the
decision threshold for a desired false alarm rate. A similar
computation can be performed when each observation is
modeled as an independent Laplacian random variable.
B. Likelihood Ratio Test
Characterizing the alternative (change) hypothesis H0 is
more challenging, since the observations consist of change
components that are not known a priori or cannot easily be
described by parametric distributions. When both conditional
pdfs are known, a likelihood ratio can be formed as
L(x) =

p(D(x) | H1 )
p(D(x) | H0 )

(9)

This ratio is compared to a threshold defined as

=

P (H0 )(C10 C00 )

P (H1 )(C01 C11 )

(10)

where P(Hi ) is the a priori probability of hypothesis Hi and

Cij is the cost associated with making a decision in favor of
hypothesis Hi when Hj is true. In particular, C10 is the cost
associated with false alarms and C01 is the cost associated
with misses. If the likelihood ratio at x exceeds , a decision
is made in favor of hypothesis H1 ; otherwise, a decision is
made in favor of hypothesis H0 . This procedure yields the
minimum Bayes risk by choosing the hypothesis that has the
maximum a posteriori probability of having occurred given
the observations (I1 (x),I2 (x)).
Both hypotheses can be modeled by the observations com
prising D(x)
under Hi as IID zero-mean Gaussian random
variables with variance i2 . In this case, the block-based
likelihood ratio is given by



X

1
0N
1
p(D(x)
| H1 )
2
= N exp
D2 (y)

212
20
1
p(D(x)
| H0 )
yx

(11)
The parameters 02 and 12 were estimated from unchanged
[i.e., very small D(x)] and changed [i.e., very large D(x)]
regions of the difference image, respectively. As before, a
Laplacian noise model is also considered.[12] describes hypothesis tests on the difference and ratio images from SAR
data assuming the true and observed intensities were related
by a gamma distribution.
Bruzzone and Prieto [13] noted that while the variances
estimated as above may serve as good initial guesses, using
them in a decision rule may result in a false alarm rate
different from the desired value. They proposed an automatic
change detection technique that estimates the parameters of
the mixture distribution p(D) consisting of all pixels in the
difference image. The mixture distribution p(D) can be written
as
p(D(x)) = p(D(x) | H0 )P (H0 ) + p(D(x) | H1 )P (H1 )
(12)
The means and variances of the class conditional distributions p(D(x) | Hi ) are estimated using an expectationmaximization (EM) algorithm [14] initialized in a similar way
to the algorithm of Aach et al. In [15], Bruzzone and Prieto
proposed a more general algorithm where the difference image
is initially modeled as a mixture of two nonparametric distributions obtained by the reduced Parzen estimate procedure
[16]. These nonparametric estimates are iteratively improved
using an EM algorithm.
C. Probability Mixture Model
Instead of classifying the pixels as change or no change they
are softly classified into mixture components corresponding to
different generative models of change. The change might be
due to motion, pictoral changes, illumination changes etc
In particular, we assume that the image I(x; t) at location
x at time t is generated, or explained, by one of n causes
ICi , i = 1, 2, ..., n. The causes ICi (x; t; ai ), can be thought of
as overlapping layers and are simply images that are generated
given a vector of parameters ai .

Given n causes, the probability of observing the image

I(x; t) is a mixture model given by
p(I(x, t)|aj , j nj=1 ) =

n
X

wi (x)pi (I(x, t)|ai , i )

A. Cross Correlation Based Detection

The wi (x) are ownership probabilities. They specify the

relative probabilities that the different models account for the
appearance change at pixel x . The causes provide parametric
models over the entire image region, while the weights represent the relative likelihoods pi (I(x, t)|ai , i ) of each cause at
every pixel. Finally, the i s are scale parameters that are used
to control a from of deterministic annealing in the estimation
of the parameters.
A maximum likelihood estimate of the global model
parameters,a1 , ..., an and the ownership probabilities,
w1 (x), ..., wn (x) that yields a soft assignment of pixels to
models has to estimated.This is done using Expectation
Maximization algorithm.
The EM algorithm alternates between solving for the
weights, wi (x, i ), given an estimate of the parameters
a1 , ...., an (the Expectation step), and then updating the parameters, a1 , ...., an , with the weights held fixed (the Maximization step).
If the parameters of the different models are known, then
we can compute the probability that pixel x belongs to cause
i.
i ,

pi (I(x; t|ai )
j=1 pj (I(x, t)|aj , j )

Pn

(14)

The maximum likelihood estimate of the model parameters,

given the ownership weights, satisfies
n
XX
x

i=1

wi (x, i )

log pi (I(x, t)|ai , i )) = 0

ai

(15)

(ICi , i ) =

4
2 + IC2 i
I Ci i

The cross-correlation coefficient is a measure of the similarity between two variables. The change detection problem can
be understood as the process to partition two input images
into two distinct regions, namely changed and unchanged,
according to the binary change detection mask. Each region
in the pair of the images of the corresponding position is
considered as two sets of variables, whose cross-correlation
coefficient is calculated in order to provide an optimal partition
of the changed and unchanged regions. In the optimal partition,
it is obvious that the cross-correlation coefficient of the set of
the unchanged variables should be the maximum, while the
absolute-value of that of the changed variables should be the
minimum, because the corresponding unchanged regions are
similar while the changed regions are quite different. Genetic
Algorithm is used to obtain the optimal non-dominated solution as the change detection using cross-correlation coefficient
is a multi-objective optimization problem. Let us consider two
registered images X1 = {x1 (i, j)|1 i M, 1 j N }
and X2 = {x2 (i, j)|1 i M, 1 j N } of size MN
pixels acquired on the same scene at different time. The main
objective of change detection is to generate a binary image
CM = {cm(i, j)|1 i M, 1 j N } which is called
a change mask that identifies changed and unchanged regions
according to the following generic rule:

1 pixel(i, j) is changed
(18)
cm(i, j) =
0 pixel(i, j) is unchanged
The cross-correlation of the changed and unchanged regions
can be defied as follow:
P

where it can be shown that the derivative of the log likelihood

is given by
logpi (I(x, t)|ai , i
ICi (x, t; ai )
= (ICi , i )
ai
ai

IV. M ISCELLANEOUS

(13)

i=1

wi (x, i ) =

discussed earlier. The MDL principle selects the hypothesis

that more concisely describes (i.e., using a smaller number of
bits) the observed pair of images.

(16)
(17)

and ICi = I(x, t) ICi (x, t; ai )for the ith model.

D. Minimum Description Length
A change detection algorithm based on a concept called
self-consistency between viewpoints of a scene is cosidered
in Leclerc et al. [17]. The algorithm involves both raw images
and three-dimensional (3-D) terrain models, the main goal
of the work is to provide a framework for measuring the
performance of stereo algorithms. However, a notable feature
of this approach is its use of the minimum description length
(MDL) model selection principle [18] to classify unchanged
and changed regions, as opposed to the Bayesian approaches

cm(i,j)=0 (X1 (i, j) 01 )(X2 (i, j) 02 )

u = q P
P
( cm(i,j)=0 (X1 (i, j) 01 )2 )( cm(i,j)=0 (X2 (i, j) 02 )2 )
(19)

cm(i,j)=1 (X1 (i, j) 11 )(X2 (i, j) 12 )

c = q P
P
( cm(i,j)=1 (X1 (i, j) 11 )2 )( cm(i,j)=1 (X2 (i, j) 12 )2 )
(20)

For r = 0 or 1 and s = 0 or 1
X
1
Nr =
1, rs =
Nr
cm(i,j)=r

Xs (i, j)

(21)

cm(i,j)=r

For a given change mask, there exists the cross-correlation

coefficients corresponding to the set of the unchanged and
changed variables. The change detection could be described
as a multi-objective optimization problem to maximize u and
minimize the absolute-value of c . Usually, multiple-objective
optimization problem is different from single-objective optimization problem. In the single-objective optimization problem, one attempts to obtain the best solution which is better
than all other alternatives. In the case of multiple-objectives

optimization problem, there may be not a solution that is

best with respect to all objectives. Therefore, there usually
exist a set of solutions for the multiple -objective optimization
problem which cannot simply be compared with each other
and are called non-dominated solutions or Pareto optimal
solutions, no improvement in any objective function is possible
without sacrificing at least one of the other objective functions.
For a multiple-objective optimization problem which is
represented as follow:
max{Z1 = f1 (x), f1 (x), Z2 = f2 (x), . . . , Zq = fq (x)
s.t.gi (x) 0

i = 1, . . . , m

(22)

Z is used to denote the feasible region in the criterion space

and S is used to denote the feasible region in the decision space
as follow:
Z = {z = Rq |Z1 = f1 (x), . . . , Zq = fq (x), x S}

(23)

For a given point z Z , it is non-dominated if and only

if there does not exist another point zZ , such that for the
maximization case
zk > zk0 f or some k {1, 2, . . . , q}

(24)

and also zl zl0 f or all l 6= k The change mask can

be considered as a binary encoding for the genetic representation. Let CMkg , k = 1, . . . , K be the k th individual at
the g th generation. A Pareto ranking-based fitness assignment
method which gives an equal reproductive potential for all
non-dominated individuals is as follows: Rank 1 is given
to the non-dominated solutions and they are removed from
contention. Then rank 2 is assigned to the next set of non dominated individuals. This process continues until the entire
population is ranked. If we let Pk denote the fitness of the k th
individual in the ranked population, the linear ranking method
proposed by [19] takes the form
q
(25)
Pk = q (k 1)
K 1
where parameter q is the fitness of the best individual and K
is the number of the last rank.
The genetic algorithm process for the change detection can
be summarized as follows:
(1)Generate an initial population of solutions CMk0 . Set
t=0
(2) Select 2 solutions from the population as the parent
strings randomly.
(3)Generate child strings from the parent strings using the
fusion crossoveroperator.
(4)Select solutions by using VEGA in parent strings
and child strings as the new population.
(5)Set t=t+1
(6)Repeat steps 2 to 5 until t reaches the pre-set maximal
generation.
(7)Find the set of non-dominated solutions in the
population.

B. Minimum Mean Square Error Based Detection

The difference image computed by the absolute-valued log
ratio of the intensity values of two input images is partitioned
into two distinct regions according to the change mask. For
each region, the mean square error between its difference
image values and the average of its difference image values
is calculated. In singleband images, the accurate solution of
the change mask with minimum MSE can be obtained in an
acceptable time. In multispectral images, it is considered as a
multi-objective optimizations problem. GA (same as the one
used in Section A) is used to obtain the optimal compromised
solution. Change error matrix and Kappa coefficient are used
to assess the effectiveness of the change detection techniques.
In [20], the cost value which is the weighted sum of the
MSE of the changed and unchanged regions is computed.
The lower the MSE, the better the partition is. Cost function
approaches to zero when CM part itions the difference image
Xd into changed and unchanged regions adequately, and vice
versa.
In fact, assume that all distinct pixels are independent, the
value of the changed pixels in the difference image is larger
than the non-changed ones. Under such circumstances, the
change mask that allows the lowest weighted sum of the
MSE isnt consistent with this principle. If there is a group of
points, most of which have close values of pixel, the change
mask which allows the lowest weighted sum of the MSE is
to categorize those points with higher value and lower value
into one class. Hence, a novel cost function is proposed as the
MSE:
X
X
V =
(Xd (i, j)1 )2 +
(Xd (i, j)0 )2 (26)
cm(i,j)r

cm(i,j)0

C. Clustering
A change detection algorithm based on clustering characteristic of 2-D histogram formed by pixel gray levels and the local
average gray levels is proposed in [21] for precise detection,
stronger anti-noise capability and faster computation than
traditional 2-D maximum entropy algorithm. In this method
first, the 2-D histogram is segmented into two initial clusters
representing change region and unchanged region respectively
by using classical segmentation method. Then, the traditional
2-D maximum entropy principle is improved properly to adjust
the initial clusters. Finally, changes are detected according to
the two relative more accurate clusters that have been adjusted.
Let us assume that the two images, X1 and X2 , have been
co-registered. Let X represent the values of the pixels in
the difference image obtained by applying UID technique to
X1 and X2 . In order to obtain a 2-D histogram of the difference
image, we define the local average of a pixel, f (x, y), as the
average intensity of its neighbors denoted by g(x, y). A 2-D
histogram with f (x, y) and g(x, y) as the 2 axes and their
frequency of occurance as their probability is constructed.
The decision threshold (s, t) is searched initially by using 2D maximum entropy principle which is proposed by Abutaleb

in [22]

D. Predictive Models
HA =

s X
t
X

pij log pij

i=0 j=0

HB =

L1
X

L1
X

pij log pij

(27)

i=s+1 j=t+1

(s , t ) = Arg

M ax

(HA + HB )

0<s<L1,0<t<L1

(28)

Initially s is assumed to be the equal to t. The decision

threshold is acquired by
(t ) = Arg M ax (HA + HB )
0<t<L1

(29)

Then, the 2-D histogram is segmented into two initial clusters

1 and 2 representing unchanged region 1 and change
region 2 respectively The discriminant is defined as:

1 f (x, y) + g(x, y) < 2t
(30)
f (x, y) =
2 others
The improper pixels in 2-D histogram, which are segmented
initially, can be adjusted properly based on the 2-D histogram
characteristics that :
1) The pair (f (x, y), g(x, y)) appears mostly around the
main diagonal line. In addition, the pixels within the cluster
are close to the main diagonal line and edge points or noises
are far from that.
2) The clustering characteristic is clearer in the 2-D histogram than in 1-D Histogram. The pixels having the same
intensity but different spatial features can be distinguished in
the second dimension (local average gray level) of the 2-D
histogram.
3) Usually changed areas are smaller than unchanged areas
in the difference image, so the 2-D histogram presents a big
cluster and a small cluster under many conditions.
After the 2-D histogram has been segmented into two
regions, the 2-D maximum entropy principle is modified as
follows:
H(n+1),(d+1) = H1,(n+1),(d+1) + H2,(n+1),(d+1)

(31)

L1
P L1
P

pij log pij

H1 ,(n+1),(d+1) =

pij 1,(n+1),(d+1)

H2 ,(n+1),(d+1) =

pij 2,(n+1),(d+1)

i=0 j=0
L1
P L1
P

pij log pij

i=0 j=0

(32)

where Hi represents entropy in the region i

After the two more accurate regions 1 and 2 in the 2D histogram are acquired, the change region and unchanged
region can be detected by :

1 1 ([f (x, y)], [g(x, y)]) > 0
f (x, y) =
(33)
2 2 ([f (x, y)], [g(x, y)]) > 0

Some change detection algorithms exploit the close relationships between nearby pixels both in space and time (when
an image sequence is available).
1) Spatial Model: A classical approach to change detection
is to fit the intensity values of each block to a polynomial
function of the pixel coordinates x . In two dimensions, this
corresponds to
Ik (x, y) =

p X
pi
X

k i j
ij
xy

(34)

i=0 j=0

where is the order of the polynomial model. Hsu et al. [23]

discussed generalized likelihood ratio tests using a constant,
linear, or quadratic model for image blocks. The null hypothesis in the test is that corresponding blocks in the two images
0
are best fit by the same polynomial coefficients ij
, whereas
the alternative hypothesis is that the corresponding blocks
1
2
are best fit by different polynomial coefficients (ij
, ij
) .
k
In each case, the various model parameters ij are obtained
by a least-squares fit to the intensity values in one or both
corresponding image blocks. The likelihood ratio is obtained
based on derivations by Yakimovsky [24] and is expressed as
F (x) =

02N
1N 2N

(35)

where N is the number of pixels in the block, 12 is the

variance of the residuals from the polynomial fit to the block in
I1 , 22 is the variance of the residuals from the polynomial fit
to the block in I2 , and 02 is the variance of the residuals from
the polynomial fit to both blocks simultaneously. The threshold
in the generalized likelihood ratio test can be obtained using
the t-test (for a constant model) or the F-test (for linear
and quadratic models). Hsu et al. used these three models
to detect changes between a pair of surveillance images and
concluded that the quadratic model outperformed the other
two models, yielding similar change detection results but with
higher confidence.
Skifstad and Jain [25] suggested an extension of Hsus
intensity modeling technique to make it illumination-invariant.
The authors suggested a test statistic that involved spatial
partial derivatives of Hsus quadratic model, given by
!
X I1
I2
I1
I2
T (x) =
(y)
(y) +
(y)
(y)
x
x
y
y
yx
(36)
Here, the intensity values Ij (x) are modeled as quadratic
functions of the pixel coordinates
2) Temporal Model: This model exploits the temporal consistency of pixels in the same location at different times.The
pixel intensities over time were modelled as an autoregressive
(AR) process.In [26] it assumed each pixel was identically and
independently distributed according to the same (time varying)
Gaussian distribution related to the past by the same (time
varying) AR(1) coefficient. Under these assumptions, they
derived maximum likelihood estimates of the mean, variance,

and correlation coefficient at each point in time, and used these

in likelihood ratio tests where the null hypothesis is that the
image intensities are dependent, and the alternate hypothesis
is that the image intensities are independent .
The paper [27] described an algorithm called Wallflower
that used a Wiener filter to predict a pixels current value
from a linear combination of its previous values. Pixels whose
prediction error is several times worse than the expected error
are classified as changed pixels. The predictive coefficients are
adpatively updated at each frame. This can also be thought
of as a background estimation algorithm (see Section VIII).
The Wallflower algorithm also tries to correctly classify the
interiors of homogeneously colored, moving objects by determining the histogram of connected components of change
pixels and adding pixels to the change mask based on distance
and color similarity. Morisette and Khorrams work [28], can
be viewed as a supervised method to determine an optimal
linear predictor.
V. C ONCLUSION
Change detection algorithms pertaining to basic logics like
image differencing and ratioing are viewed in brief. Different
criterion for deciding threshold have been stated. A larger class
of tests viz., significance and hypothesis tests have been discussed. These include the popular likelihood ratio test, probabilistic mixture models and MDL approach. Miscellaneous
algorithms include cross correlation based detection, MMSE
based and histogram clustering based detection techniques.
Predictive modeling of the given image sequence comprises
of spatial and temporal modeling in spatial and time domain
respectively.Despite the substantial amount of work in the
field, change detection is still an active and interesting area
of research. Future algorithm developments are expected to be
fueled by increasingly more integrated approaches combining
elaborate models of change, implicit pre- and post-processing,
robust statistics, and global optimization methods. Comprehensive and continued growth of societal applications of change
detection and interpretation in key areas such as biotechnology
and geospatial intelligence form the future prospects.
R EFERENCES
[1] [D. E. Lloyd, Automatic target classification using moment invariant of
image shapes, Technical Report, RAE IDN AW126, Farnborough, UK
Dec. 1985.]
[2] [J. Kittler and J. Illingworth, Minimum error thresholding, Pattern
Recogn. 19.41-47 1986]
[3] S. Cho, R. Haralick, and S. Yi, Improvement of Kittler and Illingworthss
minimum error thresholding, Pattern Recogn. 22, 609617 1989
[4] [S. Negahdaripour, Revised definition of optical flow: Integration of
radiometric and geometric cues for dynamic scene analysis, IEEE Trans.
Pattern Anal. Mach. Intell., vol. 20, no. 9, pp. 961979, Sep. 1998.]
[5] G. D. Hager and P. N. Belhumeur, Efficient region tracking with parametric models of geometry and illumination, IEEE Trans. Pattern Anal.
Mach. Intell., vol. 20, no. 10, pp. 10251039, Oct. 1998.
[6] Y. Nakagawa and A. Rosenfeld, Some experiments on variable thresholding, Pattern Recogn. 11(3), 191204 1979
[7] Y. Yasuda, M. Dubois, and T. S. Huang, Data compression for check
processing machines, Proc. IEEE 68, 874885 1980.
[8] L. di Stefano, S. Mattoccia, and M. Mola, Achange-detection algorithm
based on structure and color, in Proc. IEEE Conf. Advanced Video and
Signal-Based Surveillance, 2003, pp. 252259.

[9] T. Aach, A. Kaup, Statistical model-based change detection in moving

video, Signal Process., vol. 31, pp.165-180, Mar., 1993
[10] T. Aach, A. Kaup, Bayesian algorithms for adaptive change detection
in image sequences using Markov random fields, Signal Process.: Image
Commun., vol. 7, pp.147-160, Aug., 1995
[11] R. Thoma, M. Bierling, Motion compensating interpolation considering
covered and uncovered background, Signal Process.: Image Commun.,
vol. 1, no.2, pp.191-212, Oct.
[12] E. Rignot, J. van Zyl, Change detection techniques for ERS-1 SAR
data, IEEE Trans. Geosci. Remote Sens., vol. 31, no.4, pp.896-906, Jul.,
1993
[13] L. Bruzzone and D. F. Prieto, Automatic analysis of the difference image
for unsupervised change detection, IEEE Trans. Geosci. Remote Sens.,
vol. 38, no. 3, pp. 11711182, May 2000.
[14] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood
from imcomplete data via the EM algorithm, J. Roy. Stat. Soc., vol. 39,
no. 1, pp. 138, 1977.
[15] L. Bruzzone and D. F. Prieto, An adaptive semiparametric and contextbased approach to unsupervised change detection in multitemporal
remote-sensing images, IEEE Trans. Image Processing, vol. 11, no. 4,
pp. 452466, Apr. 2002.
[16] K. Fukunaga and R. Hayes, The reduced Parzen classifier, IEEE Trans.
Pattern Anal. Mach. Intell., vol. 11, no. 4, pp. 423425, Apr. 1989.
[17] Y. G. Leclerc, Q.-T. Luong, and P. V. Fua, Self-consistency and MDL:A
paradigm for evaluating point-correspondence algorithms, and its application to detecting changes in surface elevation, Int. J. Comput. Vis., vol.
51, no. 1, pp. 6383, Jan. 2003.
[18] J. Rissanen, Minimum-description-length principle, in Encyclopedia of
Statistical Sciences, 5th ed. New York: Wiley, 1987, pp. 523527.
[19] Baker, J., Adaptive selection methods for genetic algorithms in Grefenstette [267], pp.100-111
[20] Turgay CelikChange Detect ion in Satellite Images Usinga Genet ic
Algorithm Approach IEEE GEOSCIENCE AND REMOTE SENSING
LETTERS, VOL. 7, NO. 2, APRIL 2010
[21] Junping Zhang ; Wenbang Sun ; Wenyang Tang Geoscience and
Remote Sensing Symposium, 2006. IGARSS 2006. IEEE International
Conference on DOI: 10.1109/IGARSS.2006.197 Publication Year: 2006
, Page(s): 767 - 770
[22] Abutaleb, A Automatic thresholding of gray-level picture using twodimensional entropies. Computer Vision Graphicsand Imaging Processing,
1989,47:22 32.
[23] Y. Z. Hsu, H.-H. Nagel, and G. Reckers, New likelihood test methods
for change detection in image sequences, Comput. Vis., Graph. Image
Process., vol. 26, pp. 73106, 1984.
[24] Y. Yakimovsky, Boundary and object detection in real world images, J.
ACM, vol. 23, no. 4, pp. 599618, Oct. 1976.
[25] K. Skifstad and R. Jain, Illumination independent change detection for
real world image sequences, Comput. Vis. Graph. Image Process., vol.
46, no. 3, pp. 387399, 1989.
[26] Z. Jain and Y. Chau, Optimum multisensor data fusion for image change
detection, IEEE Trans. Syst., Man, Cybern., vol. 25, no. 9, pp. 13401347,
Sep. 1995.
[27] K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, Wallflower: Principles and practice of background maintenance, in Proc. ICCV, 1999, pp.
255261.
[28] J. Morisette and S. Khorram, An introduction to using generalized linear
models to enhance satellite-based change detection, in Proc. IGARSS, vol.
4, 1997, pp. 17691771.
[29] Lei Song,Shaoning Pang,Gang Chen,Sarrafzadeh H.,Tao Ban,Inoue,: An
incremental learning approach to continuous image change detection,747752,2013
[30] Wenbang Sun ; Hexin Chen ; Haiyan Tang ; Guang Yu Image and Signal
Processing (CISP), 2010 3rd International Congress:Unsupervised image
change detection means based on improved 2-D entropy;2282-2286,2010.
[31] Lu Xu ; Shaoqing Zhang ; Zongyi He ; Yan Guo: The comparative
study of three methods of remote sensing image change detection
Geoinformatics, 2009 17th International Conference,1-4,2009
[32] Yunchen Pu,Wei Wang,Qiongcheng Xu:Image Change Detection Based
on the Minimum Mean Square Error Computational Sciences and Optimization (CSO), Fifth International Joint Conference,2012
[33] Wenbang Sun,Hexin Chen,Li Xue, Qinling Liu: Research of unsupervised image change detection algorithm based on clustering characteristic
of 2-D histogram. CMCE International Conference. pp:341-344,2012

[34] Xi Ye,Hong Zhang,Chao Wang,Bo Zhang,Fan Wu,Yixian Tang: SAR

image change detection based on object-based method Geoscience and
Remote Sensing Symposium (IGARSS),IEEE International. pp:20832086,2013