Information Fusion 22 (2015) 105–118

Contents lists available at ScienceDirect

Information Fusion
journal homepage: www.elsevier.com/locate/inffus

Quadtree-based multi-focus image fusion using a weighted

focus-measure
Xiangzhi Bai a,b,⇑, Yu Zhang a, Fugen Zhou a, Bindang Xue a
a
Image Processing Center, Beijing University of Aeronautics and Astronautics, Beijing 100191, China
b
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China

a r t i c l e i n f o a b s t r a c t

Article history: The purpose of multi-focus image fusion is integrating the partially focused images into one single image
Received 12 November 2013 which is focused everywhere. To achieve this purpose, we propose a new quadtree-based algorithm for
Received in revised form 26 February 2014 multi-focus image fusion. In this work, an effective quadtree decomposition strategy is presented.
Accepted 21 May 2014
According to the proposed decomposition strategy, the source images are decomposed into blocks with
Available online 6 June 2014
optimal sizes in a quadtree structure. And in this tree structure, the focused regions are detected by using
a new weighted focus-measure, named as the sum of the weighted modified Laplacian. Finally, the
Keywords:
focused regions could be well extracted from the source images and reconstructed to produce one fully
Multi-focus image fusion
Quadtree decomposition strategy
focused image. Moreover, the new weighted focus-measure performs better than the commonly used
Quadtree structure focus-measures on the detection of the focused regions, since it is sensitive to the homogeneous regions.
Weighted focus-measure The proposed algorithm is simple yet effective, because of the quadtree decomposition strategy and the
Sum of the weighted modified Laplacian new weighted focus-measure. To do the comparison, the proposed algorithm is compared with several
existing fusion algorithms, in both the qualitative and quantitative ways. The experimental results show
that the proposed algorithm yields good results.
Ó 2014 Elsevier B.V. All rights reserved.

1. Introduction are processed according to some criteria and transformed back to

It is known that the lens of a camera can only focus at one dis-
tance or on a plane at a time [1,2]. Thereby, only the objects falling
in the depth of field (DOF) of the lens appear to be sharp, while the
objects out of the DOF are blurred [1–3]. As a result, the multiple
objects of interest may not be all focused in one single image. To
produce one image in which all objects are in focus, image fusion
is a kind of method. As is known, the image fusion techniques have
been widely used in many applications, such as medical image
fusion [4], infrared and visual image fusion [5] and remote sensing
image fusion [6]. The term fusion for the multi-focus images is
integrating the multiple images of the same scene to form one fully
focused image, which is suitable for visual perception or computer
processing [1–3,7,8].
In the past decades, different genres of multi-focus image fusion
algorithms have been proposed. Most of these algorithms are done
either in the spatial domain or transform domain.
The transform domain algorithms usually extract coefficients
with respect to some localized bases [2]. Then, these coefficients
⇑ Corresponding author at: Image Processing Center, Beijing University of
Aeronautics and Astronautics, Beijing 100191, China.
E-mail address: jackybxz@buaa.edu.cn (X. Bai).
produce the fusion image. However, the transform domain algo-
rithms may encounter some conditions when dealing with the high
dimensional signals like images [2]. For example, the transform
domain algorithms usually generate the fusion image globally,
thus their fusion image would be easily resulted in deterioration
due to misregistration [7,9].
The spatial domain algorithms are based on a local operation.
They can avoid the global problems and preserve more original
information from the source images than the transform domain
algorithms. In general, spatial domain algorithms can be classified
as: pixel-based, block-based, and region-based [3]. The pixel-
based fusion algorithms just consider the single pixels or use
the information in the local neighborhoods, which could lead to
some undesirable side effects [2], such as reduced contrast, and
blocking artifacts. To resolve this problem, some block-based algo-
rithms [8–10] have been proposed. In the block-based algorithms,
the source images are firstly decomposed into blocks with equal
size. Then the focused blocks can be detected by measuring the
focus on the corresponding blocks. However, a large block may
contain both the focused and defocused regions and the focus of
a small block could not be well measured [3]. Therefore, the
performance of these algorithms is restricted by the selected
block-size.

http://dx.doi.org/10.1016/j.inffus.2014.05.003
1566-2535/Ó 2014 Elsevier B.V. All rights reserved.
106 X. Bai et al. / Information Fusion 22 (2015) 105–118
In recent years, several more reasonable fusion algorithms have
been proposed. In [11], Aslantas et al. gave a method to choose the
optimal block-size. Since the optimization process is done recur-
sively, this algorithm is time-consuming. In [1,12,13], several
region-based algorithms are proposed for multi-focus image fusion.
At first, the source images are segmented into regions, and then the
focused regions could be found out by comparing the focus-mea-
sures of the corresponding regions. However, the region-based
algorithms are time-consuming and their performance greatly
relies on its previous segmentation results. In [3], De et al. proposed
a quadtree-based algorithm for multi-focus image fusion. In this
algorithm, the source images are firstly decomposed into blocks
with different sizes in a quadtree structure. In this tree structure,
the focused blocks are detected by measuring the focus on the cor-
responding blocks. But in some cases, their quadtree decomposition
method might be not effective, so that the optimal subdivision of
the source images could not be achieved.
To improve the quality of the fusion image, a new quadtree-
based multi-focus image fusion algorithm is proposed. We make
use of an effective quadtree decomposition strategy to decompose
the source images into blocks with optimal sizes in a quadtree
structure. In this structure the focused blocks are detected by using
a new weighted focus-measure, named as the sum of the weighted
modified Laplacian. Naturally, the focused blocks are merged to
form the focused regions [3]. Then, the focused regions are recon-
structed to verify the consistency by using two sequential filters.
Moreover, the proposed algorithm is compared with some existing
algorithms qualitatively and quantitatively. The main contribution
of this paper can be concluded as: (1) proposing an effective quad-
tree decomposition strategy and (2) presenting a new weighted
focus-measure.
2. Proposed algorithm
The goal of this paper is to generate one fully focused image
from the multi-focus images. To achieve this goal, a new quad-
tree-based algorithm for multi-focus image fusion is proposed in
this section. The general process of the proposed algorithm and a
demonstration example are illustrated in Fig. 1. The process of
the proposed algorithm is: (1) decompose the source images into
blocks with optimal sizes, according to an effective quadtree
decomposition strategy; (2) find out the focused blocks by measur-
ing the focus of each block-pair in the tree structure; (3) recon-
struct the focused regions by using two sequential filters.
This section is organized as follows. Firstly, the method of
detecting the focused regions in a quadtree structure is introduced.
Secondly, to detect the focused blocks accurately, a new focus-
measure, named as the weighted modified of Laplacian, is pro-
posed. Thirdly, the focused regions are reconstructed to produce
the fusion image. Finally, the comparison between the proposed
algorithm and the previous work is discussed.
2.1. Detecting the focused regions
Detecting the focused regions from the source images is the
most crucial part of the proposed algorithm. Here, we make use
of the quadtree structure to achieve this purpose. First, the source
images are decomposed into blocks with different sizes in a quad-
tree structure, according to a quadtree decomposition method.
Then in the tree structure, the focused blocks can be detected by
measuring the focus of the corresponding blocks. To obtain an opti-
mal subdivision of the source images, the key is to find an appro-
priate quadtree decomposition method.
In [3], De et al. used a quadtree-based method to detect the
focused regions. To decompose the source images, they defined
the normalized difference in focus-measures (NDFM) and a con-
stant multiplier (M), as Eqs. (1) and (2) [3]. At the beginning, each
source image is subdivided into four blocks. In this paper, the cor-
responding blocks are called as a block-pair (two blocks) or a
block-set (more than two blocks). Then, the NDFM for each
block-pair is compared with the corresponding threshold T, which
equals to the product of the NDFM calculated from its parent block-
pair and M. If NDFM > T, the current block-pair would not be sub-
divided again. Otherwise, each block in the current pair should
be further subdivided into four smaller blocks. However, in some
cases, the source images could not be subdivided properly, such
as the example shown in Fig. 2. In Fig. 2, it is assumed that the
source images are centrosymmetric, and the gradients of the
source images are distributed uniformly. Since the source images
are centrosymmetric, the NDFMs for the corresponding blocks are
equal. Then, the constant value M and the threshold T can be calcu-
lated and their values are both zero. Consequently, all the blocks
will not be further subdivided, and the optimal subdivision of the
source images could not be achieved. Furthermore, De’s quadtree
decomposition method is based on the assumption that the energy
distribution of an image is related to the energy distribution of its
subregions. However, the energy distribution of an image is
random, thus their decomposition method may not perform
effectively in some cases.
maxðFM 1 ;    ; FMm Þ  minðFM 1 ;    ; FM m Þ
NDFM ¼ Pm ð1Þ
i¼1 FM i
where FM is the focus-measure, and m is the number of the source
images.
P4
1 2  
i¼1 NDFM i
M¼4 1
 std NDFM 21 ;    NDFM 24  100 ð2Þ
NDFM
n
where NDFM1 is the NDFM for the root block-pair, NDFM2i ;
i ¼ 1 . . . 4g are the NDFMs for the block-pairs at the second level,
and std represents the standard deviation operator.
2.1.1. An effective quadtree decomposition strategy
To achieve the optimal subdivision of the source images, we
propose an effective quadtree decomposition strategy in this part.
For convenience, the proposed strategy is described for two source
images and the strategy can be easily extended to three or more
source images.
At the beginning, two source images represent the root block-
pair at the first level of the quadtree structure. If the decomposition
criterion is fulfilled in the root block-pair, then each block in the
pair is subdivided into four quadrants. And this process is repeated
on the smaller block-pairs, until the decomposition criterion is not
fulfilled in all the block-pairs.
Ideally, two situations may occur in a block-pair [3]: (1) one
block is fully focused and the other is fully defocused (see Situation
1 in Fig. 3); (2) both of the blocks are partially focused (see Situa-
tion 2 in Fig. 3). Obviously, Situation 1 is the desired situation, in
which the focused block is fully discriminated from the defocused
block. Thereby, the decomposition criterion for a block-pair can be
simply described as: if the block-pair fits Situation 1, then the fully
focused block can be found out in this pair; otherwise, the block-
pair fits Situation 2, each block in the pair should be further subdi-
vided into four quadrants.
To find out the situation, the maximum difference in focus-mea-
sures (MDFM) and the sum of the maximum difference in gradients
(SMDG) are firstly calculated on the block-pair. If MDFM P
0:98  SMDG, then the block-pair fits Situation 1, otherwise, this
block-pair fits Situation 2. The definitions and explanations of
MDFM and SMDG are described in detail as follows.
First, calculate the gradient map of each source image.
 X. Bai et al. / Information Fusion 22 (2015) 105–118
Fig. 1. The general process of the proposed algorithm and a demonstration example.
Fig. 2. A simple example of De’s detection method.
Fig. 3. Ideal situations on a block-pair with focus on shaded regions. Situation 1:
one block is fully focused, the other fully defocused; Situation 2: both are partially
focused.
GMi ¼ gradientðIi Þ; i ¼ 1; 2; ð3Þ
where {Ii, i = 1, 2} are the source images; {GMi, i = 1, 2} are the corre-
sponding gradient maps.
For the multi-focus images, a focused block should preserve
more sharp edges than the corresponding defocused blocks [3,9].
In images, the gradients indicate the edge information. Then the
strength of the gradients could be used as a measure of focus.
107
Therefore, to measure the focus of the block-pairs, the gradients
of the source images should be firstly calculated. The gradient used
here is the proposed weighted modified gradient (detailed infor-
mation is described in Section 2.2).
Second, find out the maximum gradient map and the minimum
gradient map of the source images, respectively. Their calculations
are mathematically expressed as Eqs. (4) and (5), respectively.
GMmax ðx; yÞ ¼ maxðGM1 ðx; yÞ; GM 2 ðx; yÞÞ; ð4Þ
GMmin ðx; yÞ ¼ minðGM1 ðx; yÞ; GM2 ðx; yÞÞ; ð5Þ
where GMmax and GMmin are the maximum and minimum gradient
map of all source images, respectively.
In Eqs. (4) and (5), GM1 and GM2 are the gradients of the source
images, respectively. Because GMmax at each location is chosen as
the greater gradient of GM1 and GM2 at the corresponding location,
GMmax approximates the gradient map of a fully focused image,
with respect to the source images [1]. Similarly, GMmin at each
location is selected as the minimum gradient of GM1 and GM2 at
the corresponding location, then it approximates the gradient
map of a fully defocused image, with respect to the source images.
Third, select out the gradient map, maximum gradient map and
minimum gradient map for the block-pair at the block area B, as
shown in Fig. 4.
gradi ðx; yÞ ¼ GMi ðx; yÞ; i ¼ 1; 2; ð6Þ
ðx;yÞ2B
gradmax ðx; yÞ ¼ GMmax ðx; yÞ; ð7Þ
ðx;yÞ2B
gradmin ðx; yÞ ¼ GMmin ðx; yÞ; ð8Þ
ðx;yÞ2B
where {gradi, i = 1, 2} are the gradient maps of the corresponding
blocks of the source images at the block area B; gradmax is the
maximum gradient map of {gradi, i = 1, 2}; gradmin is the minimum
gradient map of {gradi, i = 1, 2}.
108 X. Bai et al. / Information Fusion 22 (2015) 105–118
Fig. 4. A pair of block at the block area B. (X, Y) is the upper left corner, and size of
the block-pair is given by K.
Because all the block-pairs are part of the source images, the
gradient maps {gradi, i = 1, 2}, maximum gradient map gradmax
and minimum gradient map gradmin of the block-pair could be just
copied from the corresponding part of {GMi, i = 1, 2}, GMmax and
GMmin, respectively. By this way, the gradients of the subregions
need not to be recomputed. Therefore, the proposed algorithm will
be accelerated.
Fourth, calculate the focus-measure of each block in the pair, as
the mathematical expression given below.
XX
FM i ¼ gradi ðx; yÞ; i ¼ 1; 2;
where FMi is the focus-measure of the block i in the pair.
In Eq. (9), the gradient value gradi(x, y) is the focus-measure of
the pixel at the location (x, y) of the source image Ii. FMi simply
sums up all the gradients in the block i to measure the focus of
the whole block i, which has been verified as an effective way in
some commonly constructed focus-measures [3,9]. Though there
may exist some noise gradients in the gradient map gradi(x, y),
the sum of the noise gradients in the block i would be very small,
compared with the corresponding focus-measure FMi. Thus, the
focus-measure could partly suppress the influence of the noise
gradients.
Fifth, find out the maximum and minimum focus-measures of
the blocks in the pair, respectively.
FM max ¼ maxfFM1 ; FM 2 g;
FM min ¼ minfFM 1 ; FM2 g;
where FMmax and FMmin are the maximum and minimum focus-
measures of all the block in the pair, respectively.
In Eqs. (10) and (11), FMmax equals to the maximum focus-
measure of the blocks in the pair, FMmin equals to the minimum
focus-measure of the blocks in the pair. For a partially focused block,
the focus-measure of this block will become greater, with the
enlarging of the focused area in the block. And, the focus-measure
of this block will become smaller, with the shrinking of the focused
area in the block. Thus, if a block-pair fits in Situation 1, in which a
block is a fully focused and another is fully defocused, FMmax of this
block-pair would equal to the focus-measure of the fully focused
block and FMmax should be the extreme maximum focus-measure
of this block. Meanwhile, FMmin of this block-pair would equal to
the focus-measure of the fully defocused block and FMmin should
be the extreme minimum focus-measure of this block.
Then, the maximum difference in focus-measures (MDFM) for
the block-pair can be calculated as the difference between
maximum focus-measure FMmax and minimum focus-measure
FMmin, as expressed in Eq. (12).
MDFM ¼ FM max  FM min :
In Eq. (12), FMmax represents the maximum focus-measure of
the corresponding blocks in the pair, and FMmin represents the
minimum focus-measure of the corresponding blocks in the pair.
The MDFM equals to the maximum difference in focus-measures
of the block-pair. According to the explanation of Eqs. (10) and
(11), if a block-pair fits in Situation 1, FMmax of the block-pair
would equal to the focus-measure of the fully focused block and
obtain the extreme maximum focus-measure of this block. FMmin
of this block-pair would equal to the focus-measure of the fully
defocused block and obtain the extreme minimum focus-measure
of this block. In this case, the MDFM of this block-pair will obtain
the extreme maximum value. Therefore, if MDFM of a block-pair
equals to the difference in the focus-measures of a fully focused
block and the corresponding fully defocused block, then this
block-pair should fit Situation 1, in which there is a fully focused
block and a fully defocused block.
Finally, the sum of the maximum difference in gradients
(SMDG) can be calculated as Eq. (13).
XX
SMDG ¼ ðgradmax ðx; yÞ  gradmin ðx; yÞÞ
XX XX
¼ gradmax ðx; yÞ  gradmin ðx; yÞ: ð13Þ
In Eq. (13), SMDG of a block-pair is calculated as the sum of the
difference of the maximum gradient map gradmax and the mini-
mum gradient map gradmin at each location. The expression of
the SMDG can be rewritten as the difference of the sum of the max-
imum gradients and the sum of the minimum gradients, as shown
ð9Þ
in Eq. (13). As mentioned previous, the maximum gradient map
gradmax approximates the gradient map of a fully focused block,
and the minimum gradient map gradmin approximates the gradient
map of a fully defocused block. Thus, SMDG of a block-pair approx-
imates the difference in focus-measures of a fully focused block
and a corresponding fully defocused block. According to the expla-
nation of Eq. (12), if MDFM of a block-pair equals to the corre-
sponding SMDG, then this block-pair fits Situation 1, in which a
block is fully focused and another is fully out of focus. Also, this
relation of MDFM and SMDG is verified as followings.
According to Eqs. (6) and (9)–(12), MDFM of a block-pair, is actu-
ally equal to the absolute value of the sum of the difference of grad1
and grad2 at each location. According to Eqs. (6)–(8) and (13), SMDG
of the block-pair is equal to the sum of the difference of the greater
gradient of grad1 and grad2 and the smaller gradient of grad1 and
ð10Þ grad2 at each location. Thus, MDFM of each block-pair is always less
than or equal to the corresponding SMDG. And only when the gra-
ð11Þ
dient grad1 is less than the gradient grad2 at each location or the
gradient grad1 is greater than the gradient grad2 at each location,
MDFM of a block-pair will equal to the corresponding SMDG. That
is, only when a block is fully focused and a block is fully defocused,
MDFM of a block-pair will equal to the corresponding SMDG.
Therefore, if MDFM of a block-pair equals to the corresponding
SMDG, there must be a fully focused block and a fully defocused
block in the pair, i.e., this block-pair fits the Situation 1. After that,
the focused block can be simply found out by comparing the focus-
measures of the blocks in the pair. Otherwise, this block-pair fits
Situation 2, in which both blocks in the pair are partially focused.
Then, each block in this pair should be further subdivided in to four
quadrants.
However in the real images, there may exist some noise pixels
in any one of the corresponding blocks. As a result, for a block-pair
in Situation 1, MDFM of the block-pair usually appears a little less
than the corresponding SMDG. To suppress this defect, we choose
0.98  SMDG as the threshold value to discriminate the situation.
For a block-pair, if MDFM P 0:98  SMDG, this block-pair fits Situ-
ation 1. Otherwise, this block-pair fits Situation 2.
Hence, the proposed decomposition strategy is reasonable. In
ð12Þ
addition, in the proposed quadtree decomposition, the subdivision
of each block-pair only depends on the current block-pair itself
rather than its parent block or the energy distribution of the image.
Therefore, the focused blocks could be detected more reliably by
the proposed strategy than De’s decomposition method.
 X. Bai et al. / Information Fusion 22 (2015) 105–118
2.1.2. Detecting the focused blocks in a quadtree structure
In this part, we make use of the proposed quadtree decomposi-
tion strategy to detect the focused blocks from m (m P 2) source
images. The detection process is described as follows.
At the beginning, the m source images are input as the root
block-set at the first level of the quadtree structure. For the root
block-set, if MDFM P 0:98  SMDG, then this block-set fits Situa-
tion 1 and the fully focused block can be found out by measuring
the focus of the blocks in this set. Otherwise, this block-set fits Sit-
uation 2 and each block in the set should be further subdivided
into four smaller blocks at the second level. This process is
repeated on the smaller block-sets, until all focused blocks are
found out or the maximum level of the quadtree structure is
reached. Finally, the detected focused blocks are merged naturally
to form the focused regions [3].
In this paper, the source images are decomposed to the mini-
mum 2  2 sized blocks, and the corresponding level is the maxi-
mum level, as calculated in Eq. (14).
max level ¼ logfmaxðM  1; N  1Þg þ 1;
where M  N is the size of the source images; log is the logarithm
operator.
Before the quadtree decomposition, the dimensions of the
source images should be extended to a squared size and a power
of 2. In addition, the source images are input as the first level of
the quadtree structure. Therefore, if the minimum possible size
of the blocks is 2  2, the maximum level of the quadtree structure
equals to the integer part of log {max (M  1, N  1)} + 1.
When the source images are decomposed to the minimum
2  2 sized blocks, it would be good enough for the precise detec-
tion of the focused regions. Therefore, we choose its corresponding
decomposition level as the maximum level.
The process of detecting the focused blocks in a quadtree
structure is algorithmically summarized in Algorithm 1.
Algorithm 1.
(1) Read the m source images as the root block-set;
(2) Compute the gradient map of each source image;
(3) Calculate the maximum possible level (max level) of the
quadtree structure;
(4) Decompose the source images into a quadtree structure:
Set current level l equal to 1;
while (l < max level)
if the number of the block-sets at level l equals to zero
then
break;
end
for each block-set at level l
Compute the corresponding SMDG and MDFM;
if MDFM P 0.98  SMDG then
Compute the focus-measure of each block in the set;
if only one block has the maximum focus-measure
then
Extract this block from the corresponding source
image to the fusion image;
end if
else
Subdivide each block in this set into four quadrants;
end if
next block-set
l = l + 1;
loop
109
Fig. 5 shows a demonstration of the process of detecting the
focused regions from the source images in a quadtree structure.
In the detected focused regions (see (e and f)), the white regions
are the focused regions detected from the source image (a), the gray
regions are the focused regions detected from the source image (b),
and the black regions are the regions detected from both source
images. It can be seen from (c) that the proposed algorithm could
well decompose the source images into blocks with optimal sizes
in a quadtree structure. And in this structure, the focused blocks
have been well detected by using the proposed algorithm, as shown
in (e). From (d), we can see that De’s decomposition method decom-
poses the source images into almost all equal-sized blocks, rather
than achieve the optimal subdivision of the source images. Similar
to the block-based algorithms, the focused regions detected by De’s
method yield the block artifacts, as shown in (f). In addition, the
source images are decomposed to a manual level by the De’s algo-
rithm, thus the performance of De’s algorithm would greatly
depends on the decomposition level, similar to the block-based
algorithms. Whereas, the proposed decomposition method decom-
ð14Þ
pose the source images to the maximum level, and the subdivision
is optimized. Thereby, the proposed quadtree-based detection
method is more effective and robust than De’s method.
2.2. A new weighted focus-measure: the sum of the weighted modified
Laplacian
In order to detect the focused regions precisely, a new weighted
focus-measure called the sum of the weighted modified Laplacian
is proposed. The detailed information about the new focus-
measure is described as follows.
For the multi-focus images, a focused block should preserve
more sharp edges than the corresponding defocused blocks [3,9].
In images, the gradients represent the edge information. The
strength of the gradients could be used as a focus-measure, such
as the well-known focus-measures [9,14]: EOL, EOG, SML, etc.
However in the homogeneous regions, the corresponding gradi-
ents are small. The focus-measures of both the focused block and
the corresponding defocused blocks would also be small. There-
fore, the focus-measure of a focused block could not be clearly
distinguished from the corresponding defocused blocks by using
some commonly used focus-measures. And, the focus-measure of
a block in the homogenous regions could be easily affected by
the noise pixels.
To overcome this problem, we weight the gradient at each loca-
tion in a local window. By this way, the strength of the final gradi-
ent at each location would be enhanced by the nearby gradients.
For example, the gradients in the homogenous region would
become larger, if there are sharp gradients around this region.
Therefore, the difference in focus-measures between a focused
block and the corresponding defocused blocks would be increased.
Then, the focus-measure of a focused block could be well distin-
guished from the corresponding defocused blocks, either in the
texture regions or in the homogeneous regions.
In [9], SML has been experimentally proved to outperform the
other focus-measures. Hence, an assumption is made that the
weighted version of SML would be effective. The experimental
results prove that the weighted version of SML performs very well
on measuring the focus. Therefore, we propose the weighted ver-
sion of SML, named as the sum of the weighted modified Laplacian
(SWML).
2.2.1. Sum of the modified Laplacian
In [14], Nayar et al. noted that: when calculating the Laplacian,
the second derivatives in the x and y directions may have the
110 X. Bai et al. / Information Fusion 22 (2015) 105–118

Fig. 5. A demonstration of the detection of the focused regions from the source images in a quadtree structure. (a–b) The source images; (c) the quadtree structure obtained
at the tenth level by the proposed quadtree decomposition strategy; (d) the quadtree structure obtained at the fifth level by De’s decomposition method; (e) the focused
blocks detected by the proposed algorithm; and (f) the focused blocks detected by De’s algorithm.

opposite signs, so that they tend to cancel each other. Therefore, a 2.2.2. Sum of the weighted modified Laplacian
small value may be obtained at a high gradient location. To sup- To improve the detection accuracy of the focused blocks in the
press this defect, Nayar et al. proposed the modified Laplacian homogenous regions, we present a weighted version of ML, named
(ML), which takes the absolute values of the second derivatives as the weighted modified Laplacian (WML).
in the Laplacian. The discrete approximation to ML can be First, the ML gradients of the source images are calculated
expressed as: according to Eq. (15). Then, the WML gradient at the pixel location
(x, y) is calculated as the weighted sum of the ML gradients in a
MLðx; yÞ ¼ j2f ðx; yÞ  f ðx  step; yÞ  f ðx þ step; yÞj
small local window. The WML can be mathematically expressed as
þ j2f ðx; yÞ  f ðx; y  stepÞ  f ðx; y þ stepÞj; ð15Þ
X
xþN X
yþN
MLði; jÞ
where step is a variable spacing, which is used to adapt to the pos- WMLðx; yÞ ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; MLðx; yÞ > T;
2 2
i¼xN j¼yN 1 þ ði  xÞ þ ðj  yÞ
sible variations.
Then, the sum of the modified Laplacian (SML) at the location ð17Þ
(x, y) can be calculated in a small window around (x, y), as shown
in Eq. (16). where N determines the size of the window and T is a threshold
value.
X
xþN X
yþN
In Eq. (17), the WML gradient at the pixel location (x, y) is calcu-
SMLðx; yÞ ¼ MLði; jÞ; MLði; jÞ > T; ð16Þ
lated as the weighted sum of the ML gradients in a local window,
i¼xNj¼yN
the size of which is (2  N + 1)  (2  N + 1). In the WML gradient
where N determines the size of the window and T is a threshold map, the strength of the ML gradient at each location has been
value. propagated within a local window. With the increase of N, the size
 X. Bai et al. / Information Fusion 22 (2015) 105–118
of the window will be increased and the propagation range of the
ML gradient will be expanded. Also, the WML gradient at each
location, which equals the weighted sum of all the ML gradients
in the local window, will become much greater. Especially for the
homogenous region, if there are sharp gradients around these
regions within the local window, the gradients of the homogenous
regions would be strongly enhanced. Meanwhile, the related
weight of the gradient at each location of the local window is
inversely proportional to the spatial distance between the gradient
location (i, j) and the center of the local window (x, y). Therefore,
the strength of the WML gradient at each location is mainly con-
tributed by the nearby ML gradients. And the strength of the gradi-
ents of the defocused region would not be influenced very much by
the nearby focused regions. Overall, if an appropriate value of N is
chosen, then the WML gradients would be well distributed in the
gradient map. Thereby it can be further used to accurately measure
the focus of the entire image.
Then, the sum of the weighted modified Laplacian (SWML) at
the location (x, y) can be calculated as:
X
xþN X
yþN
SWMLðx; yÞ ¼ WMLði; jÞ; ð18Þ
i¼xNj¼yN
where N determines the size of the window.
In Eq. (18), SWML of a window is computed as the sum of the
WML gradients in the window, similarly to SML. Here, we assume
that an appropriate value of N has been chosen in Eq. (17), and then
the WML gradients of the source images could be properly distrib-
uted in the gradient maps. For the multi-focus images, the WML
gradients in a focused region could be much greater than the
WML gradients in the corresponding defocused regions, either in
the texture regions or in the homogenous regions. Thus, SWML of
a focused region could be clearly discriminated from the corre-
sponding defocused regions. Also, because of the sum effect of
SWML, this new focus-measure could partly suppress the noise
gradients. Therefore, the new weighted focus-measure SWML
would be able to effectively measure the focus of the images.
Fig. 6 shows a comparison example of SML and SWML. (a and b)
are the source images. (c and d) are the ML gradient maps of (a and
b). (f and g) are the WML gradient maps of (a) and (b). From (c and
f), it can be seen that the strength of the ML gradients has been
locally propagated in the WML gradient map. In the homogeneous
regions, the strength of the gradients has been effectively
enhanced, such as the flower area in (f). The same phenomenon
can be seen from (d and g). In addition, we make use of SML and
SWML to detect the focused regions from the source images by
the proposed algorithm. The detected regions are shown in (e and
h), respectively. In the detected focused regions, the white regions
are the focused regions detected from the source image (a), the gray
regions are the focused regions detected from source image (b), and
the black regions are the focused regions detected from both source
images. Obviously, the detected focused regions by using SWML
(see (h)) have better consistency than SML (see (e)), especially in
the homogeneous regions, such as the brick areas or the flower
areas. It implies that the proposed focus-measure SWML performs
better than SML in the detection of the focused regions.
2.2.3. Parameter selection
First, the settings of the parameters step and T in Eq. (15) are
given. ML is a commonly used gradient. In [9], the experiments
have verified the advantage of the SML over other focus-measures
in measuring the focus. In this paper, the corresponding parame-
ters step and T are set equal to 1 and 5, as suggested in [9]. By this
way, the gradients introduced by the noise pixels could be
effectively suppressed. And the ML gradients of the source images
could be well generated, such as the example shown in Fig. 6.
111
Then, we only need to choose an appropriate value of N to gen-
erate the WML gradients of an image. From Eq. (17), we can know
that the value of N determines the size of the weighted window,
which is (2  N + 1)  (2  N + 1). N also determines the propaga-
tion range of the ML gradients in the WML gradient map. It is obvi-
ous that the value of N should be affected by the size of the
homogenous regions in the used image. If the image has
large homogenous regions, like the background area in the ‘Lab’
image (see Fig. 1), the value of N should be large. If the image
has small homogenous regions, such as the brick areas in the
‘Flower’ image (see Fig. 6(a and b)), the value of N should be small.
For a specified image, if the value of N is very small, the energy of
the gradients could not be adequately propagated. If the value of N
is very large, the gradients in the defocused region might be over-
whelmed by the gradients in the focused regions. Different values
of N have been experimentally tested, ranging from 5 to 10. And we
find that if the value of N equals to 8, the WML gradients would be
well propagated in most of the cases. Then, the new focus-measure
SWML could accurately measure the focus of the corresponding
blocks.
2.3. Reconstruction of the focused regions
Although the proposed quadtree detection method and the new
weighted focus-measure WML are very effective, there are still
some small spurious regions inside the detected focused regions,
as shown in Figs. 5(e) and 6(h). It is obvious that the small isolated
blocks should not exist in a focused region [15]. The size of a
focused region must be much wider than the spurious blocks
inside it [1,3]. In this paper, we make use of two sequential filters
to reconstruct the focused regions. First, a morphological filter is
used to process the focused regions. In this way, the small lines
or burrs could be eliminated and the nearby regions could be con-
nected. Then, a small region filter is applied to remove the small
isolated blocks in the focused regions detected from each source
image. Finally, the focused regions could be well extracted from
the source images and combined to form the final fusion image.
2.3.1. Morphological filter
In the detected focused regions, there may be some small lines
or burrs in the connection portions, and some nearby regions may
be disconnected by the inappropriate decisions. Thus, an alternat-
ing opening and closing operators with a small disk structuring
element [1] is applied to process the focused regions detected from
each source image. In this way, the small lines or burrs could be
eliminated, the connection portions of the focused regions could
be smoothed, and the nearby regions would be combined as a
whole region.
The morphological filter is used to do a minor adjustment of the
focused regions. Thus, a small size of the disk structuring element
would be enough for the deletion of the small burrs and the con-
nection of the nearby regions. Also, different sizes of the disk struc-
turing elements, with radius ranging from 2 to 5, have been used to
process the detected focused regions. We found that, when the
radius of the disk structuring element equals to 3, the small lines
or burrs could be well deleted and the nearby regions could be
right connected. Therefore, we use 3 as the radius of the disk
structuring element.
2.3.2. Small region filter
Usually, there may exist some small isolated regions inside a
focused region. To handle this problem, we utilize a small region fil-
ter to delete the small isolated regions or fill the holes in the focused
regions detected from each source image [15]. Suppose that the size
of the source images is M  N. Based on the visual observation, if the
area of an image region is less than (M  N)/40, this region is a very
112 X. Bai et al. / Information Fusion 22 (2015) 105–118

Fig. 6. A comparison example of SML and SWML. (a and b) The source images; (c and d) the ML gradient maps of the source images; (f and g) the WML gradient maps of the
source images; (e and h) the focus regions detected by using SML and SWML, respectively.

small region comparing with the whole image. Therefore, the small where Ii (i = 1, . . . , m) are the source images; GMi (i = 1, . . . , m) are
regions in this paper are defined as those regions whose area are the corresponding gradient maps of the source images.
less than (M  N)/40. The experimental results demonstrate that Because the proposed WML gradient is very effective and the
the small isolated regions could be effectively removed in most of corresponding gradient maps have been already achieved. Thus,
the cases by comparing the size of the region with (M  N)/40. After we adopt the proposed WML gradient here.
that, the consistency of the finally detected focused regions would Fig. 7 shows a demonstration example of the reconstruction
become better. process. (a and b) are the source images. (d) is the focused regions
detected by the proposed detection method. There are some iso-
2.3.3. Generating the fusion image lated blocks inside the detected focused regions. From (e), it can
The final fusion image can be produced as the following steps. be seen that the small lines are deleted and the nearby regions
are well connected by using the morphological filter. In (f), all
(1) The finally detected focused regions are copied directly from the small isolated regions are deleted by using the small region fil-
the source images to the fusion image. ter. Finally, the focused regions are well extracted and combined to
(2) There may exist some regions belonging to no image or more form one fully focused image, as shown in (c).
than one image, here the value of each pixel in these regions
is copied as the pixel from the source image i which has the 2.4. Comparison with the previous work
maximum gradient at this location. If the gradients of differ-
ent source images at a pixel location have the same value, Through the overall process of the proposed algorithm, it can be
then the value of the pixel at this location is copied as the concluded that compared with De’s algorithm, the proposed
average value of the pixels from all the source images. The algorithm mainly has two improvements: (1) presenting an effec-
mathematical expression is shown below [2]. tive quadtree decomposition strategy and (2) introducing a new
weighted focus-measure.
8 m
> X In De’s algorithm [3], the subdivision of each block-set greatly
>
> 1
Ii ðx; yÞ; if GMp ðx; yÞ ¼ GMq ðx; yÞ;
>m
> relies on the energy distribution of the source images and the cor-
>
>
< i¼1 responding parent block-set. Because the energy distribution of the
IF ðx; yÞ ¼ p – q; p; q 2 ½1; m ð19Þ image is always random, the optimal decomposition of the source
>
>
> I
> i ðx; yÞ; i ¼ arg maxðGMj ðx; yÞÞ; images usually could not be achieved. Whereas in the proposed
>
> j
>
: quadtree decomposition strategy, the subdivision of each block-
j 2 ½1; m
set only depends on the block-set itself, and the subdivision of each
 X. Bai et al. / Information Fusion 22 (2015) 105–118
Fig. 7. A demonstration of the reconstruction process. (a and b) The multi-focus ‘Disk’ images; (c) the fusion image of the proposed algorithm; (d) the focused regions
detected by the proposed algorithm; (e and f) the focused regions processed sequentially by using the morphological filter and small region filter.
block-set is based on a reasonable approximation method. Thus,
the source images could be optimally subdivided by the proposed
quadtree decomposition strategy. Therefore, the proposed quad-
tree decomposition strategy is more reasonable and reliable than
De’s decomposition method. Different from the general focus-
measures, the proposed focus-measure uses the modified
Laplacian to suppress the noise interference and also uses the
weighted method to properly propagate the energy of the gradient.
Thus, the focused blocks could be precisely detected by using the
proposed focus-measure SWML, either in the texture regions or
in the homogenous regions. Therefore, SWML is superior to the
commonly used focus-measures, especially in the detection of
focused blocks in the homogenous regions.
Moreover, different experiments have demonstrated that the
proposed algorithm is effective and outperforms some existing
algorithms.
3. Experimental results and discussions
To show the effectiveness of the proposed algorithm, we com-
pare our algorithm with some existing algorithms, including De’s
algorithm [3], contrast gradient based algorithm [16], Gradient
pyramid based algorithm [17], DWT based algorithm [18], PCA
based algorithm [19], and shift invariant DWT based algorithm
(SIDWT) [20].
De’s algorithm is based on the quadtree structure. The Gradient
pyramid, DWT, and SIDWT based algorithms are based on the
multi-scale decomposition, so these algorithms are also decomposi-
tion based algorithms. And, the PCA based algorithm is a commonly
used and effective algorithm for image fusion. Moreover, the pro-
posed algorithm is an effective algorithm for multi-focus image
fusion, through quadtree decomposition of the source images.
Thereby, using these algorithms as the comparison algorithms is
reasonable.
In this paper, the experiments have been performed on several
commonly used multi-focus image sets [1,3,8,9,12,13,26,27]. The
performance of the algorithms is evaluated by both the qualitative
and quantitative methods. Some comparison examples are shown
below.
113
3.1. Qualitative comparison
For the multi-focus image fusion, good visual effect means
effective fusion. To do the qualitative comparison, the performance
of the algorithms should be evaluated by measuring the vision
quality of the fusion images. Here, three comparison examples
are given, as shown in Figs. 8–10 and some discussions on the
experimental examples are stated as follows.
Fig. 8 shows the fusion results of the ‘Disk’ image set. It is obvi-
ous that the fusion images by the Gradient pyramid based
algorithm (Fig. 8(c)) and PCA based algorithm (Fig. 8(d)) appear
low in contrast. The fusion images by the Gradient pyramid based
(Fig. 8(c)), PCA based (Fig. 8(d)), contrast pyramid based (Fig. 8(e)),
DWT based (Fig. 8(f)) and SIDWT based (Fig. 8(g)) algorithms suffer
from the blurring effects, labeled by the yellow and orange rectan-
gles. For De’s algorithm, its fusion image (Fig. 8(h)) yields blocking
artifacts and blurring edges, labeled by the yellow rectangles. The
fusion image (Fig. 8(i)) produced by the proposed algorithm shows
the best visual effect.
As is known, the contrast pyramid based, DWT based, gradient
contrast based, PCA based and SIDWT based algorithms are the
transform domain techniques, and they would have a smooth
effect on the details of the image. Thereby, their fusion images
would suffer from the blurring effects around the edge area. As
for De’s algorithm, their quadtree decomposition method may be
not effective in some cases. In the experimental examples, the
source images are decomposed into almost all equal-sized blocks
by De’s algorithm. Therefore, the fusion images of De’s algorithm
usually suffer from blocking artifacts. While the proposed algo-
rithm could well extract the focused regions from the source
images, and maintain them in good consistency, therefore the
fusion images would be produced with good visual quality. And
the same conditions can be seen in Figs. 8 and 9.
Fig. 9 shows the fusion results of the ‘Pepsi’ image set. The
fusion images of the Gradient pyramid based algorithm (Fig. 9(c))
and PCA based algorithm (Fig. 9(d)) appear to be reduced in con-
trast. The fusion images of the contrast pyramid based (Fig. 9(e)),
DWT based (Fig. 9(f)) and SIDWT based (Fig. 9(g)) algorithms have
the blurring effects around the text regions, labeled by the yellow
114 X. Bai et al. / Information Fusion 22 (2015) 105–118
Fig. 8. The multi-focus ‘Disk’ images and its fusion images. (a and b) The source images; (c–i) the fusion images by using contrast pyramid based algorithm, DWT based
algorithm, gradient pyramid based algorithm, PCA based algorithm, SIDWT based algorithm, De’s algorithm and the proposed algorithm, respectively.
rectangles. Also, the fusion image of De’s algorithm (Fig. 9(h)) has
the blocking artifacts in the character ‘P’ area, labeled by the yellow
rectangle. In the end, the fusion image (Fig. 9(i)) of the proposed
algorithm is fully focused and good for the human visual
perception.
Fig. 10 shows the fusion results of the ‘Lab’ image set. The fusion
images by the frequency domain algorithms (Figs. 10(c)–(g)) all
have the blurring effects around the head area, labeled by the yel-
low rectangles. The fusion images of the gradient pyramid based
algorithm (Fig. 10(c)) and PCA based algorithm (Fig. 10(d)) appear
in low contrast. The fusion image (Fig. 10(h)) of De’s algorithm
yields the blocking artifacts in some portions, labeled by the
orange rectangles. As expected, our algorithm produces a fusion
image with the best visual quality, as shown in Fig. 10(i).
The experimental examples show that the proposed algorithm
performs better than the other algorithms. Also, it indicates that
the proposed algorithm is competent for the fusion of the multi-
focus images.
3.2. Quantitative comparison
To assess the fusion images objectively, the fusion algorithms
are also evaluated by three quantitative indices, which are gradient
similarity metric (QGM) [1,3], edge based similarity metric (QAB/F)
[21,22] and normalized mutual information (QMI) [23], respec-
tively. Their mathematical definitions are given below, followed
by the results and discussions.
3.2.1. Gradient similarity metric: QGM
The first metric is the gradient similarity between the source
images and the fusion image. It is based on the observation that
a pixel has a high gradient value when it is focused [1,8]. For a
set of multi-focus images {Ii, i = 1  m}, the gradient maps {GMi,
i = 1  m} are firstly calculated. Then, these gradients are combined
into GM by taking the maximum gradient value at each location.
The mathematical expression can be described as
GMðx; yÞ ¼ max fGMi ðx; yÞg: ð20Þ
i¼1;...;m
Thereby, GM(x, y) has the highest gradient at each location. It
could be used to approximate the gradient map of the original
scene which is focused everywhere. Assuming that GM’ is the gra-
dient map of the fusion image. Then, GM and GM0 are more similar,
the performance of the corresponding algorithm is better. This gra-
dient similarity can be calculated as follows [1,3].
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P 2
ðGMðx; yÞ  GM0 ðx; yÞÞ
Q GM ¼ 1  qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P
ffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi :
P ð21Þ
GM2 ðx; yÞ  GM02 ðx; yÞ
As is known, the Robert gradient [21] is simple and easy for
computation. Because the Robert operator [21] performs very well
in the experiments, the Robert operator is used here.
3.2.2. Edge based similarity metric: QAB/F
An edge based similarity metric (QAB/F) proposed by Xydeas and
Petrovic [22,23], evaluates the amount of the edge information
 X. Bai et al. / Information Fusion 22 (2015) 105–118 115

Fig. 9. The multi-focus ‘Pepsi’ images and its fusion images. (a and b) The source images; (c–i) the fusion images by using contrast pyramid based algorithm, DWT based
algorithm, gradient pyramid based algorithm, PCA based algorithm, SIDWT based algorithm, De’s algorithm and the proposed algorithm, respectively.

transferred from the source images to the fusion image. Generally, Usually, MI(X, Y) is normalized through dividing MI(X, Y) by
for two source images A and B, and a fusion image F, QAB/F is com- (H(X) + H(Y))/2 [25]. H(X) and H(Y) are the entropy of X and Y,
puted as respectively.
PN PM AF For the multi-focus image fusion, it has three variables: two
A
m¼1 ðQ ðn; mÞw ðn; mÞ þ Q BF ðn; mÞwB ðn; mÞÞ
Q AB=F ¼ n¼1
PN PM ð22Þ multi-focus images (IA and IB) and one fusion image (IF). QMI can
A þ wB ðn; mÞÞ
n¼1 m¼1 ðw ðn; mÞ be calculated as the sum of the normalized mutual information
between each source image and the fusion image. The mathemat-
where wA(n, m) and wB(n, m) are the local importance perceptual
ical expression is:
factors, QAF and QBF are the edge information preservation values.
In this paper, the parameters are used as suggested in [22]. MIðIA ; IF Þ MIðIB ; IF Þ
Q MI ðA; B; FÞ ¼ þ ð24Þ
ðHðIA Þ þ HðIF ÞÞ=2 ðHðIB Þ þ HðIF ÞÞ=2
3.2.3. Normalized mutual information: QMI
Mutual information (MI) is a quantitative measure of the
mutual dependence of two variables [24]. It shows the measure- 3.2.4. Quantitative comparison results
ment of the information shared by two images. Mathematically, The three quantitative indices are calculated according to their
MI between two discrete random X and Y variables is defined as definitions, and the results are shown in Tables 1–3. The bold
XX pðx; yÞ values are the maximum values in the corresponding rows. If the
MIðX; YÞ ¼ pðx; yÞ  log2 ð23Þ values of the three indices are larger, the performance of the
x2X y2Y
pðxÞ  pðyÞ
corresponding algorithm is better.
116 X. Bai et al. / Information Fusion 22 (2015) 105–118

Fig. 10. The multi-focus ‘Lab’ images and its fusion images. (a and b) The source images; (c–i) the fusion images by using contrast pyramid based algorithm, DWT based
algorithm, gradient pyramid based algorithm, PCA based algorithm, SIDWT based algorithm, De’s algorithm and the proposed algorithm, respectively.

Table 1
Gradient similarity metric.

Image sets Contrast pyramid DWT Gradient pyramid PCA SIDWT De’s algorithm Proposed algorithm
Clock 0.9735 0.9751 0.9621 0.9543 0.9819 0.9778 0.9762
Disk 0.9725 0.9755 0.9535 0.9239 0.9757 0.9725 0.9774
Flower 0.9648 0.9669 0.9445 0.9322 0.9645 0.9607 0.9671
Lab 0.9807 0.9818 0.9646 0.9463 0.9831 0.9813 0.9834
OpenGL 0.9556 0.9642 0.9376 0.8853 0.9662 0.9592 0.9608
Pepsi 0.9852 0.9854 0.9670 0.9449 0.9822 0.9860 0.9862

Table 2
Edge based similarity metric.

Image sets Contrast pyramid DWT Gradient pyramid PCA SIDWT De’s algorithm Proposed algorithm
Clock 0.6880 0.6619 0.6597 0.6137 0.7003 0.7384 0.7373
Disk 0.6849 0.6503 0.6440 0.5365 0.6822 0.7340 0.7383
Flower 0.6444 0.6222 0.6222 0.6197 0.6583 0.6821 0.6962
Lab 0.6910 0.6616 0.6585 0.5651 0.6839 0.7442 0.7470
OpenGL 0.6946 0.6784 0.6707 0.5893 0.7058 0.7289 0.7308
Pepsi 0.7548 0.7293 0.7342 0.6362 0.7445 0.7847 0.7847

The gradient similarity metric (QGM) measures the gradient sim-
ilarity between the source images and fusion image. It can be seen
from Table 1 that, the proposed algorithm performs the best in the
most of the cases, which implies that the proposed algorithm can
well preserve the sharp gradients from the source images.
The edge based similarity metric (QAB/F) evaluates the amount of
edge information transferred from the source images to the fusion
image. Compared with QGM, QAB/F could be more precise and
reliable, because QAB/F is associated with both the strength and
orientation information of the edges. The comparison results in
Table 2 show that the proposed algorithm performs better than
other algorithms in most of the cases. It indicates that the proposed
algorithm could well preserve the prominent texture information
from the source images.
 X. Bai et al. / Information Fusion 22 (2015) 105–118 117

Table 3
Normalized mutual information.

Image sets Contrast pyramid DWT Gradient pyramid PCA SIDWT De’s algorithm Proposed algorithm
Clock 0.9724 0.8827 0.8670 0.9703 0.9436 1.2075 1.2244
Disk 0.8275 0.7404 0.7466 0.8373 0.8033 1.1373 1.1459
Flower 0.7468 0.6659 0.6661 0.7566 0.7127 1.0171 1.1255
Lab 1.0015 0.9324 0.8953 1.0271 0.9935 1.2280 1.2456
OpenGL 0.9810 0.8914 0.7856 1.0279 0.9575 1.2714 1.2906
Pepsi 0.9643 0.8925 0.8656 0.9759 0.9289 1.2086 1.2483

Table 4
Comparison of the computational costs (s).

Image sets Contrast pyramid DWT Gradient pyramid PCA SIDWT De’s algorithm Proposed algorithm
Clock(500  503) 0.0523 0.0750 0.0800 0.0075 0.3072 1.5036 0.9946
Disk(640  480) 0.0613 0.0906 0.0973 0.0092 0.3922 2.3228 1.7222
Flower(764  576) 0.0846 0.1322 0.1412 0.0141 0.5952 3.1220 1.8370
Lab(640  480) 0.0630 0.0895 0.1018 0.0097 0.3961 2.2095 1.8393
OpenGL(758  569) 0.0895 0.1270 0.1424 0.0130 0.5713 2.7748 1.8204
Pepsi(512  512) 0.0520 0.0790 0.0836 0.0080 0.3208 1.5156 1.0290

Note: the size in each bracket represents the size of the corresponding source images.

The normalized mutual information (QMI) is a quantitative mea-
sure of the mutual dependence between the source images and the
fusion image. It is clear from Table 3 that, the proposed algorithm
yields the best results in all the cases. This is because the proposed
algorithm can well combine the independent information from
each source image to form one single image. Here, the independent
information is referred as the focused regions detected from each
source image.
3.3. Comparison of the computational costs
Finally, we have compared the computational costs of the
fusion algorithms. All the algorithms are running in a computer
with Intel Processor (Intel Core i5-3470, 3.20 GHz) and 4 GB mem-
ory. The computational times are listed in Table 4.
Table 4 shows that, the transform domain algorithms (the con-
trast pyramid based, DWT based, gradient contrast based, PCA
based and SIDWT based algorithms) are all running fast. The spa-
tial domain methods (the proposed algorithm and De’s algorithm)
are running slightly slower than the transform domain algorithms.
Also, it is obvious that the proposed algorithm is running faster
than De’s algorithm in all the cases. In the experimental examples,
the proposed algorithm can subdivide the source images optimally,
whereas De’s algorithm subdivides the source images into almost
all equal-sized blocks. Thus, the decomposition times of the source
images by using De’s algorithm would be a lot more than the pro-
posed algorithm. And the computational cost of De’s algorithm will
be heavily increased with the increasing of its decomposition level.
Therefore, the proposed algorithm is faster than De’s algorithm.
Although the proposed algorithm has a little higher computational
cost than the transform domain algorithms, the proposed algo-
rithm outperforms all the transform domain methods and the
De’s algorithm from both the qualitative and quantitative evalua-
tions. Therefore, the proposed algorithm is competent for the
multi-focus image fusion.
4. Conclusion
This paper presents a simple yet effective quadtree-based algo-
rithm for multi-focus image fusion. In this work, we propose an
effective quadtree decomposition strategy and also present a
new weighted focus-measure, thus the focused regions could be
detected from the source images in a quadtree structure, effec-
tively and precisely. And the detected focused regions could be
well extracted from the source images and reconstructed to pro-
duce the fusion image.
Because SWML is sensitive to the homogeneous regions, the
focused regions could be detected more accurately by using SWML
than the commonly used focus-measures. In the proposed algo-
rithm, all source images are decomposed to the maximum level
of the quadtree structure, therefore the focused regions could be
detected precisely in the tree structure. Moreover, the proposed
algorithm is simple, effective and reliable, because of the proposed
quadtree decomposition strategy and the weighted focus-measure.
The experimental results show that the proposed algorithm per-
forms very well for multi-focus image fusion.
Acknowledgments
The authors thank the anonymous reviewers and editor very
much for their helpful and valuable comments, which substantially
improve the quality of the paper. The authors do acknowledge the
source images used in their experiments. Multi-focus image sets
‘Clock’, ‘Disk’, ‘Lab’ and ‘Pepsi’ are obtained from website [26].
Image sets ‘Flower’ and ‘OpenGL’ are obtained from website [27].
This work was partly supported by the National Natural Science
Foundation of China (61271023), Program for New Century Excel-
lent Talents in University (NCET-13-0020) and Fundamental
Research Funds for the Central Universities (YWF-14-YHXY-029,
YWF-13-T-RSC-028, YWF-13-JQCJ-026).
References
[1] I. De, B. Chanda, B. Chattopadhyay, Enhancing effective depth-of-field by image
fusion using mathematical morphology, Image Vis. Comput. 24 (12) (2006)
1278–1287.
[2] A. Saha, G. Bhatnagar, Q.M.J. Wu, Mutual spectral residual approach for
multifocus image fusion, Digit. Signal Process. 23 (4) (2013) 1121–1135.
[3] I. De, B. Chanda, Multi-focus image fusion using a morphology-based focus
measure in a quad-tree structure, Inf. Fusion 14 (2) (2013) 136–146.
[4] G. Qu, D. Zhang, P. Yan, Medical image fusion by wavelet transform modulus
maxima, Opt. Express 9 (4) (2001) 184–190.
[5] X. Bai, F. Zhou, B. Xue, Fusion of infrared and visual images through region
extraction by using multi-scale center-surround top-hat transform, Opt.
Express 19 (9) (2011) 8444–8457.
[6] F. Nencini, A. Garzelli, S. Baronti, L. Alparone, Remote sensing image fusion
using the curvelet transform, Inf. Fusion 8 (2) (2007) 143–156.
118 X. Bai et al. / Information Fusion 22 (2015) 105–118
[7] A.A. Goshtasby, S. Nikolov, Image fusion: advances in the state of the art, Inf.
Fusion 8 (2) (2007) 114–118.
[8] S. Li, J.T. Kwok, Y. Wang, Combination of images with diverse focuses using the
spatial frequency, Inf. Fusion 26 (7) (2001) 169–176.
[9] W. Huang, Z. Jing, Evaluation of focus measures in multi-focus image fusion,
Pattern Recogn. Lett. 28 (4) (2007) 493–500.
[10] W. Huang, Z. Jing, Multi-focus image fusion using pulse coupled neural
network, Pattern Recogn. Lett. 28 (9) (2007) 1123–1132.
[11] V. Aslantas, R. Kurbanan, Fusion of multi-focus images using differential
evolution algorithm, Expert Syst. Appl. 37 (12) (2010) 8861–8870.
[12] S. Li, B. Yang, Multifocus image fusion using region segmentation and spatial
frequency, Image Vis. Comput. 26 (7) (2008) 971–979.
[13] L. Chen, J. Li, C.L.P. Chen, Regional multifocus image fusion using sparse
representation, Opt. Express 21 (4) (2013) 5182–5197.
[14] S.K. Nayar, Y. Nakagawa, Shape from focus, IEEE Trans. Pattern Anal. Mach.
Intell. 16 (8) (1994) 824–831.
[15] Y. Zhang, L. Ge, Efficient fusion scheme for multi-focus images by using
blurring measure, Digit. Signal Process. 19 (2) (2009) 186–193.
[16] A. Toet, L.J. Ruyven, J.M. Valeton, Merging thermal and visual images by a
contrast pyramid, Opt. Eng. 28 (7) (1989) 789–792.
[17] V.S. Petrovic, C.S. Xydeas, Gradient-based multiresolution image fusion, IEEE
Trans. Image Process. 13 (2) (2004) 228–237.
[18] H. Li, B.S. Manjunath, S.K. Mitra, Multisensor image fusion using the wavelet
transform, Graph. Model. Image Process. 57 (3) (1995) 235–245.
[19] P.S. Chavez, A.Y. Kwarteng, Extracting spectral contrast in Landsat thematic
mapper image data using selective principal component analysis,
Photogramm. Eng. Remote Sens. 55 (3) (1989) 339–348.
[20] M. Unser, Texture classification and segmentation using wavelet frames, IEEE
Trans. Image Process. 4 (9) (1995) 1549–1560.
[21] L.G. Roberts, Machine Perception of Three-Dimensional Solids, MIT Lincoln
Laboratory Technical Report, 1963.
[22] C. Xydeas, V. Petrovic, Objective image fusion performance measure, Electron.
Lett. 36 (4) (2000) 308–309.
[23] V. Petrovic, C. Xydeas, Sensor noise effects on signal-level image fusion
performance, Inf. Fusion 4 (3) (2003) 167–183.
[24] A. Collignon, F. Maes, D. Vandermeulen, P. Suetens, G. Marchal, Automated
multi-modality image registration based on information theory, Int. Conf.
Inform. Process. Med. Imag. (1995) 263–274.
[25] T.O. Kvalseth, Entropy and correlation: some comments, IEEE Trans. Syst. Man
Cybern. 17 (3) (1987) 517–519.
[26] http://www.quxiaobo.org/software/software_FusingImages.html
[27] http://www.imgfsr.com/sitebuilder/images