Sei sulla pagina 1di 4

149

INCAM 2019 Conference, 3-5 July 2019, IISc Bangalore.

Type-2 Fuzzy Set based Multimodal Medical Image Fusion


T.Tirupal1, B.Chandra Mohan2 and S.Srinivas Kumar3
1
Department of ECE, G.Pullaiah College of Engineering and Technology, Kurnool 518002
2
Department of ECE, Bapatla Engineering College, Bapatla 522101
and
3
Department of ECE, JNTUA, Ananthapuramu 515002
Presenting Author: T.Tirupal, Email: tirutalari@gmail.com

Program: Oral Presentation


Abstract
The main objective of image fusion for multimodal medical images is to retrieve valuable information by
combining multiple images obtained from various sources into a single image suitable for better diagnosis. In
general, the visibility of structural details in medical images is difficult to interpret. Soft computing
techniques have been applied to enhance the images and to extract the details from the images. Fuzzy sets are
more suitable for medical images since the images have more uncertainties. A fuzzy set approach is proposed
in this paper. Type-2 fuzzy sets are used in this work. The efficiency of the proposed method is demonstrated
by applying the algorithm on several pairs of medical images. The fused image is analyzed qualitatively
using the attributes like spatial frequency, entropy, and edge information preservation. Simulated results are
also compared with the existing works in the recent literature like non-fuzzy, Type-1 fuzzy and intuitionistic
fuzzy sets.
Keywords: image fusion; diagnosis; fuzzy sets; type-2 fuzzy sets; spatial frequency.
1 Introduction cause undesired edges in the fused image. Multiresolution
techniques like wavelet transforms [5], do not provide
Multimodal medical image fusion is the process of shift invariance in the coefficients and this can be
uniting two multimodal medical images to increase the overcome by using a redundant wavelet transform (RWT)
excellence of output image. In combining medical image [6]. The major drawback of RWT is increased
pairs such as CT-MRI (computed tomography-magnetic computational complexity since the wavelet is
resonance imaging), MRI-MRA (magnetic resonance undecimated and all these methods are weak in boundary
imaging - magnetic resonance angiography), Xray - VA detection of a disease (in a medical image) present in the
(Xray - vibroacoustography), MRI-PET (magnetic fused image.
resonance imaging - positron emission tomography), and Fuzzy sets play a significant role in image processing to
MRI-SPECT (magnetic resonance imaging - single eliminate the ambiguity and vagueness present in images.
photon emission computed tomography) [1-3], image Fuzzy sets take uncertainty in the form of a membership
fusion is gradually attaining significance in healthcare function which lies in the interval [0, 1] where 0
and modern medicine. These sets of images are used for represents no membership and 1 represents full
extracting clinical facts that are complementary in nature. membership. The Fuzzy set theory proposed by Zadeh [7]
For example, CT image embeds with a lesser amount of in 1965 is not able to directly model uncertainties. The
distortion and delivers details regarding condensed generalized form of fuzzy set theory introduced by
structures like bones, MRI provides neurotic soft fleshy Atanassov [8] in 1986 is intuitionistic fuzzy set (IFS)
tissue information, MRA senses easily brain defects, X- which takes two uncertainty parameters – the
ray detects fractures and aberrations in bone position, VA membership degree and non-membership degree (due to
provides deepness and wideness of disease object, PET hesitation degree). The non-membership degree gives
and SPECT provide functional and metabolic information vague knowledge and solves adequately real time
of human brain. Therefore, we can say that not a single problems. Many uncertainties exist in every phase of the
image will provide all significant information and hence image processing and using IFSs these uncertainties can
multimodal medical image fusion is essentially necessary be removed and the image is enhanced in terms of
to achieve all possible complete information in a single contrast of the image.
merged image called fused image. There are different sources of uncertainties in Type-1
Several existing techniques of fusing images comprise fuzzy sets [9]; they are inaccurate measurements,
pixel averaging, a simplest image fusion method which disagreement of the membership values with the accurate
takes pixel by pixel average of the two images but leads membership values of the data, uncertainty in the
to contrast reduction. Principal component analysis location, shape or other parameters. Type-1 fuzzy sets are
(PCA) [4], degrades the performance of the fused image. not able to directly model such uncertainties because their
Pyramid decomposition methods fail to introduce spatial membership functions are totally crisp. The intuitionistic
orientation selectivity in the decomposition process and fuzzy set algorithms like Yager’s IFS, Sugeno’s IFS [10,

1
150

11, 12, 13] eliminate the uncertainties present in


multimodal medical images and dynamically enhance the

IType 2  y,  upper ( y),  lower ( y) | y  Y  (6)
contrast of the medical image fusion. and  upper ( x)   ( x)   lower ( x),   [0,1] where upper and
To find a more robust solution, a new class of fuzzy logic lower may be the linguistic hedges like dilation or
systems called Type-2 fuzzy logic systems [14] are concentration.
proposed and introduced for handling uncertainties.
Type-2 fuzzy sets minimize the effects present in medical 3 Proposed Type-2 Image Fusion Method
images and model uncertainties because their In this section, image fusion by the proposed method,
membership functions are themselves fuzzy. The fused Type-2 fuzzy set is discussed. Here, the membership
images obtained using the proposed Type-2 fuzzy function lies in an interval range with upper and lower
algorithm have better visual quality with sharp and membership levels [15] which represents uncertainty in a
smooth edges, fine texture and high clarity and these different and better way than Type-1 and IFS. The
images are also free from the problem of artifacts. algorithm for medical image fusion based on Type-2
2 Preliminaries fuzzy set is as follows:

Type-1 fuzzy set I, is an ordinary fuzzy set that can be (1) Read the two input registered images I1 and I 2 .
represented in a finite set Y  y1 , y2 , y3 ,..., yn  (2) The two images are fuzzified using the equation [16]
mathematically as R  Rmin
 I ( R)  (7)
Rmax  Rmin
I  ( y,  I ( y)) | y  Y  (1)
Where R defines the gray level of the image that
Where the function  I ( y) : Y  [0, 1] measures the degree
ranges from 0 to L  1 ( L is the maximum gray level
of belongingness or the membership function of an value). Rmin , Rmax represents the minimum and
element y in the finite set Y and the measure of non-
maximum gray level values of the images
belongingness or non-membership function is 1   I ( y) .
respectively. The function  I (R) measures the degree
Atanassov introduced a new fuzzy set called intuitionistic
of membership function.
fuzzy set which takes both membership function  ( y ) and (3) The upper and lower membership values for both
non-membership function v ( y ) into consideration holding input images are computed using the below equations
 I ( y)  [0, 1], v I ( y)  [0, 1] . An intuitionistic fuzzy set I
in a finite set Y holding the condition 0   I ( y)  vI ( y)  1  upper ( R)  [  I ( R)]0.5
(8)
is mathematically represented as  lower ( R )  [  I ( R )]1 / 0.5

I  ( y,  I ( y), v I ( y)) | y  Y  (2) Where  I (R) represents the fuzzified input image.
(4) Find fuzzified Type-2 fuzzy set images for the two
Atanassov also introduced a new parameter  I ( y ) called input images by using below equations and denote as
intuitionistic fuzzy index or hesitation degree which I Y 1 and I Y 2
arises due to lack of knowledge or personal error in
assigning the membership degree. An IFS I in a finite set
Y with the condition  I ( y)   I ( y)  v I ( y)  1 , and

I Y1  I1 , 1upper ( R), 1lower ( R)  (9)

introduction of hesitation degree may be represented as IY 2  I , 


2
upper
2 ( R),  2lower ( R) (10)
I  ( y,  I ( y), v I ( y),  I ( y)) | y  Y  (3) (5) Decompose the two enhanced images obtained from
where 0   I ( y)  1 , for each y  Y . step 4 into m  n blocks and compute the contrast
visibility of each corresponding block using an
Type-2 fuzzy sets are the fuzzy sets for which the equation
membership function is not a single value for every 1 I (i, j )   k
element but an interval. A Type-2 fuzzy set may be CV   (13)
p  q (i , j )Bk k
written as
 Where  k and p  q are the mean and magnitude of the
IType2  y,  I ( y) | y  Y  (4)
block Bk , respectively.

Where  I ( y ) is the Type-2 membership function. The (6) Construct a decision map (DM) to determine the
uncertainty in the primary memberships of Type-2 fuzzy combination of pixels of the two images as:
set represents the footprint of uncertainty or FOU. An 1 I I
if CVi Y 1  CVi Y 2

interval-based Type-2 fuzzy set is constructed by defining  I I (20)
DM   1 if CVi Y 1  CVi Y 2
upper and lower membership values of Type-1 fuzzy set  I I
and is written as 0 if CVi Y 1  CVi Y 2

 upper  [ ( y)] CVi


IY 1
and CVi I Y 2 are the contrast visibilities of the
(5)
 lower  [ ( y)]1 /  ith block of images I Y 1 and I Y 2 respectively. DM is the
decision map built by taking the decision for each
Where 0    1. so, more practical form of representing coefficient using the contrast visibility of the
Type-2 fuzzy set is respective block.
2
151

(7) The fused image FI Type2 is obtained based on DM boundaries are common in medical images. Type-2 fuzzy
and is finally defuzzified to get a crisp image using sets are well suited for such imprecise information.
defuzzification equation
I ' (i, j )  (l max  l min )  FI Type2 (i, j )  l min (23)
Where l min and l max in the above equation represent the
lowest and extreme gray level values of the fused
(a) (b) (c) (d)
image FI Type2 .

4 Experimental Results
Experiments on two different modality images are
performed in this work and all the source images are (g)
(e) (f) (h)
assumed to be properly registered. The first example Figure 1. Fusion results for CT and MRI images. (a) CT image
which is shown in figure 1 deals with CT and MRI (b) MRI image (c) Fused image by non-fuzzy (d) Fused image
images [17]. Fusing these images provides abundant by Type-1 fuzzy set (e) Fused image by IFS (f) Fused image by
information regarding bones, hard and soft tissues YIFS (g) Fused image by SIFS (h) Fused image by Type-2 fuzzy
information in a single image which helps in better
diagnosis of a disease. Figure 1(h) is the fusion result of a
proposed method which visually proves that the fused
image is of high contrast and luminance than the fused
images of other methods such as non-fuzzy, Type-1 fuzzy
and intuitionistic fuzzy sets. Further comparison of (a) (b) (c) (d)
results is performed using different objective criteria [18]
and the results are listed in Table 1.
The second example deals with T1 weighted MR image
and MRA image with some illness as white structures in
the image shown in figure 2. Fusing these two images (e) (f) (g) (h)
provides soft tissues information with abnormalities in a Figure 2. Fusion results for T1-weighted MR and MRA images.
more clear way with high spatial resolution. Figure 2(h) (a) T1-MR image (b) MRA image (c) Fused image by non-fuzzy
gives the fused image for the proposed method. Table 1 (d) Fused image by Type-1 fuzzy set (e) Fused image by IFS (f)
gives the further comparison of results in terms of Fused image by YIFS (g) Fused image by SIFS (h) Fused image
objective criteria. The third example deals with MRI and by Type-2 fuzzy
PET images shown in figure 3 and these images are taken
from Harvard University website [19]. Here, MRI image
is registered to corresponding PET image and both
images are gray colored. Fusion of these two images
gives anatomy of brain tissues and functional information
(a) (b) (c) (d)
of the brain with high spatial resolution and no spatial
distortion in a single image. Figure 3(h) shows the fused
image for the proposed method. Table 1 briefs the
objective criteria for MRI and PET medical data set.
The fourth set of images are MRI and SPECT brain
tumor images of size 256 x 256 taken from Harvard (e) (f) (g) (h)
University website [19] and are shown in figure 4 for Figure 3. Fusion results for MRI and PET images having Mild
evaluation of different fusion algorithms. Fusing the two Alzheimer’s disease. (a) MRI image (b) PET image (c) Fused
images, we get anatomical information and functional image by non-fuzzy (d) Fused image by Type-1 fuzzy set (e)
knowledge of the human brain in a single image. Figure Fused image by IFS (f) Fused image by YIFS (g) Fused image
4(h) shows the fused image by the proposed method and by SIFS (h) Fused image by Type-2 fuzzy
tumor is clearly enhanced when compared with other
methods, it is also compared in terms of fusion metrics in
Table 1.
5 Conclusion
(a) (b) (c) (d)
This paper presents a new method for fusing multimodal
medical images using Type-2 fuzzy sets. The proposed
algorithm is tested on several pairs of medical images. It
is found that the proposed method gives better results
visually and quantitatively with high contrast and
luminance. Improvement in the quality of the fused image (e) (f) (g) (h)
may be attributed to the fuzziness of the membership Figure 4. Fusion results for MRI and SPECT images having
function itself in Type-2 fuzzy sets. Medical images are Mild Alzheimer’s disease. (a) MRI image (b) SPECT image (c)
usually of low contrast. Vague regions and imprecise Fused image by non-fuzzy (d) Fused image by Type-1 fuzzy set
(e) Fused image by IFS (f) Fused image by YIFS (g) Fused
image by SIFS (h) Fused image by Type-2 fuzzy

3
152

Table 1. Objective evaluation of different image fusion methods


Spatial Edge
PSNR Entropy
Figure Fusion Method UQI SSIM CC Frequency Information
(dB) (bits/symbol)
(cycles/millimeter) Preservation
Non-Fuzzy [4] 63.75 0.623 0.996 0.822 5.797 24.19 0.542
Type-1 Fuzzy Set [9] 57.36 0.393 0.988 0.797 4.615 35.21 0.497
IFS [10] 59.52 0.665 0.991 0.848 5.357 11.24 0.75
1
YIFS [11] 59.99 0.738 0.993 0.647 6.806 32.23 0.745
(CT-MRI)
SIFS [12] 65.78 0.698 0.998 0.948 5.773 17.29 0.859
Proposed Method
62.09 0.723 0.956 0.616 6.683 27.44 0.789
(Type-2 Fuzzy Set)
Non-Fuzzy [4] 53.70 0.364 0.898 0.840 5.299 25.89 0.508
Type-1 Fuzzy Set [9] 52.83 0.212 0.962 0.670 2.166 37.98 0.357
2 IFS [10] 52.4 0.767 0.956 0.654 5.167 24.45 0.59
(T1-weighted YIFS [11] 64.39 0.415 0.999 1 6.960 38.12 0.710
MR-MRA) SIFS [12] 61.73 0.843 0.996 0.902 6.698 31.99 0.651
Proposed Method
65.14 0.777 0.976 0.952 6.880 32.48 0.739
(Type-2 Fuzzy Set)
Non-Fuzzy [4] 54.54 0.495 0.798 0.569 3.727 35.46 0.747
Type-1 Fuzzy Set [9] 53.15 0.083 0.981 0.197 2.251 55.09 0.709
IFS [10] 53.7 0.822 0.974 0.39 3.928 39.41 0.54
3 YIFS [11] 60.56 0.640 0.997 0.833 4.406 43.53 0.743
(MRI-PET)
SIFS [12] 59.32 0.855 0.994 0.453 4.655 42.38 0.771
Proposed Method
64.82 0.866 0.983 0.623 6.899 49.70 0.793
(Type-2 Fuzzy Set)
Non-Fuzzy [4] 55.26 0.378 0.890 0.073 1.225 44.18 0.505
Type-1 Fuzzy Set [9] 57.16 0.179 0.899 0.578 1.659 48.42 0.451
IFS [10] 54.81 0.527 0.977 0.318 1.282 29.41 0.54
4
YIFS [11] 56.64 0.446 0.986 0.656 4.181 36.07 0.754
(MRI-SPECT)
SIFS [12] 67.59 0.621 0.999 0.537 2.618 30.61 0.713
Proposed Method
64.96 0.630 0.9771 0.625 3.849 101.47 0.724
(Type-2 Fuzzy Set)

6 References [10] Balasubramaniam, P., Ananthi, V.P.: Image fusion


using intuitionistic fuzzy sets, Information Fusion,
[1] Baum, K.G., Raerty, K., Helguera, M., Schmidt, E.: 2014, 20, pp. 21–30
Investigation of PET/MRI image fusion schemes [11] Tirupal, T., Chandra Mohan, B., Srinivas Kumar,
for enhanced breast cancer diagnosis., in Proc. of S.: Multimodal Medical Image Fusion based on
IEEE seventh symposium conference on ’Nuclear Yager’s Intuitionistic Fuzzy Sets, Iranian Journal of
Science (NSS) (2007) Fuzzy Systems, vol.16, no.1, February 2019, pp.33-
[2] Gholam Hosseini, H., Alizad, A., Fatemi, M.: 48.
Integration of Vibro-Acoustography imaging [12] Tirupal, T., Chandra Mohan, B., Srinivas Kumar,
modality with the traditional mammography. S.: Multimodal medical image fusion based on
International Journal of Biomedical Imaging, Sugeno’s intuitionistic fuzzy sets, ETRI Journal,
Hindawi Publishing Corporation (2007) 2017, 39, (2), pp. 173–180
[3] James, A. P., Dasarathy, B. V.: Medical image [13] Manchanda, M., Sharma, R.: A novel method of
fusion: a survey of the state of the art. Information multimodal medical image fusion using fuzzy
Fusion (2014) transform, Journal of Visual Communication and
[4] Naidu, V. P. S., Raol, J. R.: Pixel-level image Image Representation, 2016, 40, pp. 197–217
fusion using wavelets and principal component [14] Karnik, N. N., Mendel, J. M., Liang, Q.: Type-2
analysis. Defence Science Journal, 58, 338-352, fuzzy logic systems. IEEE transactions on Fuzzy
(2008) Systems, 7, pp. 643-658, (1999)
[5] Yang, Y., Park, D. S., Huang, S., Rao, N.: Medical [15] Ensafi, P., Tizhoosh, H. R.,: Type II fuzzy image
image fusion via an effective wavelet based enhancement, in Lecture Notes in Computer
approach. EURASIP Journal on Advances in Signal Sciences, vol. 3656, M. Kamel and A. Campilho
Processing (2010) (Eds.), Springer, Berlin, Germany, pp. 159-166,
[6] Li, X., He, M., Roux, M.: Multifocus image fusion (2005)
based on redundant wavelet transform. IET Image [16] Chaira, T.: A rank ordered filter for medical image
Processing, 4, 283-293, (2010) edge enhancement and detection using intuitionistic
[7] Zadeh, L. A.: Fuzzy sets. Information Control, 8, fuzzy set. Applied Soft Computing, 12, 1259-1266,
338-353, (1965) (2012)
[8] Atanassov, K. T.: Intuitionistic fuzzy sets. Fuzzy [17] www.metapix.de/toolbox.htm
Sets and Systems, 20, 87-96, (1986) [18] Jagalingam, P., Hegde, A. V.: A Review of quality
[9] Mendel, J. M.: Uncertain rule-based fuzzy logic metrics for fused image, in Aquatic Procedia of
systems introduction and new directions, International Conference on Water Resources,
Eaglewood Cliffs, NJ: Prentice-Hall (2001) Coastal and Ocean Engineering (ICWRCOE)
(2015)
[19] www.med.harvard.edu/AANLIB/home.html

Potrebbero piacerti anche