Multimodal Medical Image Fusion Based

on Fuzzy Sets with Orthogonal

Teaching–Learning-Based Optimization

T. Tirupal, B. Chandra Mohan and S. Srinivas Kumar

Abstract The purpose of an image fusion for medical images is to associate a num-
ber of images gained from many bases to a solitary image appropriate for better
analysis. The vast majority of the best in class image fusing systems are based on
non-fuzzy sets, and the fused image so obtained lags with complementary infor-
mation. Fuzzy sets are strong-minded to be more appropriate for medical image
processing as more hesitations are considered compared with non-fuzzy sets. In this
paper, a procedure for efficiently fusing multimodal medical images is presented. In
the proposed method, images are initially converted into intuitionistic fuzzy images
(IFIs), and another target work called intuitionistic fuzzy entropy (IFE) is utilized
for membership and non-membership capacities to accomplish the finest estima-
tion of the bound. Next, the IFIs are compared using the fitness function, entropy.
Then, orthogonal teaching–learning-based optimization (OTLBO) is introduced to
optimize combination factors that change under teaching phase, and learner phase of
OTLBO. Finally, the fused image is achieved using optimal coefficients. Simulations
on several pairs of multimodal medical images are performed and matched with the
current fusion approaches. The dominance of the proposed technique is presented
and justified. Fused image quality is also verified with various quality metrics, such
as peak signal-to-noise ratio (PSNR), universal quality index (UQI), structural simi-
larity (SSIM), correlation coefficient (CC), entropy (E), spatial frequency (SF), edge
information preservation (QAB/F ), and standard deviation (SD).

Keywords Image fusion · Diagnosis · Fuzzy sets · IFI · OTLBO

Spatial frequency

T. Tirupal (B) · S. Srinivas Kumar

JNTUK, Kakinada, India
e-mail: tirutalari@gmail.com
S. Srinivas Kumar
e-mail: samay_ssk2@yahoo.com
B. Chandra Mohan
Bapatla, India
e-mail: chandrabhuma@gmail.com

© Springer Nature Singapore Pte Ltd. 2019 487

N. K. Verma and A. K. Ghosh (eds.), Computational Intelligence: Theories,
Applications and Future Directions—Volume II, Advances in Intelligent Systems
and Computing 799, https://doi.org/10.1007/978-981-13-1135-2_37
488 T. Tirupal et al.

1 Introduction

Multimodal medical image fusion (MMIF) [1] is the progression of bonding two mul-
timodal medical images to increase the quality of output image. In merging medical
image pairs, such as computed tomography–magnetic resonance imaging (CT–MRI),
magnetic resonance imaging–magnetic resonance angiography (MRI–MRA), X-
ray–vibro-acoustography (X-ray–VA) [2], MRI–PET (magnetic resonance imag-
ing—positron emission tomography) [3], and MRI–SPECT (magnetic resonance
imaging–single photon emission computed tomography), image fusion is gradually
achieving consequence in healthcare and modern medicine. These sets of images
are used for mining clinical facts that are complimentary in nature. For example,
CT image ambiances with a lesser amount of distortion and conveys details regard-
ing condensed structures like bones. MRI offers neurotic soft fleshy tissue evidence
and MRA intellects easily brain defects. X-ray identifies fractures and irregularities
in bone position while VA provides depth and width of a disease object. PET and
SPECT provide functional and metabolic information of the human brain. Hence, all
significant data cannot be obtained in a single image and henceforth MMIF is essen-
tially necessary to achieve all probable comprehensive data in a single combined
image called fused image.
Fuzzy sets play a substantial role in image handling to reduce the obscurity and
ambiguity existing in images. Ambiguity in fuzzy sets is taken as a membership
function which deceits the interim [0, 1] where 0 characterizes no membership and 1
characterizes full membership. Fuzzy set theory anticipated by Zadeh [4] in 1965 is
not able to directly model uncertainties. Atanassov [5] in 1986 presented an intuition-
istic fuzzy set (IFS), a general form of fuzzy set hypothesis. IFS takes two ambiguity
parameters—the membership degree and non-membership degree (due to the hesi-
tation degree). Real-time complications with indefinite understanding are resolved
effectively by the non-membership degree. In each period of image handling, numer-
ous vulnerabilities happen. These vulnerabilities are reduced by means of IFSs, and
the image is boosted in terms of contrast of the image.
Rao et al. [6] formerly advanced a universal optimization process called
teaching–learning-based optimization (TLBO). The TLBO shows nearly common
features of former evolutionary computation (EC) procedures which are a population-
based iterative learning process. In any case, TLBO analyzes for an optimum over
every learner attempting to accomplish the understanding of the teacher. TLBO
attains optimum outcomes and is preserved as the best-learned individual in the soci-
ety than through learners experiencing genomic processes like mutation, crossover,
and selection. TLBO is successfully applied to many real-world problems [7, 8] due
to its modest idea and high proficiency, and it emerged as a very pretty optimization
technique. In general, TLBO yields better results to the other EC methods.
In this paper, an orthogonal design process is included in the elementary TLBO
optimization. In this procedure, every student (learner) over the span of students in
the orthogonal design is isolated into a few partial vectors where each of the stu-
dents goes about as a factor. Orthogonal design, among all the different groupings, is
Multimodal Medical Image Fusion Based on Fuzzy Sets … 489

included to look through the finest scales. An orthogonal design process [9] demon-
strates with orthogonal array (OA) and factor, however, an illustrative arrangement
of courses of action for research to achieve respectable measures. The array is called
orthogonal in light of the fact that all sections are assessed freely of each other, and
the primary result of one factor does not inconvenience the evaluation of the funda-
mental impact of another factor. Experimental results proved that this technique is
superior to the performance of the traditional EAs. To make orthogonal design faster
and robust, TLBO is applied to an orthogonal design which is called OTLBO [10]
and is implemented in this paper.

2 Preliminaries

Medical images are poorly illuminated, contain many uncertainties in the form of
noise, having vague boundaries, gray levels are overlapped, invisible blood ves-
sels, and difficult to extract objects from the image. Fuzzy sets in image processing
increase contrast, flat the areas of interest, and refine the boundaries and fine erections
of the image.

2.1 Fuzzy Sets

Fuzzy set I is an ordinary fuzzy set, characterized by a determinate regular Y 

{y1 , y2 , y3 , . . . , yn } factually as

I  {(y, μI (y))|y ∈ Y } (1)

The capacity μI (y) : Y → [0, 1] speaks to the level of belongingness or the

membership capacity of a component y in the limited set Y and 1 − μI (y) represents
the measure of non-belongingness or non-membership function.
Atanassov presented a novel fuzzy set the so-called intuitionistic fuzzy set that
takes both membership function μ(y) and the non-membership function v(y) into
contemplation holding μI (y) → [0, 1], vI (y) → [0, 1]. An intuitionistic fuzzy set
I in a determinate set Y holding the situation 0 ≤ μI (y) + vI (y) ≤ 1 is statistically
denoted as

I  {(y, μI (y), vI (y))|y ∈ Y } (2)

Atanassov also introduced a novel constraint πI (y) called the hesitation degree or
intuitionistic fuzzy index, rising due to the absence of information or individual fault
in conveying the membership degree. An IFS, I in a finite set Y with πI (y) + μI (y) +
vI (y)  1, and the introduction of the hesitation degree may be represented as
490 T. Tirupal et al.

I  {(y, μI (y), vI (y), πI (y))|y ∈ Y } (3)

where 0 ≤ πI (y) ≤ 1, for each y ∈ Y .

In this paper, an innovative objective function called IFE is introduced which
plays a major part in image handling. Fuzzy entropy is calculated as in [11] and is
utilized to improve the developed algorithm.


n
IFE(Z; α)  πZ (yi ) exp(1 − πZ (yi )) (4)
i1

where πZ (yi )  1 − (μZ (yi ) + vZ (yi )) denotes the hesitation degree, μZ (yi ) denotes
the membership degree, and vZ (yi ) denotes the non-membership degree.
IFE utilizes Eq. (4) for designing and takes α standards going [0.1–1]. The extreme
entropy standards are calculated by α optimization.

αopt  max(IFE(Z; α)) (5)

The membership degrees of the IFIs are found by utilizing the known α values,
and IFI is obtained as

FIFI  {(y, μZ (y; α), vZ (y; α), πZ (y; α))|y ∈ Y } (6)

2.2 TLBO

TLBO is a universal optimization technique which depends on population. It builds

on the result of supervision of an educator based on the yield of students in a class.
TLBO procedure imitates the teaching–learning capability of instructor and students
in the teaching space. Based on its operation, this algorithm is classified into two
portions: one is “teaching phase,” where what a student learns from the teacher
is explained. And the other is “learner phase,” where what a student gains from
interacting with other students is illustrated.
Learning from the teacher is termed as teacher phase. The parameter “mean”
is considered here. The algorithm shifts the “mean” of learners toward its teacher
in teacher phase by moving forward. Learning through collaboration with others is
termed as learner phase, and mutual interface inclines to surge the understanding of
the student. The arbitrary communication increases the learner’s knowledge, and for
one learner the additional learner is arbitrarily selected. The TLBO algorithm [6] is:

Step 1: Set the quantity of learners and iterations.

Step 2: Compute the mean of each variable and find the finest solution.
Step 3: Based on the best solution adjust the solution using
Multimodal Medical Image Fusion Based on Fuzzy Sets … 491

Xnew  Xold + ri (Xold − Tf Me ) (7)

Step 4: Compare the solutions, if the new solution is not superior to present, then
retains the former solution.
Step 5: Select two solutions randomly, on the off chance that the new arrangement
is superior to existing. At that point, another arrangement is discovered
utilizing.

⎧
⎨ X l + rand ∗ (X l − X l ) if (f (X l ) < f (X l ))
m m n (i) (r)
l
Xnew(i)  (8)
⎩ X l + rand ∗ (X l − X l ) otherwise
m n m

Step 6: The obtained new solution is superior to the previous solution and is taken
as the best solution otherwise the previous solution is the best solution.
Step 7: After satisfying termination criteria, the final optimal coefficients are
obtained. After completing the maximum number of iterations the algo-
rithm is terminated.

2.3 Entropy

The degree of information content of an image represents entropy. Higher entropy

signifies that an image contains maximum information content. It is defined as [12]:


L
H − hIf (i) log2 hIf (i) (9)
i0

where hIf (i) speaks to the standardized histogram of the combined image and L means
the measure of recurrence receptacles in the histogram.

3 The Proposed Image Fusion Based on OTLBO

Orthogonal design, developed as a logical intends to consider multifactor and mul-

tilevel issues. Orthogonal design abstracts to process orthogonal gathering L of M
lines, where each line implies a strategy to be computed. An orthogonal design-based
administrator is engaged with the standard TLBO to accomplish a more exact arrange-
ment. OTLBO is proposed for improving the current population and to enrich the
fitness function when the learner gets updated. The flowchart for the OTLBO is pre-
sented in Fig. 1. The following is the procedure used for fusing multimodal medical
images, and the schematic of the developed medical image combination arrangement
is shown in Fig. 2.
492 T. Tirupal et al.

Fig. 1 Flowchart for orthogonal teaching–learning-based optimization (OTLBO)

Multimodal Medical Image Fusion Based on Fuzzy Sets … 493

Input image, Input image, I 2

Fuzzification Fuzzification

Calculate optimum Calculate optimum

value ( α ) value ( α )

Find Fuzzified Find Fuzzified

Intuitionistic Fuzzy Intuitionistic Fuzzy
Image (IFI) Image (IFI)

Decomposition Decomposition

Compute Entropy Compute Entropy

of image blocks of image blocks

OTLBO

Generate fused image using

optimal co-efficients

Noise filtered fused image

Fig. 2 Schematic of the proposed medical image combination scheme

494 T. Tirupal et al.

(1) Read two input registered multimodal medical images.

(2) Utilizing below Eq. (10), the fuzzified first image I1 of size M × N is
Iij1 − lmin
μZ1 (Iij1 )  (10)
lmax − lmin

where Iij1 characterizes the dim level of the primary image and assortments from
0 to L − 1 (L is the outrageous dark-level esteem). lmin and lmax speak to the
base and most extreme dark-level estimations of the primary image.
(3) Calculate the optimum value of α using entropy Eqs. (4) and (5) for the first
image and this α varies for different input images.
(4) With the optimum value of α, find fuzzified IFI for the input image I1 using
beneath equations and indicate as IY 1 .

μIFI 1 (Iij1 )  μZ1 (Iij1 ) (11)

where μIFI 1 is the membership degree of the first input image.

vIFI 1 (Iij1 ; α)  (1 − μIFI 1 (Iij1 ; α)α )1/ α (12)

where vIFI 1 (Iij1 ; α) is the optimized non-membership degree of the first image.

πIFI 1 (Iij1 ; α)  1 − μIFI 1 (Iij1 ; α) − vIFI 1 (Iij1 ; α) (13)

where πIFI 1 (Iij1 ; α) is the optimized hesitation degree of the first image.
 
IY 1  (Iij1 , μIFI 1 (Iij1 ; α), vIFI 1 (Iij1 ; α), πIFI 1 (Iij1 ; α)) (14)

(5) Repeat the above procedure from step 2 to step 4 for the second input image, I2
to find IY 2 .
 
IY 2  (Iij2 , μIFI 2 (Iij2 ; α), vIFI 2 (Iij2 ; α), πIFI 2 (Iij2 ; α)) (15)

where μIFI 2 (Iij2 ; α), vIFI 2 (Iij2 ; α), πIFI 2 (Iij2 ; α)) are optimized membership,
non-membership and hesitation degrees of the second image, respectively.
(6) Images IY 1 and IY 2 are decayed into m × n windows, and in this algorithm, 5 × 5
is the window size taken and then the entropy of each block of two images is
computed separately.
(7) Then, perform OTLBO algorithm to find optimal coefficients.
(8) Lastly, utilizing these optimal coefficients, the final fused image is attained, and
the fused image is filtered for noise removal.
Multimodal Medical Image Fusion Based on Fuzzy Sets … 495

4 Implementation and Experimental Results

The execution of the proposed technique on various sets of multimodal medicinal

images is presented and is compared with Jin and Wang [8] method. Here, all the
pairs of multimodal medical images are registered and are available in [13–15]. The
algorithm developed and implemented in MATLAB 8.1 [16].
The first example shown in Fig. 3-1a, b discourses CT and MRI images with
a size of 256 × 256. The CT image provides information about bones and hard
tissues while the MRI image gives soft tissue information. Fusing these two images
provide abundant information in a single image, which helps to diagnose a disease.
Figure 3-1d is the merged image of the proposed system, which visually proves that
the image is of higher contrast and luminance than the fused image of the existing
procedure [8]. Table 1 gives the independent evaluation of fusion results for MRI and
CT images. Further comparisons of the results are performed using objective criteria
[12] such as PSNR measured in dB (decibels), UQI, SSIM, CC, E is measured in
bits per pixel, SF is measured in cycles per millimeter, QAB/F , and SD is measured
in pixel intensity. PSNR (65.18) that is large for the proposed method than PSNR
(61.23) of the proposed method. SF, SD, bits per pixel, UQI, SSIM, CC are large
compared to the existing method. It is observed from the figure that the proposed
method provides better performance regarding contrast, luminance, and visibility of
the fused image.
The second case addresses a T1-weighted MR picture and the MRA picture with
white structures as some ailment appeared in Fig. 3-2a, b. The combination of these
two pictures gives correlative data inside a solitary picture, which is useful for better
therapeutic diagnostics. Table 1 gives the target assessment of combination outcomes
for T1-weighted MR and the MRA picture. The proposed method has large PSNR of
69.73, and correlation coefficient is 0.892. Figure 3-2d provides the fused image for
the proposed method with high spatial resolution having a pixel intensity of 79.44.
The third illustration tends to MRI and PET pictures appeared in Fig. 3-3a, b.
Table 1 gives the objective evaluation of fusion results for MRI and PET images.
The fused image of the developed algorithm has PSNR of 69.93, UQI of 0.74, SSIM
of 0.824, CC of 0.863, bits per pixel of 6.635, SF of 44.24, edge information of
0.767, and pixel density of 70.64. The fused image gives detailed information about
the size of the disease present, which is not visible in other fused images and which
is better diagnosed by doctors.
The fourth course of action of pictures is MRI and SPECT brain tumor pictures
appeared in Fig. 3-4a, b. It is seen from the Fig. 3-4d that the combined picture of
the proposed strategy contained by the tumor is evidently enhanced when contrasted
with the existing technique. Table 1 gives the objective evaluation of fusion results
for MRI and SPET images. The fused image has a pixel intensity of 52.64, and edge
information transferred is 0.673 with large PSNR of 67.49. It is clearly observed
that high accuracy is attained by the developed algorithm. Efficiently it abstracts the
complementary and redundant information from MRI and SPECT images thereby
producing an extremely consistent fused output image for detection of the tumor.
 496

Table 1 Performance comparison of fused images of TLBO algorithm [8] and the proposed OTLBO algorithm
Modality pair Fusion PSNR UQI SSIM CC E SF QAB/F SD
technique
CT–MRI TLBO [8] 61.23 0.762 0.912 0.913 6.294 21.442 0.711 46.85
OTLBO 65.18 0.798 0.998 0.948 6.703 27.29 0.859 90.76
MR–MRA TLBO [8] 64.33 0.667 0.783 0.814 5.926 25.73 0.631 49.45
OTLBO 69.73 0.823 0.846 0.892 6.968 41.36 0.751 79.44
MRI–PET TLBO [8] 62.16 0.776 0.801 0.835 6.273 43.18 0.625 50.37
OTLBO 69.93 0.784 0.824 0.863 6.635 44.24 0.767 70.64
MRI–SPECT TLBO [8] 65.78 0.48 0.841 0.863 5.595 35.63 0.408 21.57
OTLBO 67.49 0.771 0.935 0.875 6.618 43.56 0.673 52.64
X-ray–VA TLBO [8] 61.28 0.714 0.813 0.837 6.796 39.51 0.775 39.02
OTLBO 67.69 0.882 0.915 0.872 7.185 53.64 0.872 48.82
PET–CT TLBO [8] 65.75 0.781 0.889 0.874 6.268 35.49 0.734 43.08
OTLBO 67.63 0.855 0.945 0.887 6.765 44.65 0.758 83.59
T. Tirupal et al.
Multimodal Medical Image Fusion Based on Fuzzy Sets … 497

Fused image by Fused image by

S.No Modality 1 Modality 2
TLBO [8] Proposed OTLBO

1.

(a) CT (b) MRI (c) (d)

2.

(a) MR (b) MRA (c) (d)

3.

(a) MRI (b) PET (c) (d)

4.

(a) MRI (b) SPECT (c) (d)

5.

(a) Xray (b) VA (c) (d)

6.

(a) PET (b) CT (c) (d)

Fig. 3 Fusion results of multimodal medical images using TLBO algorithm [8] and the proposed
OTLBO algorithm

The fifth example tends to X-ray and vibro-acoustography VA pictures as shown

in Fig. 3-5a, b. Table 1 gives the target assessment of combination outcomes for X-ray
and VA pictures. Thus, by fusing two images, the diagnostic values of two modality
images are more than individual images. The fused image has PSNR of 67.69 larger
498 T. Tirupal et al.

than that of existing method. It has a high average number of bits per pixel of 7.185
for the proposed method which gives more data content than the current technique.
The last case speaks to multimodality images, for example, the PET and CT
image appeared in Fig. 3-6a, b. The PET picture shows the cerebrum work, has a low
spatial assurance. A CT picture gives thick structures like bones and embeds with less
bending; it cannot recognize practical changes. Intertwining these two pictures, we
get both practical information and extra spatial highlights without spatial mutilation
and this is found in Fig. 3-6d for the developed algorithm.

5 Conclusions

This paper presents the optimal block size for the fusion of multimodal medical
images by a global optimization procedure called OTLBO. The optimum solution
is achieved by an efficient search of a large sample space using orthogonal design.
Here, for a fixed number of iterations, algorithm is run and optimal coefficients
are obtained. So, finally the final fused image is acquired. The obtained results are
compared with existing TLBO with respect to different quantitative measures. From
the result analysis, it is obvious that OTLBO outstrips the existing method in terms
of accuracy. We can achieve that OTLBO is an intense technique in advancing the
blocks that are discrete and multimodal in giving quality ideal outcomes.

References

1. Yang, Y., Park, D.S., Huang, S., Rao, N.: Medical image fusion via an effective wavelet based
approach. EURASIP J. Adv. Signal Proc. (2010)
2. Gholam, H.H., Alizad, A., Fatemi, M.: Integration of Vibro-acoustography imaging modality
with the traditional mammography. Int. J. Biomed. Imaging. Hindawi Publishing Corporation
(2007)
3. Baum, K.G., Raerty, K., Helguera, M., Schmidt, E.: Investigation of PET/MRI image fusion
schemes for enhanced breast cancer diagnosis. In: Proceedings of IEEE Seventh Symposium
Conference on Nuclear Science (NSS) (2007)
4. Zadeh, L.A.: Fuzzy sets. Information. Control 8, 338–353 (1965)
5. Atanassov, K.T.: Intuitionistic fuzzy sets. Fuzzy Sets Syst. 20, 87–96 (1986)
6. Rao, R.V., Savsani, V.J., Vakharia, D.P.: Teaching-learning-based optimization: a novel method
for constrained mechanical design optimization problems. Comput. Aided Des. 43(3), 303–315
(2011)
7. Vedat, T.: Design of planar steel frames using teaching-learning based optimization. Eng. Struct.
34, 225–232 (2012)
8. Jin, H., Wang, Y.: A fusion method for visible and infrared images based on contrast pyramid
with teaching learning based optimization. Infrared Phys. Technol. 64, 134–142 (2014)
9. Fang, K.T., Ma, C.X.: Orthogonal and Uniform Design. Science Press, Beijing (2001)
10. Suresh, C.S.: A teaching learning based optimization based on orthogonal design for solving
global optimization problems. Springer Plus 2, 130 (2013)
11. Chaira, T.: A novel intuitionistic fuzzy c means clustering algorithm and its application to
medical images. Appl. Soft Comput. 11(2), 1711–1717 (2011)
Multimodal Medical Image Fusion Based on Fuzzy Sets … 499

12. Jagalingam, P., Hegde, A.V.: A review of quality metrics for fused image. In: Aquatic Procedia
of International Conference on Water Resources, Coastal and Ocean Engineering (ICWRCOE)
(2015)
13. Tirupal, T., Mohan, B.C., Kumar, S.S.: Multimodal medical image fusion based on sugeno’s
intuitionistic fuzzy sets. ETRI J. 39(2), 173–180 (2017)
14. www.med.harvard.edu/aanlib
15. www.metapix.de
16. www.mathworks.in