MODIFIED FCN BASED ON DETECTION AND
SEGMENTATION OF SKIN LESION
C Durga Devi M.E,1
Assistant Professor, Department of Electronics and
Communication Engineering
DMI College of Engineering
Palanchur, Chennai, India
Abstract— Automatic skin lesion segmentation in
dermoscopic images is a challenging task due to the low
contrast between lesion and the surrounding skin, the irregular
and fuzzy lesion borders. A fully automated method is used for
skin lesion segmentation by leveraging the discriminative
power of a 19 layer deep convolutional neural network. Inorder
to update the result of convolutional neural network, fully
convolutional network is implemented. Fully convolutional
network is used for automatically detecting efficient output at
any situation. A loss function is based on Jaccard distance is
used to eliminate the need of sample re-weighting. Jaccard
distance is the type of loss function which is used as strong
imbalance between the foreground and background pixels.
Fully convolutional network need minimum pre and post
processing, which allow its adoption in variety of medical
image segmentation task.
Keywords— loss function, Convolutional Neural Network
I. INTRODUCTION
A skin lesion is a part of the skin that has an abnormal
growth or appearance compared to the skin around it. A skin
lesion is generally called an eruption. Eruptions are divided
into primary lesions, which occur in normal skin, and
secondary lesions, which are caused secondarily by other
eruptions.
Fig 1.1 Skin Lesion
Computational systems for skin cancer diagnosis have
been proposed in order to aid dermatologists in early
assessment of such diseases, or even to monitor pigmented
skin lesions. Dermoscopy images are widely applied for
image analysis of pigmented skin lesions. Such images may
be acquired from dermatoscope devices or specific cameras
to provide a better visualization of the pigmentation pattern
on the skin surface.
Primary Skin Lesion are abnormal condition present at
birth or over a person’s lifetime. An eruption that occurs in
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Sneha Thanvi2
PG Scholar, Department of Electronics and Communication
Engineering
DMI College of Engineering
Palanchur, Chennai, India
normal skin without any pre existing eruptions is called a
primary lesion.
These include patches, where the only change is color,
papules, nodules and tumors, which are elevated blisters,
cysts and pustules, which contain serum, keratinized
substances, pus etc and urticaria which is temporarily
elevated. In the case of malignant lesions, i.e., skin cancer,
the cells split quickly, and may invade other parts of the
body. Indeed, these cells do not die, as generally occurs with
normal cells. Types of primary skin lesions are:
Papule:
A papule is a localized elevated lesion of 10 mm or less
in diameter with a hemispheric or flat shape. It is
characterized by a surface that can be smooth, eroded,
ulcerative, hyperkeratosis or crusted. It may be caused by a
proliferative or inflammatory change in the epidermis, or by
dermal edema.
Leukoderma:
Leukoderma is a white patch produced by
Depigmentation or local anemia. Depigmentation is caused
by abnormal production of melanin’s such as in vitiligo
vulgarisms. Nevus anemic us causes local anemia leading to
leukoderma. Leukoderma in the periphery of an eruption is
called a white halo.
Nodule and Tumor:
A nodule is a localized lesion that appears as a papule
with a diameter of 10 to 20 mm. It can have various causes,
such as tumor formation, granulomatous change,
inflammation or edema. An intensely proliferative nodule
with an elevation of 30 mm or more in diameter is called a
tumor.
Blister:
A blister is a skin elevation of 5 mm or more in diameter
enclosed by a membrane and containing transparent fluid that
is mainly plasma and cellular material. A small blister with a
diameter of less than 5 mm is called a vesicle. A hemorrhagic
blister containing serum mixed with blood is referred to as a
bloody bulla.
Pustule:
A pustule is a yellowish blister with purulent contents
Pigmented Macule:
A pigmented Macule is a patch of brown, yellow, blue or
other color, depending on the deposited substance. It is most
commonly caused by deposition of melanin, the next most
common causes being deposition of hemosiderin, carotin,
bile pigment, drugs or other foreign substances (e.g., metal,
charcoal).
Cyst:
A cyst is a closed tumorous lesion covered by a
membranous lining, which does not always elevate above the
skin. The covering consists of epithelial tissue or connective
tissue containing keratinous substances
Wheal and Urticaria:
Urticaria is localized edema that disappears in a short
period of time
Erythema:
Erythema is patchy redness produced by vasodilation and
hyperaemia in the dermal papillae and the sub papillary layer.
In Erythema, although the blood volume increases in the
dermal blood vessels, there is no blood leakage into the extra
vascular dermis. Thus the bloody color fades under the
pressure of a glass plate
Purpura:
Purpura is purple to bright red haemorrhaging in the
skin . The color of the blood does not fade in diascopy,
because haemorrhage causes blood leakage into the dermis,
which distinguishes it from Erythema. A Purpura of 2 mm or
less in diameter is called a petechia.
A Purpura that is larger than a petechia is called
ecchymosis, and an even larger elevated Purpura is called a
hematoma. The red of a Purpura is fairly bright shortly after
bleeding begins
Secondary Skin Lesion
Secondary skin lesions are the result of irritated or
manipulated primary skin lesions. For example, if someone
scratches a mole until it bleeds, the resulting lesion, a crust, is
now a secondary skin lesion. Types of secondary skin lesions
are:
Skin atrophy occurs when skin becomes thin or has a
smooth or finely wrinkled surface. The secretary function is
reduced and the skin surface dies. Aging leads to skin
atrophy including subcutaneous lipoatrophy, striaeatrophicae
caused by steroids and kraurosis vulvae and macular atrophy.
Crust is created when dried blood forms over a scratched
and irritated skin lesion.
Ulcer are typically caused by a bacterial infection or
physical trauma. They are often accompanied by poor
circulation.
II. Ease of Use
1.1 SPECIAL LESIONS
Some primary lesions, limited to a few skin diseases, can
be called specialized lesions.
Comedones or blackheads are plugs of whitish or
blackish sebaceous and keratinous material lodged in the
pilosebaceous follicle, usually seen on the face, the chest, or
the back, rarely on the upper part of the arms. Example: acne.
Milia are whitish nodules, 1 to 2 mm in diameter, that have
no visible opening onto the skin surface. Examples: in healed
burn or superficial traumatic sites, healed bullous disease
sites, or newborns.
Telangiectasias are dilated superficial blood vessels.
Examples: spider hemangioma, chronic radio dermatitis.
Burrows are very small and short (in scabies) or tortuous
and long (in creeping eruption) tunnels in the epidermis.
1.2 TEXTURE AND COLOR
Some skin lesions have visible or palpable texture that
suggests a diagnosis. Verrucous lesions have an irregular,
pebbly, or rough surface. Examples include warts and
seborrhea Kerasotes. Lichenification is thickening of the skin
with accentuation of normal skin markings; it result from
repeated scratching or rubbing. Indurations or deep
thickening of the skin, can result from edema, inflammation,
or infiltration, including by cancer. Indurate skin has a hard,
resistant feeling. Indurations is characteristic of panniculitis,
some skin infections, and cutaneous metastatic cancers.
Umbilicated lesions have a central indentation and are
usually viral. Examples include molluscum contagiosum and
herpes simplex. Xanthomas, which are yellowish, waxy
lesions, may be idiopathic or may occur in patients who have
lipid disorders. Some skin lesions have color that suggests a
diagnosis.
Red skin can result from many different inflammatory or
infectious diseases. Cutaneous tumors are often pink or red.
Superficial vascular lesions such as port-wine stains may
appear red.
Orange skin is most often seen in hypercarotenemia, a
usually benign condition of carotene deposition after excess
dietary ingestion of betacarotene.
Yellow skin is typical of jaundice, xanthelasmas and
xanthomas and pseudoxanthoma elasticum.
Green fingernails suggest Pseudomonas aeruginosa
infection.
Violet skin may result from cutaneous hemorrhage or
vasculitis. Vascular lesions or tumors, such as Kaposi
sarcoma and hemangiomas, can appear purple. A lilac color
of the eyelids or heliotrope eruption is characteristic of
dermatomyositis.
Shades of blue, silver, and gray can result from
deposition of drugs or metals in the skin, including
minocycline, amiodarone, and silver (argyria Ischemic skin
appears purple to gray in color. Deep dermal nevi appear
blue.
Black skin lesions may be melanocytic, including nevi
and melanoma. Black eschars are collections of dead skin
that can arise from infarction, which may be caused by
infection.
1.3 DIAGNOSIS
In order to diagnose a skin lesion, a dermatologist or
doctor will conduct a full physical exam. This will include
observing the skin lesion and asking for a full account of all
symptoms. To confirm a diagnosis, they make take skin
samples, perform a biopsy of the affected area, or take a
swab from the lesion to send to a lab. Treatment is based on
the underlying cause or causes of skin lesions. A doctor will
take into account the type of lesion, personal health history,
and any treatments previously attempted. The skin lesion
classification systems should have a high performance,
considering that they will be used to assist in dermatological
diagnosis. The evaluation and improvement of the
performance of the classifier is an essential characteristic of
pattern recognition. A relevant problem that affects the
performance of classifiers is the definition of which features
are meaningful, in order to adequately represent the classes.
Therefore, the application of several descriptions may be
necessary, considering the large quantity of features present
in images.
First-line treatments are often topical medications to help
treat the inflammation and protect the affected area. Topical
medication can also provide mild symptom relief to stop
pain, itching, or burning caused by the skin lesion. If skin
lesions are the result of a systemic infection, such as shingles
or chickenpox, then prescribed oral medications helps to ease
the symptoms of the disease, including skin lesions. Skin
lesions that are infected are typically lanced and drained to
provide treatment and relief. Suspicious-looking moles that
have been changing over time may need to be removed
surgically. A type of vascular birthmark called hemangioma
results from malformed blood vessels. Laser surgery is often
used to remove this type of birthmark.
1.4 CAUSES OF SKIN LESION
The most common cause of a skin lesion is an infection
on or in the skin. One example is a wart. The wart virus is
passed from one person to another through direct skin-to-
skin contact. The herpes simplex virus, which causes both
cold sores and genital herpes, is also passed through direct
contact. A systemic infection such as chickenpox or
shingles, can cause skin lesions all over your body. MRSA
and cellulites are two potentially life-threatening infections
that involve skin lesions. Some skin lesions are hereditary,
such as moles and freckles. Birthmarks are lesions that exist
at the time of birth. Others can be the result of an allergic
reaction, such as allergic eczema and contact dermatitis.
Some conditions, like poor circulation or diabetes, cause
skin sensitivity that can lead to lesions.
III. Background of the Study
Harald Ganster ,Axel Pinz, Reinhard Röhrer, Ernst
Wildling, Michael Binder, and Harald Kittler proposed the
automated method for the recognition of melanoma. A
system for the computerized analysis of images obtained
from ELM has been developed to enhance the early
recognition of malignant melanoma. As an initial step, the
binary mask of the skin lesion is determined by several basic
segmentationn algorithms together with a fusion strategy. A
set of features containing shape and radiometric features as
well as local and global parameters is calculated to describe
the malignancy of a lesion. Significant features are then
selected from this set by application of statistical feature
subset selection methods. The final KNN classification
delivers a sensitivity of 87% with a specificity of 92%. a
fully connected feed-forward network with 21 input units
that represent 21 skin lesion features one hidden layer with
four hidden units, and one output unit for the categorization
into benign or malignant class. After training and application
of node pruning the hidden layer could be reduced to three
units with a small amount of connections to the remaining six
input units. The actual classification is the final step in
automated analysis of skin lesions.
Francesco Peruch, Federica Bogo, Michele Bonazza,
Vincenzo-Maria Cappelleri and Enoch Peserico(2014)
proposed a new technique for melanocytic lesion
segmentation, Micking Expert MEDS and extensive tests of
its accuracy, speed and robustness. MEDS combines a
thresholding scheme reproducing the cognitive process of
dermatologists with a number of optimizations that may be of
independent interest. MEDS is simple, with a single
parameter tuning its “tightness”. It is in the order of
magnitude or more over state-of-the-art techniques. And it is
extremely accurate very experienced dermatologists disagree
with its segmentations less extremely fast, segmenting
medium resolution images in a fraction of a second even with
the modest computational resources of a cell phone an
improvement of than they disagree with the segmentations of
state-of the-art techniques, and in fact less than they disagree
with the segmentations of dermatologists of moderate
experience.
Amir Reza Sadri, Maryam Zekri, Saeed Sadri, Niloofar
Gheissari, Mojgan Mokhtari, and Farzaneh Kolahdouzan
introduces a new approach for the segmentation of skin
lesions in dermoscopic images based on WN. The WN is a
member of fixed grid WNs that is formed with no need of
training. In this Wavelet Network, after formation of wavelet
lattice, determining shift and scale parameters of wavelets
with two screening stage and selecting effective wavelets,
OLS algorithm is used to calculate the network weights and
to optimize the network structure. The existence of two
stages of screening, increases globality of the wavelet lattice
and provides a better estimation of the function especially for
larger scales. R, G, and B values of a dermoscopy image are
considered as the network inputs and the network structure
formation. Then the image is segmented and the skin lesions
exact boundary is determined accordingly. The segmentation
algorithm were applied to 30 dermoscopic images and
evaluated with eleven different metrics, using the
segmentation result obtained by a skilled pathologist as the
ground truth. Experimental results show that this method acts
more effectively in comparison with some modern
techniques which have been successfully used in many
medical imaging problems.
Hoo-Chang Shin, Member, Holger R. Roth, Mingchen
Gao, Le Lu, Ziyue Xu, Isabella Nogues, Jianhua Yao, Daniel
Mollura, Ronald M. Summers proposed a method based on
deep CNN. Remarkable progress has been made in image
recognition, primarily due to the availability of large-scale
annotated datasets and the revival of deep CNN. CNNs
enable learning data-driven, highly representative, layered
hierarchical image features from sufficient training data.
However, obtaining datasets as comprehensivelyannotated as
ImageNet in the medical imaging domain remains a
challenge. There are currently three major techniques that
successfully employ CNNs to medical image classification
training the CNN from scratch, using off-the-shelf pretrained
CNN features, and conducting unsupervised CNN pre-
training with supervised fine-tuning. Another effective
method is transfer learning, i.e., fine tuning CNN models.
Two specific CAD problems, namely thoraco-abdominal
lymph node detection and ILD classification are used. This
method achieves the state-of-the-art performance on the
mediastinal LN detection, with 85% sensitivity at 3 false
positive per patient, and report the first five-fold cross-
validation classification results on predicting axial CT slices
with ILD categories. Extensive empirical evaluation, CNN
model analysis and valuable insights can be extended to the
design of high performance CAD systems for other medical
imaging tasks.
Lequan Yu, Student Member, Hao Chen, Qi Dou, Jing
Qin, and Pheng-Ann Heng proposed a method for melanoma
recognition using residual networks. Automated melanoma
recognition in dermoscopy images is a very challenging task
due to the low contrast of skin lesions, the huge intra class
variation of melanomas, the high degree of visual similarity
between melanoma and non-melanoma lesions, and the
existence of many artifacts in the image. In order to meet
these challenges, a novel method for melanoma recognition
by leveraging very deep CNNs is used. Compared with
existing methods employing either low-level hand-crafted
features or CNNs with shallower architectures, substantially
deeper networks can acquire richer and more discriminative
features for more accurate recognition. To take full advantage
of very deep networks, a set of schemes is used to ensure
effective training and learning under limited training data.
First, the residual learning is applied to cope with the
degradation and overfitting problems when a network goes
deeper. This technique can ensure that the networks benefit
from the performance gains achieved by increasing network
depth. Then, a FCRN is constructed for accurate skin lesion
segmentation, and further enhance its capability by
incorporating a multi-scale contextual information
integration scheme. Finally, seamlessly integrate the
proposed FCRN and other very deep residual networks to
form a two-stage framework. This framework enables the
classification network to extract more representative and
specific features based on segmented results instead of the
whole dermoscopy images, further alleviating the
insufficiency of training data. This framework is extensively
evaluated on ISBI 2016 Skin Lesion Analysis Towards
Melanoma Detection Challenge dataset. Experimental results
demonstrate the significant performance gains of the
proposed framework, ranking the first in classificationand the
second in segmentation among 25 teams and 28 teams,
respectively. This method corroborates that very deep CNNs
with effective training mechanisms can be employed to solve
complicated medical image analysis tasks, even withlimited
training data.
Sergio Pereira, Adriano Pinto, Victor Alves, and Carlos A.
Silva used a CNN method for the segmentation of brain
tumors. Among brain tumors, gliomas are the most common
and aggressive, leading to a very short life expectancy in
their highest grade. Thus, treatment planning is a key stage to
improve the quality of life of oncological patients. MRI is a
widely used imaging technique to assess these tumors, but
the large amount of data produced by MRI prevents manual
segmentation in a reasonable time, limiting the use of precise
quantitative measurements in the clinical practice. So,
automatic and reliable segmentation methods are required;
however, the large spatial and structural variability among
brain tumors make automatic segmentation a challenging
problem. In this method, an automatic segmentation method
is used based on CNN exploring small 3*3 kernels. The use
of small kernels allows designing a deeper architecture,
besides having a positive effect against overfitting, given the
fewer number of weights in the network. This method makes
the use of intensity normalization as a pre-processing step,
which though not common in CNN based segmentation
methods, proved together with data augmentation to be very
effective for brain tumor segmentation in MRI images. This
method was validated in the Brain Tumor Segmentation
Challenge 2013 database obtaining simultaneously the first
position for the complete, core, and enhancing regions in
Dice Similarity coefficient metric for the Challenge data set.
Also, it obtained the overall first position by the online
evaluation platform.
Shun Miao, Member, Z. Jane Wang, Senior Member, and
Rui Liao, proposed a CNN regression approach to address
the two major limitations of existing intensity-based 2-D/3-D
registration technology: 1) slow computation and 2) small
capture range. Different from optimization-based methods,
which iteratively optimize the transformation parameters
over a scalar-valued metric function representing the quality
of the registration, the proposed method exploits the
information embedded in the appearances of the digitally
reconstructed radiograph and Xray images, and employs
CNN regressors to directly estimatethe transformation
parameters. An automatic feature extraction step is
introduced to calculate 3-D pose-indexed features that are
sensitive to the variables to be regressed while robust to other
factors. The CNN regressors are then trained for local zones
and applied in a hierarchical manner to break down the
complex regression task into multiple simpler sub-tasks that
can be learned separately. There are currently three major
techniques that successfully employ CNNs to medical image
classification: training the “CNN from scratch” ,using “off-
the shelf CNN” features as complementary information
channels to exist hand crafted image features, performing
unsupervised pre training on natural or medical images and
fine-tuning on medical target images using CNN or other
types of deep learning models. Weight sharing is further
more employed in the CNN regression model to reduce the
memory footprint. This approach has been quantitatively
evaluated on three potential clinical applications,
demonstrating its significantly advantage in providing highly
accurate real-time 2-D/3-D registration with a significantly
enlarged capture range when compared to intensity based
methods.
Qi Dou, Hao Chen, Lequan Yu, Lei Zhao, Jing Qin,
Defeng Wang, Vincent CT Mok, Lin Shi and Pheng-Ann
Heng proposed a novel automatic method to detect CMBs
from magnetic resonance images by exploiting the 3D CNN.
CMBs are small haemorrhages nearby blood vessels. They
are recognized as important diagnostic biomarkers for many
cerebro vascular diseases and cognitive dysfunctions. In
current clinical routine, CMBs are manually labelled by
radiologists but this procedure is laborious, time consuming
and error prone. Compared with previous methods that
employed either low-level hand-crafted descriptors or 2D
CNNs, this method can take full advantage of spatial
contextual information in MR volumes to extract more
representative high-level features for CMBs, and hence
achieve a much better detection accuracy. To further improve
the detection performance while reducing the computational
cost, a cascaded framework is used under 3D CNNs for the
task of CMB detection. First, exploit a 3D FCN strategy to
retrieve the candidates with high probabilities of being
CMBs, and then apply a well trained 3D CNN discrimination
model to distinguish CMBs from hard mimics.Compared
with traditional sliding window strategy, the proposed 3D
FCN strategy can remove massive redundant computations
and dramatically speed up the detection process. A large
dataset is applied with 320 volumetric MR scansand
performed extensive experiments to validate this method,
which achieved a high sensitivity of 93.16% with an average
number of 2.74 false positives per subject, outperforming
previous methods using low-level descriptors or 2D CNNs The figure 3.1 shows the block diagram of
by a significant margin. This method, in principle, can be Methodology
adapted to other biomarker detection tasks from volumetric
medical data.
Zhennan Yan, Yiqiang Zhan, Zhigang Peng, Shu Liao,
Yoshihisa Shinagawa, Shaoting Zhang, Dimitris N. Metaxas,
and Xiang Sean Zhou designed a multi-stage deep learning
framework for image classification and apply it on body part
recognition. In general image recognition problems,
discriminative information often lies in local image patches.
For example, most human identity information exists in the
image patches containing human faces. The same situation
stays in medical images as well. “Body part identity” of a
transversal slice which body part the slice comes from - is
often indicated by local image information, e.g. a cardiac
slice and an aorta arch slice are only differentiated by the Fig. 3.1.Block diagram of methodology
mediastinum region. Specifically, this method aims at: 1)
discover the local regions that are discriminative and non-
3.2 FCN ARCHITECTURE
informative to the image classification problem, and 2) learn
a image-level classifier based on these local regions. These Pixel-wise classification is performed and FCN is
two tasks are achieved by the two stages of learning scheme, essentially served as a filter that projects the entire input
respectively. In the pre-trained stage, a CNN is learned in a image to a map where each element represents the
multi instance learning fashion to extract the most probability that the corresponding input pixel belongs to the
discriminative and non-informative local patches from the tumor.
training slices. In the boosting stage, the pre-learned CNN is
further boosted by these local patches for image There are two path used in FCN layer. They are,
classification. The CNN learned by exploiting the Convolutional path
discriminative local appearances becomes more accurate than Deconvolutional path
those learned from global image context. The key hallmark
of this method is that it automatically discovers the In contrast image segmentation calls for classifying
discriminative and non-informative local patches through each pixel into foreground or background in combination
multi instance deep learning. Thus, no manual annotation is with full resolution output. Figure 3.2 shows the FCN
required. This method is validated on a synthetic dataset and Architecture
a large scale CT dataset. It achieves better performances than
state-of-the-art approaches, including the standard deep
CNN.
Adriano Pinto, Sergio Pereira, Higino Correia, J.
Oliveira, Deolinda M. L. D. Rasteiro and Carlos A. Silva
proposed a discriminative and fully automatic method for the
segmentation of gliomas, using appearance- and context-
based features to feed an Extremely Randomized Forest
Gliomas are among the most common and aggressive brain
tumours. Segmentation of these tumors is important for
surgery and treatment planning, but also for follow-up
evaluations. However, it is a difficult task, given that its size Fig 3.2 FCN Architecture
and locations are variable, and the delineation of all tumor
tissue is not trivial, even with all the different modalities of With the successive convolution and pooling
the MRI. Some of these features in this method are computed layers, CNNs can integrate contextual information from
over a non-linear transformation of the image. The method regional to global scales, resulting in reduced resolution in
was evaluated using the publicly available Challenge the output layer. In order to address this conflict between
database from BRATS 2013, having obtained a Dice score of multi-scale information aggregation and full-resolution pixel
0.83, 0.78 and 0.73 for the complete tumor, and the core and wise classification, a strategy of up-sampling and de-
the enhanced regions, respectively. The results are convolutional layers is used to recover lost resolution while
competitive, when compared against other results reported
carrying over the global perspective from pooling layers.
using the same database.
The up-sampling layer performs the reverse
IV. PROPOSED SYSTEM operation of pooling and reconstructs the original size of
activation and the de-convolutional layer densifies the
coarse activation map obtained from up-sampling through
3.1 INTRODUCTION swapping the forward and backward passes of a
A fully automatic framework based on deep convolution, thus a single input activation is projected into
convolutional neural network is used for skin lesion multiple outputs after deconvolution, yielding an enlarged
segmentation on dermoscopic images. and dense feature map.
 Echo to the convolution path where image
information is aggregated from fine details to global
concept, a hierarchical structure of deconvolutional layers is
used to recover image details at different levels, with the
lower layers encoding overall image information and higher
layers capturing fine details regarding skin tumors. In this
way, the network can take both global information and fine
details into account for tumor segmentation.
3.3 LOSS FUNCTION
For any pixel in the input image, the corresponding
FCN output represents an estimated posterior probability
that this pixel belongs to the skin tumor the skin tumor.
Since FCN essentially performs a pixel-wise classification,
cross entropy is usually used as the loss function.
Where, ∈{0, 1} is the actual class of with = 1 for tumor and
= 0 for background. Using equation the loss function is
calculated for cross entropy.
In Dermoscopy, a skin tumor usually occupies a
small region in the whole dermoscopic image. As the result,
the pixel-wise classification tends to be biased towards the
background and therefore there is a higher chance for tumor
to be partially segmented or even missed. A typical solution
is to assign a weight to each pixel during training that
compensates the different frequencies of pixels from each
class, so the contributions of tumor and background are re-
balanced. However, this pixel wise reweighting procedure
brings additional computation cost especially when image
augmentations are employed in the training procedure.
Novel loss function based on Jaccard distance. Jaccard index
is used, which is also known as the Jaccard similarity
coefficient, is one of the most frequently used evaluation
measures in medical image segmentation.
The Jaccard distance, which measures dissimilarity
between two sets, is complementary to the Jaccard index.
Let M represent the ground truth of segmentation, which is
normally a manually identified tumor region, and C
represent a computer-generated mask, the Jaccard distance is
defined as
where,
M represent the ground truth of segmentation,
which is normally a manually identified tumor region.
C represent a computer-generated mask
dJ (M,C) itself is not differentiable, which makes it difficult
to be directly applied into back propagation. Using equation
(3.2) the Jaccard distance is calculated. Even some
minimization methods for non differentiable functions can
be used, but it would be computationally expensive to
generate a binary mask from continuous FCN output for
each iteration during optimization. Thus, it uses the
following loss function
Equation (3.3) is used to calculate the loss function
for each iteration during optimization. With this loss
function, a weight map is not needed to rebalance pixels
from tumor region and background. Meanwhile, the
proposed loss function is differentiable which can be
efficiently integrated into back propagation during network
training.
3.4 FCN TRAINING
Training a deep network model with a limited
number of samples is a challenging task. FCN has 19 layers
and 290, 129 parameters to be learned. Even though the
number of training samples available is considered as a
large-scale study in the field of digital dermoscopic image
analysis, it is still relatively small as compared to the size of
the network. In order to address this issue, firstly apply FCN
model to convert image segmentation into a pixel-wise
classification problem. As the result, each pixel can be
considered as an independent sample during network
training, which greatly boosts the number of equivalent
training samples. Then, initialize the network weights using
Xavier’s technique and employ the following strategies to
improve the efficiency of network training while reducing
overfitting.
1) Adam stochastic optimization:
Stochastic gradient descent (SGD) with mini-batch
is usually employed as the optimization algorithm for neural
network training. It is well known that learning rate is one of
the critical hyper parameters that have a significant impact
on classification performance. However, selecting proper
learning rate and strategy can be fairly challenging.
Then Adam optimization algorithm, or adaptive
moments, is adapted to adjust the learning rate based on the
first and the second-order moments of the gradient at each
iteration.
Here the momentum is incorporated as an estimate
of the first moment and the effective step size at each
iteration depends on the ratio between the biascorrected first
and second-order moments with a smaller value indicating
that there is greater uncertainty about whether the direction
of corresponds to the direction of the true gradient. Adam is
fairly robust to the choice of hyper parameters, and set the
learning rate as 0.003 to speed up the training procedure in
this method.
2) Batch normalization:
Batch normalization is employed to reduce the
internal covariate shift by normalizing the input distribution
of every layer to the standard Gaussian distribution for each
training mini-batch. For this purpose, add a batch
normalization layer to the output of every convolutional and
de-convolutional layer. The batch normalization is
performed over all locations in the same feature map such
that different elements in the same map are normalized in
the same way. In this method, 18 is taken as the batch size.
3) Dropout:
 Dropout provides a powerful but computationally the total number of epochs is set as 500, and the optimal
inexpensive way to reduce overfitting when training a very epoch that yields the best performance on the validation
deep FCN model with limited data. This technique sets the dataset is saved. Also saved is the corresponding trained
output of each neuron in a given layer to zero with FCN model. After cross validation is completed, another
probability p, thus removes the contribution of those FCN model is trained using the entire training data and the
dropped out neurons from both forward pass and back- total number of epochs is set as the maximum value among
propagation. The subset of disabled neurons is drawn the optimal epochs obtained from cross validation. The
independently for each mini batch and forms a different outputs of these six FCN models are then averaged to
network architecture, then dropout trains the ensemble of all predict the final segmentation.
sub-networks that have different architectures but share
weights in one epoch. In this way, a neuron cannot rely on
the presence of particular other neurons and it is therefore,
forced to learn more robust features that are useful among V. RESULT
different random subsets. This makes the trained FCN model 4.1 TRAINED IMAGES
more robust and improves the generalization ability. The Input dataset image is trained. The figure 4.1
shows the different types of trained lesion images taken
4) Image augmentation: from the dataset.
In order to improve the robustness of the proposed
FCN model under a wide variety of image acquisition
conditions, image augmentation is used to further reduce
model overfitting by artificially enlarging the training
dataset with various image transformations. For the tumor
segmentation on dermoscopic images, primarily look for
invariance to mild geometric transformation as well as
robustness to pixel value variations, so the following two
types of image augmentations is implemented. These
augmentations only require little extra computation, so the
transformed images are generated from the original images
for every mini-batch within each iteration. The other type of
image augmentation focuses on randomly normalizing the
contrast of each channel in the training images.
Fig 4.1 Trained images
3.5 IMPLEMENTATION
Training data fit a model. Unsupervised learning refers to
FCN-based lesion segmentation algorithm was
analysis in which one attempts to learn something about the
implemented with Python based on Theano and Lasagne
data. other than predicting an output value of interest.
packages. The aim is to have the FCN model automatically
learn those features that are useful for lesion segmentation,
so a simple pre-processing is employed to facilitate the
4.2 TRAINED LABELS
following learning procedure while preserving the original
By using thresholding the training images are
image information. By observing most of images in the
converted in to training labels. The figure 4.2 shows the
training set have a height to width ratio of 3:4, resize the
trained labels of lesion image.
images to 192×256 using bi-linear interpolation. RGB
channels are kept as the input to the FCN model and each
channel is rescaled to [0, 1]. The output of FCN model is a
posterior probability map where each pixel value represents
the probability that the pixel belongs to the lesion. A dual-
threshold method is used to obtain a binary tumor mask
from this probability map. Specifically, a relatively high
threshold (thH= 0.8) is firstly applied to the FCN output to
generate a series of tumor candidates. The mass of each
candidate region, which is the area weighted by the pixel
values within this region, is calculated. The tumor center is
chosen as the centroid of the region that has the largest mass
among these candidates. Then a lower threshold = 0.5) is
applied to the probability map. After filling small holes with
morphological dilation, the final tumor mask is determined
as the region that embraces the tumor center. Since this post- Fig 4.2 Trained Labels
processing only involves thresholding and morphological
operations, the whole lesion segmentation can be done Labeled data is a group of samples that have been
within two seconds. tagged with one or more labels. Labeling typically takes a
Finally, a bagging-type ensemble strategy is set of unlabeled data and augments each piece of that
implemented to combine outputs of different FCNs to unlabeled data with meaningful tags that are informative.
further improve the image segmentation performance on the
testing images. For each fold in the 5-fold cross validation,
 After obtaining a labeled dataset, the models can be 4.5 SEGMENTED CONTOUR
applied to the data so that new unlabeled data can be The result of image segmentation is a set of
presented to the model and a likely label can be guessed or segments that collectively cover the entire image or a set of
predicted for that piece of unlabeled data. contours extracted from the image.

4.3 INPUT IMAGE

Dermatoscopy is also known as dermoscopy or
epiluminescence microscopy is the examination of skin
lesions with a dermatoscope. When the images or video
clips are digitally captured or processed, the instrument can
be referred to as a digital epiluminescence dermatoscope.

Fig 4.5 Segmented contour

Each of the pixels in a region are similar with respect to

some characteristic or computed property such as color,
intensity or texture. The fig 4.5 shows the image of
segmented contour which highlights the boundary of the
Fig 4.3 Input image segmented portion.

Input image given is a raw image as dermoscopic 4.6 CONVOLUTION FEATURES

image. The dermatoscope is the standard piece of equipment The features of the trained dataset images in the
used to perform a dermoscopy examination the fig 4.3 convolutional layer are obtained using fully convolutional
shows the dermoscopic image which is given as the input. network. The fig 4.6 shows the layer convolutional features
of the trained images.

4.4 SEGMENTED BINARY IMAGE

A binary image is a digital image that has only two
possible values for each pixel. Typically, the two colors used
for a binary image are black and white. The color used for
the object in the image is the foreground color while the rest
of the image is the background color.

Binary images are produced from color images by

segmentation. Thresholding is based on a clip-level to turn a
gray-scale image into a binary image. The accuracy of the
segmentation process greatly affects subsequent process.
The fig 4.4 shows the segmented binary image.

Fig 4.6 convolution features

Convolutional Neural Networks are very similar to

ordinary Neural Networks .They are made up of neurons
that have learnable weights and biases. Each neuron receives
some inputs, performs a dot product and optionally follows
it with a non-linearity.

4.7 RELU FEATURES

Rectified linear unit act as the activation function
Fig 4.4 Segmented binary image for each convolutional or deconvolutional layer. It accepts
only Positive value and negative value are considered as
Zero. The fig 4.7 shows the layer Relu features of the
trained image.
Fig 4.7 Relu features
Relu layer will apply an elementwise activation
function. A unit employing the rectifier is also called a
rectified linear unit. The function returns 0 if it receives any
negative input, but for any positive value it returns that
value back.
4.8 SEGMENTED OUTPUT IMAGE
After obtaining the features, the image is
segmented to detect the portion of lesion. The fig 4.8 shows
the image of segmented output.
Fig 4.8 Segmented output
VI. CONCLUSION
In this project, a fully automatic framework based on
deep convolutional neural network is used for skin lesion
segmentation on dermoscopic images. Several effective
training strategies were implemented to tackle the
challenges that training a deep network may face when only
limited training data is available. A novel loss function based
on the Jaccard distance is used to further boost the
segmentation performance. Compared to the conventionally
used cross entropy, the Jaccard distance based loss function
directly maximizes the overlap between the foreground of
the ground truth and that of the predicted segmentation
mask, and thus eliminates the needs of data rebalancing
when the numbers of foreground and background pixels are
highly unbalanced, such as binary medical image
segmentation. This method is robust to various image
artifacts and imaging acquisition conditions while using
minimum pre and post-processing.
VII. FUTURE ENHANCEMENT
In the future, this segmented lesion output is used to
detect the type of lesion which can be useful for further
treatment. The lesion is detected either as malignant or
benign using a more efficient technique.
VIII. REFERENCES
[1] R. L. Siegel, K. D. Miller, and A. Jemal, “Cancer
statistics, 2016,” CA Cancer J Clin., vol. 66, pp. 7–30, 2016.
[2] C. A. Barker and M. A. Postow, “Combinations of
radiation therapy and immunotherapy for melanoma: a
review of clinical outcomes,” Int J Radiation Oncol Biol
Phys, vol. 88, no. 5, pp. 986–997, 2014.
[3] C. M. Balch, J. E. Gershenwald, and S.-J. S. et al., “Final
version of 2009 AJCC melanoma staging and classification,”
J Clin Oncology, vol. 27, no. 36, pp. 6199–6206, 2009.
[4] M. Vestergaard, P. Macaskill, P. Holt, and S. Menzies,
“Dermoscopy compared with naked eye examination for the
diagnosis of primary melanoma: a meta-analysis of studies
performed in a clinical setting,” Br. J. Dermatol., vol. 159,
no. 3, pp. 669–676, 2008.
[5] K. Korotkov and R. Garcia, “Computerized analysis of
pigmented skin lesions: a review,” Artif. intell. Med., vol.
56, no. 2, pp. 69–90, 2012.
[6] H. Ganster, P. Pinz, R. Rohrer, E. Wildling, M. Binder,
and H. Kittler, “Automated melanoma recognition,” IEEE
Trans. Med. Imag., vol. 20, no. 3, pp. 233–239, 2001.
[7] M. E. Celebi, H. A. Kingravi, B. Uddin, H. Iyatomi, Y.
A. Aslandogan, W. V. Stoecker, and R. H. Moss, “A
methodological approach to the classification of dermoscopy
images,” Comput. Med. Imag. Graph., vol. 31, no. 6, pp.
362–373, 2007.
[8] M. Silveira, J. C. Nascimento, J. S. Marques, A. R.
Marçal, T. Mendonça, S. Yamauchi, J. Maeda, and J.
Rozeira, “Comparison of segmentation methods for
melanoma diagnosis in dermoscopy images,” IEEE J. Sel.
Top. Signal Process., vol. 3, no. 1, pp. 35–45, 2009.
[9] M. E. Celebi, H. Iyatomi, G. Schaefer, and W. V.
Stoecker, “Lesion border detection in dermoscopy images,”
Comput. Med. Imag. Graph., vol. 33, no. 2, pp. 148–153,
2009.
[10] M. Emre Celebi, Q. Wen, S. Hwang, H. Iyatomi, and G.
Schaefer, “Lesion border detection in dermoscopy images
using ensembles ofthresholding methods,” Skin Res.
Technol., vol. 19, no. 1, pp. e252– e258, 2013.

[11] F. Peruch, F. Bogo, M. Bonazza, V.-M. Cappelleri, and

E. Peserico, “Simpler, faster, more accurate melanocytic
lesion segmentation through meds,” IEEE Trans. Biomed.
Eng., vol. 61, no. 2, pp. 557–565, 2014.

[12] H. Zhou, X. Li, G. Schaefer, M. E. Celebi, and P.

Miller, “Mean shift based gradient vector flow for image
segmentation,” Comput. Vis. Image Understand., vol. 117,
no. 9, pp. 1004–1016, 2013.

[13] A. R. Sadri, M. Zekri, S. Sadri, N. Gheissari, M.

Mokhtari, and F. Kolahdouzan, “Segmentation of
dermoscopy images using wavelet networks,” IEEE Trans.
Biomed. Eng., vol. 60, no. 4, pp. 1134–1141, 2013.

[14] F. Xie and A. C. Bovik, “Automatic segmentation of

dermoscopy images using self-generating neural networks
seeded by genetic algorithm,” Pattern Recognition, vol. 46,
no. 3, pp. 1012–1019, 2013.

[15] M. E. Celebi, Q. Wen, H. Iyatomi, K. Shimizu, H.

Zhou, and G. Schaefer, “A state-of-the-art survey on lesion
border detection in dermoscopyimages,” pp. 97–129, 2015.

[16] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner,

“Gradient-based learning applied to document recognition,”
Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324,
1998.

[17] A. Krizhevsky, I. Sutskever, and G. E. Hinton,

“Imagenet classificationwith deep convolutional neural
networks,” in Adv. Neural Inf. Process. Sys., 2012, pp.
1097–1105.

[18] D. C. Cire¸san, A. Giusti, L. M. Gambardella, and J.

Schmidhuber,“Mitosis detection in breast cancer histology
images with deep neuralnetworks,” in Proc. MICCAI 2013.
Springer, 2013, pp. 411–418.

[19] Z. Yan, Y. Zhan, Z. Peng, S. Liao, Y. Shinagawa, S.

Zhang, D. N. Metaxas, and X. S. Zhou, “Multi-instance deep
learning: Discover discriminative local anatomies for
bodypart recognition,” IEEE Trans. Med. Imag., vol. 35, no.
5, pp. 1332–1343, 2016.

[20] Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V.

C. Mok, L. Shi, and P.-A. Heng, “Automatic Detection of
Cerebral Microbleeds From MR Images via 3D
Convolutional Neural Networks,” IEEE Trans. Med. Imag.,
vol. 35, no. 5, pp. 1182–1195, 2016.