Available online at www.sciencedirect.

com

ScienceDirect
Procedia Computer Science 126 (2018) 1171–1177

International Conference on Knowledge Based and Intelligent Information and Engineering

Systems, KES2018, 3-5 September 2018, Belgrade, Serbia

Application of artificial intelligence methods to recognize

pathologies on medical images
Georgy Lebedeva,b*, Herman Klimenkoa, Sergey Kachkovskiyb, Vadim Konushinc, Ilya
Ryabkova,b, Alexander Gromovc,d
a
I.M. Sechenov First Moscow State Medical University, 2-4 Bolshaya Pirogovskaya st., 119991 Moscow, Russia
b
Federal Research Institute for Health Organization and Informatics, 11, Dobrolubova street, 127254, Moscow,
Russia c LLC Video Analysis Technologies, 7, Skulptora Mukhinoi street, Moscow, Russia
d
Graphics and Media Lab, Moscow State University, 119991 Moscow, Russia

Abstract

The rapid development of graphics processing units (GPU) allowed to use in practice resource-intensive methods of artificial
intelligence such as neural networks. At present, the authors got preliminary results on detection of some forms of leukocytes.
The article presents the first results of the work.
© 2018 The Authors. Published by Elsevier Ltd.
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and peer-review under responsibility of KES International.
Keywords: artificial intelligence, neural networks, pattern recognition

1. Application of methods of artificial intelligence and neural networks in medicine

The rapid development of graphics processing units (GPU) allowed to use in practice resource-intensive methods of
artificial intelligence such as neural networks. At the same time the neural networks had developed because of their
usage expansion. A popular application of neural networks is face detection and recognition on photos and videos 1.
It is logical to assume that their application is possible for detecting pathologies on medical images.
Nowadays many organizations in the world work in this area. Well-known examples of their application are
identification of hot spots on MR-images, invasive forms of breast cancer in biopsy samples, tuberculosis in
pneumonography, sports medicine2, pharmacology3, IFF(ICSI)4 etc. Actually every week we can hear news about
the success in using neural networks in new areas of medicine.
For example, Stanford researches5 have created a system for skin cancer diagnostics. To do so they hired 21
doctors who labeled more than 130000 images of skin with more than 2000 types of skin cancer. The system
outperforms any dermatologist on test dataset.
* Corresponding author. E-mail address: geramail@rambler.ru

1877-0509 © 2018 The Authors. Published by Elsevier Ltd.

This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and peer-review under responsibility of KES International.
10.1016/j.procs.2018.08.055
117 Georgy Lebedev et al. / Procedia Computer Science 126 (2018) 1171–1177

Haghpanahi et al.6 developed an algorithm for ECG analysis. They gained 64000 ECG records from 30000
patients and labeled them as belonging to one of the following categories (N=14): 12 heart arrhythmias, sinus
rhythm and noise. The model outperforms 6 cardiologists in accuracy.
Rajpurkar et al.7 developed an algorithrm for Chest X-Rays images classification. They used a dataset consisting
of more than 112000 images from 30800 patients. Labeling categories (N=14) included pneumonia, infiltration,
cardiomegaly and others. Performance of ChexNet (this is how the authors called their network) is statistically
significantly higher than radiologist performance. For the assessment a different dataset labeled by 4 radiology
specialists was used.
Despite experimental nature of the research the technologies will be widely applied in nearest future. It will make
doctors free from routine operations and will help them to focus directly on patients’ health.
The authors have conducted studies on the pathology detection on images using artificial intelligence in particular
neural networks. The nearest development in the area relates to pathology detection on X-rays and mammograms
searching pathological blood cells, analysis of eyeground pictures, ECG analysis. In the future it will be possible to
proceed to more difficult problems of computed tomogram analysis, ultrasound investigation and histology analysis.

2. Recognition of pathological cells on images of blood smears (bone marrow)

The authors have got preliminary results on detection of some forms of leukocytes. They used almost thousands
of marked photos from the microscope (further pictures) as a training set. The pictures were provided by the Dmitry
Rogachev National Research Center Pediatric Hematology, Oncology and Immunology. They were marked by
experts of the Research center.
Recognition of pathological cells can be divided into two stages – cell detection and cell classification.
At the first stage the cell detection task is executed. There are several well-known and widely used families of
network detectors, each with its own set of advantages and disadvantages - Faster R-CNN 5, YOLO6, SSD7,
RetinaNet8. For this task RetinaNet network detector was used. Such choice was made because this network detector
is the second most accurate (after Faster R-CNN, which is a lot slower). YOLO and SSD are several times faster
than RetinaNet, but the accuracy of these detectors is poor. The detector was trained using two sets of images of
different image scales. This provides for better detection of small blood cells. Besides that, the detector distinguishes
between regular blood cells and border blood cells. Border blood cells are located on the border of the image and
may not be fully visible, so they are excluded from further analysis.
At the second stage the cell classification task is executed. Similar to the situation with detectors there are various
families of classificators – VGG 9, Resnet10, Inception11, ResNeXt12, DenseNet13 etc. The best among the listed
classificators (except for VGG) were further trained in the blood cell classification task. For higher accuracy dataset
augmentation was performed using rotations, flips and crops. The trained networks were assembled in a committee,
which resulted in higher accuracy of blood cell classification. The results are shown in Fig. 2.
An example of a marked picture is given below on Fig. 1. The leukocytes are marked with rectangles. There can
be leukocytes of different types in a picture. It’s possible to specify types of each leukocyte.
 Fig. 1. Example of a marked picture.

Pictures of some types of leukocytes were numerous enough (some hundred). On this basis a neural network was
formed and researched. For classification quality is shown on Fig. 2: confusion matrix of the classifier. On the
vertical axis there is a true type of a cell, on horizontal across shown cell type predicted by the classifier.
 Fig. 2. Confusion matrix of the classifier

At the present they continue working on improving detection and classification quality and adding new types of
leukocytes by extending the training set. There are 17 types in Fig. 2. Further the quantity of types should exceed
80. The whole list is presented in table 1.
Table 1. Whole class list of the classifier.
Normal myelogram Basophil Mature Myelocyte
Blast cells Lymphoblast
Megakaryoblast
Myeloblast
Monoblast
Undifferentiated
Lymphocytes Variable
Small lymphocyte
Plasma cell
Prolymphocyte
Wide cytoplasmic
Megakaryocytes Basophilic
Blast
bare nucleus
oxyphilic
Promegakaryocyte
destroyed
Monocytic cells Monocyte
 Promonocyte
Neutrophil Metamyelocyte
Myelocyte
Band neutrophils
Promyelocyte
Segmented
Eosinophils Metamyelocytes
Myelocyte
Band
Promyelocyte
Segmented
Erythrokaryocytes Basophilic
Normoblast
Oxyphilic
Polychromatophilic
Blood Atypical mononuclear cells
Macrophage/ histiocytes Histiocytes
Cellular phagocytosis
Leishmanias
Lipophages
Pigmentophage with cellular remnants
Erythrophagocytosis
Tumor cells Ganglioneuroblastoma
Germ cell
Neuroblastoma
Osteosarcoma
Primitive neuroectodermal
Rhabdomyosarcoma
Ewing
etc
Pathological cells Anaplastic blast cells
Atypical promyelocytes Hypergranular
Hypogranular
lymphoblasts L1-L2 version
L3 version
with traits of L3
version
Megakaryocytes Megakaryoblasts
Microforms
Mononuclear forms
Nuclear separation
Myeloblasts Myelo/monoblast
Myeloblast without
inclusions
Myeloblast with
granules and Auer
bodies
Myeloblast with
granules
Myeloblast with
Auer bodies
Monoblasts Myeloblast
Monoblast
Promonocyte
Erythroblasts Basophilic
megaloblast
Megaloblastic
Normoblastic
Oxyphilic
megaloblast
Polychromatic
megaloblast
Other Despues Hypersegmentation
of nuclear segments
 Other
Hyposegmentation
Eosinophils
Other Artifacts
Changed cells
Mitosis
Cropped cells
Destroyed cells
Platelet
agglomeration
Stromal cells Fat cells
Fat drops
Osteoblasts
Osteoclasts
Endothelial cells
Platelets Platelets
Agglomerations

Wide application of neural networks is constrained because of two factors: difficulty of getting necessary medical
images and high laboriousness of markup. And for acceptable results it’s necessary to get a significant number of
images. With accumulation of marked pictures the neural networks will be used widely and the quality of their
detection and classification will be better.
The presence of certain forms of leukocytes means certain diseases. Faster and more precise diagnosis will
provide faster and more effective treatment. It's wellknown that there are not that many qualified experts in this area.
That’s why the pictures of blood to be sent to medical organizations where the experts work. All this takes time and
increases the experts’ workload. Possibility of getting preliminary analysis with the help of neural networks will
allow to reduce the time and the work. The experts’ opinions will be necessary only in difficult and undefined cases.
As results all this will increase the quality of health care, take less doctors’ time and give more information to take
the decision on treatment.

References

1. M. Nikitin, V. Konushin, A. Neural network model for video-based face recognition with frames quality assessment // Computer Optics, 2017,
Vol 41, N 5, Pages 732-742.
2. G.Lebedev, A.Gureeva, Y.Tikhonova. Software System for Dynamic Athlete Health Monitoring. // International Conference on Knowledge
Based and Intelligent Information and Engineering Systems, KES2017, 6-8 September 2017, Marseille, France / Procedia Computer Science.
Vol.112, 2017, Pages 1664-1669. https://doi.org/10.1016/j.procs.2017.08.194.
3. G.Lebedev, Y.Tikhonova. Information system for pharmacotherapy method selection // International Conference on Knowledge Based and
Intelligent Information and Engineering Systems, KES2017, 6-8 September 2017, Marseille, France / Procedia Computer Science. Vol.112,
2017, Pages 1658-1663. https://doi.org/10.1016/j.procs.2017.08.260.
4. Milewski Robert, Jamiołkowski Jacek, Milewska Anna Justyna, Domitrz Jan, Szamatowicz Jacek, Wołczyński Sławomir. Prognosis of the IVF
ICSI/ET procedure efficiency with the use of artificial neural networks among patients of the Department of Reproduction and Gynecological
Endocrinology. Ginekol Pol. 2009; 80: 900-906. https://journals.viamedica.pl/ginekologia_polska/article/view/46529.
5. Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
//Advances in neural information processing systems, 2015, Pages 91-99. http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-
object-detection-with-region-proposal-networks.pdf
6. Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. // IEEE
Conference on Computer Vision and Recognition, 2016. https://arxiv.org/abs/1506.02640.
7. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg. SSD: Single Shot
MultiBox Detector. // arXiv preprint arXiv:1512.02325, 2015.
8. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár. Focal loss for dense object detection. //arXiv preprint arXiv:1708.02002,
2017.9. Karen Simonyan, Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. // arXiv preprint
arXiv:1409.1556, 2014.
10. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Deep Residual Learning for Image Recognition. // Computer Vision and Pattern
Recognition, 2016. https://arxiv.org/abs/1512.03385.
11. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, Zbigniew Wojna. Rethinking the inception architecture for computer vision.
// IEEE Conference on Computer Vision and Pattern Recognition, 2016, Pages 2818-2826.
12. Saining Xie,Ross B. Girshick, Piotr Dollár, Zhuowen Tu, Kaiming He. Aggregated Residual Transformations for Deep Neural Networks. \\
IEEE Conference on Computer Vision and Recognition, 2017. https://arxiv.org/abs/1611.05431.
13. Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger. Densely Connected Convolutional Networks. \\ IEEE Conference
on Computer Vision and Recognition, 2017. https://arxiv.org/abs/1608.06993.