TECHNOLOGIES FOR DEVELOPING DECISION SUPPORT SYSTEMS FOR THE DIAGNOSIS OF BLOOD DISORDERS USING CONVOLUTIONAL NEURAL NETWORKS

In this study, we analyzed technologies for obtaining, processing, segmentation, and transmitting of microphotographs for subsequent recognition. We selected the most promising machine learning algorithms optimal for the processing of medical images, investigated the technologies of analyzing medical texts, studied the aspects of using the Watson neural network for analyzing the semantics of medical images, as well as the aspect of using the unified medical language UMLS for the needs of syndromic diagnostics for the evaluation of medical texts from medical histories in natural language. We also developed an interface for receiving,  processing, segmenting, and transmitting microphotographs to artificial neural networks and an interface for  the primary accepting and processing of microphotographs based on the OMERO medical image processing  platform. To send data online, a demo script for jupiter was prepared. An interface for transmitting medical texts  to the medical text semantics recognition system was also developed. The IBM Watson Annotator for Clinical  Data was used to perform preliminary analysis of medical texts. We created a database of medical images of  the bone marrow for neural network training. We made 3,500 color microphotographs (600×400 pixels) of  bone marrow smears at a resolution of ×600 (light microscopy; hematoxylin and eosin staining). We performed  marking of 11 types of bone marrow cells. We created a database of medical texts (167 patients, 40,000 words) to prepare a neural network. The database was stripped of all personal identifiers.

bone marrow microscopy, CAD, decision support systems, semantics analysis, machine vision

1. Bain , Barbara J. Diagnosis from the Blood Smear. New England Journal of Medicine. 2005;353(5):498–507. pmid:16079373

2. Gallagher PG. Red Cell Membrane Disorders. Hematology. 2005;2005(1):13–18.

3. Durant Thomas JS, Olson Eben M., Schulz Wade L, Torres R. Very Deep Convolutional Neural Networks for Morphologic Classification of Erythrocytes. Clinical Chemistry. 2017;63(12):1–9.

4. Ford J. Red blood cell morphology. International Journal of Laboratory Hematology. 2013;35:351–357. pmid:23480230

5. Ceelie H, Dinkelaar RB, van Gelder W. Examination of peripheral blood films using automated microscopy; evaluation of Diffmaster Octavia and Cellavision DM96. J Clin Pathol. 2007;60:72–79. pmid:16698955

6. Seyed HR, Hamid SZ. Automatic recognition of five types of white blood cells in peripheral blood. Computerized Medical Imaging and Graphics. 2011;35:333–343. pmid:21300521

7. Sedat N, Deniz K, Tuncay E, Murat HS, Osman K, Yavuz E. Automatic segmentation, counting, size determination and classification of white blood cells. Measurement. 2014;55:58–65.

8. Lorenzo P, Giovanni C, Cecilia DR. Leucocyte classification for leukaemia detection using image processing techniques. Artificial Intelligence in Medicine. 2014;62:179–191. pmid:25241903

9. Agaian S, Madhukar M, Chronopoulos AT. Automated Screening System for Acute Myelogenous Leukemia Detection in Blood Microscopic Images. IEEE SYSTEMS JOURNAL. 2014;8:995–1004.

10. ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. INTERNATIONAL JOURNAL OF LABORATORY HEMATOLOGY. 2016;38:209–219. pmid:26995648

11. Morteza M, Ahmad M, Nasser S, Saeed K, Ardeshir T. Computer aided detection and classification of acute lymphoblastic leukemia cell subtypes based on microscopic image analysis. Microscopy Research and Technique. 2016;79:908–916. pmid:27406956

12. Mathur A, Tripathi AS, Kuse M. Scalable system for classification of white blood cells from Leishman stained blood stain images. Journal of pathology informatics. 2013;4:15. Available from: http://www.jpathinformatics.org/text.asp?2013/4/2/15/109883.

13. Jaroonrut P, Charnchai P. Segmentation of white blood cells and comparison of cell morphology by linear and naive Bayes classifiers. BioMed. Eng. OnLine. 2015;14–63.

14. Ramesh N, Dangott B, Salama ME, Tasdizen T. Isolation and two-step classification of normal white blood cells in peripheral blood smears. Journal of pathology informatics. 2012;3:3–13.

15. Santiago A, Anna M, Laura B, Luis M, Magda R, Jose R. Automatic Recognition of Atypical Lymphoid Cells From Peripheral Blood by Digital Image Analysis. Am J Clin Pathol. 2015;143:168–176. pmid:25596242

16. Su MC, Cheng CY, Wang PC. A neural-network-based approach to white blood cell classification. The Scientific World Journal. 2014;1–9.

17. Tamalika C. Accurate segmentation of leukocyte in blood cell images using Atanassov’s intuitionistic fuzzy and interval Type II fuzzy set theory. Micron. 2014;61:1–8. pmid:24792441

18. ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. Jnl. Lab. Hem. 2016;38:209–219.

19. Dan L-P, V. Javier T, Filiberto P. Recognizing white blood cells with local image descriptors. Expert Systems With Applications. 2019;115:695–708.

20. Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision. 2004;60(2):91–110.

21. Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the international conference on computer vision. 2011;2564–2571.

22. Agrawal M, Konolige K, Blas MR. CenSurE: Center surround extremas for realtime feature detection and matching. In Proceedings of the European conference on computer vision. 2008;102–115.

23. Zhao JW, Zhang MS, Zhou ZH, Chu JJ, Cao FL. Automatic detection and classification of leukocytes using convolutional neural networks. Medical & Biological Engineering & Computing. 2016 Nov 07. https://doi.org/10.1007/s11517-016-1590-x.

24. Shahin AI, Guo YH, Amin KM, Sharawi AA. White Blood Cells Identification System Based on Convolutional Deep Neural Learning Networks. Computer Methods and Programs in Biomedicine. 2019;168:69–80. pmid:29173802

25. Choi JW, Ku Y, Yoo BW, Kim J-A, Lee DS, Chai YJ, et al. White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks. PLoS ONE 2017; 12(12):e0189259. pmid:29228051

26. Jiang M, Cheng L, Qin FW, Du L, Zhang M. White Blood Cells Classification with Deep Convolutional Neural Networks. International Journal of Pattern Recognition and Artificial Intelligence. 2018;32(9):1857006.

27. Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Artur G. Fine-grained leukocyte classification with deep residual learning for microscopic images. Computer Methods and Programs in Biomedicine. 2018;162:243–252. pmid:29903491

28. Liang GB, Hong HC, Xie WF, Zheng LX. Combining convolutional neural network with recursive neural network for blood cell image classification. IEEE Access. 2018;6:36188–36197.

29. Amjad R, Naveed A, Tanzila S, Syed IR, Zahid M, Hoshang K. Classification of acute lymphoblastic leukemia using deep learning. Microsc Res Tech. 2018;1–8.

30. Tiwari P, Qian J, Li QC, Wang BY, Gupta D, Khanna A, et al. Detection of Subtype Blood Cells using Deep Learning, Cognitive Systems Research. 2018 August 25. pii: S1389-0417(18)30376-0. https://doi.org/10.1016/j.cogsys.2018.08.022

31. Liu L, Ouyang WL, Wang XG, Paul F, Chen J, Liu XW, Matti P. Deep Learning for Generic Object Detection: A Survey. Preprint. Available from: arXiv: 1809.02165v1. Cited 6 Sep 2018.

32. Zou ZX, Shi ZW Guo YH, and Ye JP. Object Detection in 20 Years: A Survey. Preprint. Available from: arXiv: 1905.05055v1. Cited 13 May 2019.

33. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Preprint. Available from: arXiv:1311.2524v3 Cited 7 May 2014.

34. He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision. 2014:346–361.

35. Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2015:1440–1448.