vmireavizВестник медицинского института «РЕАВИЗ». Реабилитация, Врач и ЗдоровьеBulletin of the Medical Institute "REAVIZ" (REHABILITATION, DOCTOR AND HEALTH)2226-762X2782-1579РЕАВИЗ10.20340/vmi-rvz.2020.5.16vmireaviz-76Research ArticleИнформационно-вычислительные технологии в медицинеInformation technology in medicineТЕХНОЛОГИИ РАЗРАБОТКИ ПРОГРАММЫ СОДЕЙСТВИЯ ПРИНЯТИЮ РЕШЕНИЯ В ДИАГНОСТИКЕ ЗАБОЛЕВАНИЙ СИСТЕМЫ КРОВИ С ИСПОЛЬЗОВАНИЕМ СВЁРТОЧНЫХ ИСКУССТВЕННЫХ НЕЙРОННЫХ СЕТЕЙTECHNOLOGIES FOR DEVELOPING DECISION SUPPORT SYSTEMS FOR THE DIAGNOSIS OF BLOOD DISORDERS USING CONVOLUTIONAL NEURAL NETWORKShttps://orcid.org/0000-0002-3009-4744МасликоваУ. В.MaslikovaU. V.maslikova.ulyana@outlook.comhttps://orcid.org/0000-0002-1350-0704СупильниковА. А.SupilnikovA. A.a.a.supilnikov@reaviz.onlineНациональный медицинский исследовательский центр гематологииРоссияNational Medical Research Center for HematologyRussian FederationЧастное учреждение образовательная организация высшего образования «Медицинский университет «Реавиз»РоссияPrivate Institution of Higher Education ‘Medical University ‘Reaviz’Russian Federation20200402202105138150Copyright © Масликова У.В., Супильников А.А., 20212021Масликова У.В., Супильников А.А.Maslikova U.V., Supilnikov A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.reaviz.ru/jour/article/view/76

В рамках исполнения работы исследованы технологии получения, обработки, сегментации и передачи микрофотографий по протоколу для последующего распознавания. Выполнено исследование технологий получения, обработки, сегментации и передачи микрофотографий для последующего распознавания.  Отобраны наиболее перспективные алгоритмы машинного обучения, зарекомендовавшие себя в обработке медицинских изображений. Исследованы технологии анализа данных текстов медицинской  документации. Изучены аспекты применения нейросети Watson для анализа семантики медицинских  изображений. Изучены аспекты применения единого медицинского языка UMLS для нужд  синдромальной диагностики по изучению медицинских текстов истории болезни на натуральном языке. Разработан интерфейс получения, обработки, сегментации и передачи микрофотографий на вход  искусственной нейронной сети. Создан интерфейс первичного получения и обработки  микрофотографий на базе платформы обработки медицинских изображений OMERO. Для отправки данных в режиме онлайн подготовлен демо-скрипт для jupiter. Разработан интерфейс передачи текстов  медицинской документации системе распознавания семантики медицинского текста. Для анализа  медицинских текстов в первом приближении использован сервис IBM Watson Annotator for Clinical Data. Создана база данных медицинских изображений микрофотограмм костного мозга для подготовки нейросети. Получены микрофотографии мазков костного мозга при разрешении ×600 в световой  микроскопии (окраска гематоксилин-эозин) общим числом 3 500 цветных изображений 600×400  пикселей. Проведена разметка на 11 типов клеток костного мозга. Создана база данных медицинских текстов для подготовки нейросети. Подготовлена база данных медицинских текстов 167 пациентов для  обучения нейросети в объеме 40000 слов. Проведена деперсонализация личных данных пациентов.

микроскопия костного мозгаСАПРсистемы содействия принятию решенияанализ семантики текстамашинное зрениеbone marrow microscopyCADdecision support systemssemantics analysismachine visionРабота выполнена в рамках исполнения договора №15265ГУ/2020 от 14.06.2020 с Фондом содействия инновациямThis study was performed within the framework of the contract No. 15265GU/2020 dated 14.06.2020 with the Innovation Support Fund.ReferencesBain , Barbara J. Diagnosis from the Blood Smear. New England Journal of Medicine. 2005;353(5):498–507. pmid:16079373Bain , Barbara J. Diagnosis from the Blood Smear. New England Journal of Medicine. 2005;353(5):498–507. pmid:16079373Gallagher PG. Red Cell Membrane Disorders. Hematology. 2005;2005(1):13–18.Gallagher PG. Red Cell Membrane Disorders. Hematology. 2005;2005(1):13–18.Durant Thomas JS, Olson Eben M., Schulz Wade L, Torres R. Very Deep Convolutional Neural Networks for Morphologic Classification of Erythrocytes. Clinical Chemistry. 2017;63(12):1–9.Durant Thomas JS, Olson Eben M., Schulz Wade L, Torres R. Very Deep Convolutional Neural Networks for Morphologic Classification of Erythrocytes. Clinical Chemistry. 2017;63(12):1–9.Ford J. Red blood cell morphology. International Journal of Laboratory Hematology. 2013;35:351–357. pmid:23480230Ford J. Red blood cell morphology. International Journal of Laboratory Hematology. 2013;35:351–357. pmid:23480230Ceelie H, Dinkelaar RB, van Gelder W. Examination of peripheral blood films using automated microscopy; evaluation of Diffmaster Octavia and Cellavision DM96. J Clin Pathol. 2007;60:72–79. pmid:16698955Ceelie H, Dinkelaar RB, van Gelder W. Examination of peripheral blood films using automated microscopy; evaluation of Diffmaster Octavia and Cellavision DM96. J Clin Pathol. 2007;60:72–79. pmid:16698955Seyed HR, Hamid SZ. Automatic recognition of five types of white blood cells in peripheral blood. Computerized Medical Imaging and Graphics. 2011;35:333–343. pmid:21300521Seyed HR, Hamid SZ. Automatic recognition of five types of white blood cells in peripheral blood. Computerized Medical Imaging and Graphics. 2011;35:333–343. pmid:21300521Sedat N, Deniz K, Tuncay E, Murat HS, Osman K, Yavuz E. Automatic segmentation, counting, size determination and classification of white blood cells. Measurement. 2014;55:58–65.Sedat N, Deniz K, Tuncay E, Murat HS, Osman K, Yavuz E. Automatic segmentation, counting, size determination and classification of white blood cells. Measurement. 2014;55:58–65.Lorenzo P, Giovanni C, Cecilia DR. Leucocyte classification for leukaemia detection using image processing techniques. Artificial Intelligence in Medicine. 2014;62:179–191. pmid:25241903Lorenzo P, Giovanni C, Cecilia DR. Leucocyte classification for leukaemia detection using image processing techniques. Artificial Intelligence in Medicine. 2014;62:179–191. pmid:25241903Agaian S, Madhukar M, Chronopoulos AT. Automated Screening System for Acute Myelogenous Leukemia Detection in Blood Microscopic Images. IEEE SYSTEMS JOURNAL. 2014;8:995–1004.Agaian S, Madhukar M, Chronopoulos AT. Automated Screening System for Acute Myelogenous Leukemia Detection in Blood Microscopic Images. IEEE SYSTEMS JOURNAL. 2014;8:995–1004.ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. INTERNATIONAL JOURNAL OF LABORATORY HEMATOLOGY. 2016;38:209–219. pmid:26995648ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. INTERNATIONAL JOURNAL OF LABORATORY HEMATOLOGY. 2016;38:209–219. pmid:26995648Morteza M, Ahmad M, Nasser S, Saeed K, Ardeshir T. Computer aided detection and classification of acute lymphoblastic leukemia cell subtypes based on microscopic image analysis. Microscopy Research and Technique. 2016;79:908–916. pmid:27406956Morteza M, Ahmad M, Nasser S, Saeed K, Ardeshir T. Computer aided detection and classification of acute lymphoblastic leukemia cell subtypes based on microscopic image analysis. Microscopy Research and Technique. 2016;79:908–916. pmid:27406956Mathur A, Tripathi AS, Kuse M. Scalable system for classification of white blood cells from Leishman stained blood stain images. Journal of pathology informatics. 2013;4:15. Available from: http://www.jpathinformatics.org/text.asp?2013/4/2/15/109883.Mathur A, Tripathi AS, Kuse M. Scalable system for classification of white blood cells from Leishman stained blood stain images. Journal of pathology informatics. 2013;4:15. Available from: http://www.jpathinformatics.org/text.asp?2013/4/2/15/109883.Jaroonrut P, Charnchai P. Segmentation of white blood cells and comparison of cell morphology by linear and naive Bayes classifiers. BioMed. Eng. OnLine. 2015;14–63.Jaroonrut P, Charnchai P. Segmentation of white blood cells and comparison of cell morphology by linear and naive Bayes classifiers. BioMed. Eng. OnLine. 2015;14–63.Ramesh N, Dangott B, Salama ME, Tasdizen T. Isolation and two-step classification of normal white blood cells in peripheral blood smears. Journal of pathology informatics. 2012;3:3–13.Ramesh N, Dangott B, Salama ME, Tasdizen T. Isolation and two-step classification of normal white blood cells in peripheral blood smears. Journal of pathology informatics. 2012;3:3–13.Santiago A, Anna M, Laura B, Luis M, Magda R, Jose R. Automatic Recognition of Atypical Lymphoid Cells From Peripheral Blood by Digital Image Analysis. Am J Clin Pathol. 2015;143:168–176. pmid:25596242Santiago A, Anna M, Laura B, Luis M, Magda R, Jose R. Automatic Recognition of Atypical Lymphoid Cells From Peripheral Blood by Digital Image Analysis. Am J Clin Pathol. 2015;143:168–176. pmid:25596242Su MC, Cheng CY, Wang PC. A neural-network-based approach to white blood cell classification. The Scientific World Journal. 2014;1–9.Su MC, Cheng CY, Wang PC. A neural-network-based approach to white blood cell classification. The Scientific World Journal. 2014;1–9.Tamalika C. Accurate segmentation of leukocyte in blood cell images using Atanassov’s intuitionistic fuzzy and interval Type II fuzzy set theory. Micron. 2014;61:1–8. pmid:24792441Tamalika C. Accurate segmentation of leukocyte in blood cell images using Atanassov’s intuitionistic fuzzy and interval Type II fuzzy set theory. Micron. 2014;61:1–8. pmid:24792441ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. Jnl. Lab. Hem. 2016;38:209–219.ALFEREZ S, MERINO A, BIGORRA L, RODELLAR J. Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood. Jnl. Lab. Hem. 2016;38:209–219.Dan L-P, V. Javier T, Filiberto P. Recognizing white blood cells with local image descriptors. Expert Systems With Applications. 2019;115:695–708.Dan L-P, V. Javier T, Filiberto P. Recognizing white blood cells with local image descriptors. Expert Systems With Applications. 2019;115:695–708.Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision. 2004;60(2):91–110.Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision. 2004;60(2):91–110.Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the international conference on computer vision. 2011;2564–2571.Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the international conference on computer vision. 2011;2564–2571.Agrawal M, Konolige K, Blas MR. CenSurE: Center surround extremas for realtime feature detection and matching. In Proceedings of the European conference on computer vision. 2008;102–115.Agrawal M, Konolige K, Blas MR. CenSurE: Center surround extremas for realtime feature detection and matching. In Proceedings of the European conference on computer vision. 2008;102–115.Zhao JW, Zhang MS, Zhou ZH, Chu JJ, Cao FL. Automatic detection and classification of leukocytes using convolutional neural networks. Medical & Biological Engineering & Computing. 2016 Nov 07. https://doi.org/10.1007/s11517-016-1590-x.Zhao JW, Zhang MS, Zhou ZH, Chu JJ, Cao FL. Automatic detection and classification of leukocytes using convolutional neural networks. Medical & Biological Engineering & Computing. 2016 Nov 07. https://doi.org/10.1007/s11517-016-1590-x.Shahin AI, Guo YH, Amin KM, Sharawi AA. White Blood Cells Identification System Based on Convolutional Deep Neural Learning Networks. Computer Methods and Programs in Biomedicine. 2019;168:69–80. pmid:29173802Shahin AI, Guo YH, Amin KM, Sharawi AA. White Blood Cells Identification System Based on Convolutional Deep Neural Learning Networks. Computer Methods and Programs in Biomedicine. 2019;168:69–80. pmid:29173802Choi JW, Ku Y, Yoo BW, Kim J-A, Lee DS, Chai YJ, et al. White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks. PLoS ONE 2017; 12(12):e0189259. pmid:29228051Choi JW, Ku Y, Yoo BW, Kim J-A, Lee DS, Chai YJ, et al. White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks. PLoS ONE 2017; 12(12):e0189259. pmid:29228051Jiang M, Cheng L, Qin FW, Du L, Zhang M. White Blood Cells Classification with Deep Convolutional Neural Networks. International Journal of Pattern Recognition and Artificial Intelligence. 2018;32(9):1857006.Jiang M, Cheng L, Qin FW, Du L, Zhang M. White Blood Cells Classification with Deep Convolutional Neural Networks. International Journal of Pattern Recognition and Artificial Intelligence. 2018;32(9):1857006.Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Artur G. Fine-grained leukocyte classification with deep residual learning for microscopic images. Computer Methods and Programs in Biomedicine. 2018;162:243–252. pmid:29903491Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Artur G. Fine-grained leukocyte classification with deep residual learning for microscopic images. Computer Methods and Programs in Biomedicine. 2018;162:243–252. pmid:29903491Liang GB, Hong HC, Xie WF, Zheng LX. Combining convolutional neural network with recursive neural network for blood cell image classification. IEEE Access. 2018;6:36188–36197.Liang GB, Hong HC, Xie WF, Zheng LX. Combining convolutional neural network with recursive neural network for blood cell image classification. IEEE Access. 2018;6:36188–36197.Amjad R, Naveed A, Tanzila S, Syed IR, Zahid M, Hoshang K. Classification of acute lymphoblastic leukemia using deep learning. Microsc Res Tech. 2018;1–8.Amjad R, Naveed A, Tanzila S, Syed IR, Zahid M, Hoshang K. Classification of acute lymphoblastic leukemia using deep learning. Microsc Res Tech. 2018;1–8.Tiwari P, Qian J, Li QC, Wang BY, Gupta D, Khanna A, et al. Detection of Subtype Blood Cells using Deep Learning, Cognitive Systems Research. 2018 August 25. pii: S1389-0417(18)30376-0. https://doi.org/10.1016/j.cogsys.2018.08.022Tiwari P, Qian J, Li QC, Wang BY, Gupta D, Khanna A, et al. Detection of Subtype Blood Cells using Deep Learning, Cognitive Systems Research. 2018 August 25. pii: S1389-0417(18)30376-0. https://doi.org/10.1016/j.cogsys.2018.08.022Liu L, Ouyang WL, Wang XG, Paul F, Chen J, Liu XW, Matti P. Deep Learning for Generic Object Detection: A Survey. Preprint. Available from: arXiv: 1809.02165v1. Cited 6 Sep 2018.Liu L, Ouyang WL, Wang XG, Paul F, Chen J, Liu XW, Matti P. Deep Learning for Generic Object Detection: A Survey. Preprint. Available from: arXiv: 1809.02165v1. Cited 6 Sep 2018.Zou ZX, Shi ZW Guo YH, and Ye JP. Object Detection in 20 Years: A Survey. Preprint. Available from: arXiv: 1905.05055v1. Cited 13 May 2019.Zou ZX, Shi ZW Guo YH, and Ye JP. Object Detection in 20 Years: A Survey. Preprint. Available from: arXiv: 1905.05055v1. Cited 13 May 2019.Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Preprint. Available from: arXiv:1311.2524v3 Cited 7 May 2014.Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Preprint. Available from: arXiv:1311.2524v3 Cited 7 May 2014.He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision. 2014:346–361.He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision. 2014:346–361.Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2015:1440–1448.Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2015:1440–1448.

The authors declare that there are no conflicts of interest present.