Automatic diagnosis of severity of COVID-19 patients using an ensemble of transfer learning models with convolutional neural networks in CT images

• Autorzy: Shalbaf Ahmad , Gifani Parisa , Mehri-Kakavand Ghazal , Pursamimi Mohamad , Ghorbani Mahdi , Davanloo Amirhossein Abbaskhani , Vafaeezadeh Majid

Identyfikatory

DOI

10.2478/pjmpe-2022-0014

• Języki publikacji: EN

Abstrakty

EN

Introduction: Quantification of lung involvement in COVID-19 using chest Computed tomography (CT) scan can help physicians to evaluate the progression of the disease or treatment response. This paper presents an automatic deep transfer learning ensemble based on pre-trained convolutional neural networks (CNNs) to determine the severity of COVID -19 as normal, mild, moderate, and severe based on the images of the lungs CT. Material and methods: In this study, two different deep transfer learning strategies were used. In the first procedure, features were extracted from fifteen pre-trained CNNs architectures and then fed into a support vector machine (SVM) classifier. In the second procedure, the pre-trained CNNs were fine-tuned using the chest CT images, and then features were extracted for the purpose of classification by the softmax layer. Finally, an ensemble method was developed based on majority voting of the deep learning outputs to increase the performance of the recognition on each of the two strategies. A dataset of CT scans was collected and then labeled as normal (314), mild (262), moderate (72), and severe (35) for COVID-19 by the consensus of two highly qualified radiologists. Results: The ensemble of five deep transfer learning outputs named EfficientNetB3, EfficientNetB4, InceptionV3, NasNetMobile, and ResNext50 in the second strategy has better results than the first strategy and also the individual deep transfer learning models in diagnosing the severity of COVID-19 with 85% accuracy. Conclusions: Our proposed study is well suited for quantifying lung involvement of COVID-19 and can help physicians to monitor the progression of the disease.

Słowa kluczowe

EN

severity of COVID-19; convolutional neural network; ensemble

• Wydawca: Polskie Towarzystwo Fizyki Medycznej ; De Gruyter
• Czasopismo: Polish Journal of Medical Physics and Engineering
• Rocznik: 2022
• Tom: Vol. 28, Iss. 3
• Strony: 117--126
• Opis fizyczny: Bibliogr. 36 poz., rys., tab.

Twórcy

autor

Shalbaf Ahmad

• Biomedical Engineering and Medical Physics Department, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran

• https://orcid.org/0000-0002-1595-7281

autor

Gifani Parisa

• Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

autor

Mehri-Kakavand Ghazal

• Department of Medical Physics, School of Medicine, Semnan University of Medical Sciences, Semnan, Iran

autor

Pursamimi Mohamad

• Department of Medical Physics, School of Medicine, Semnan University of Medical Sciences, Semnan, Iran

autor

Ghorbani Mahdi

• Biomedical Engineering and Medical Physics Department, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran

autor

Davanloo Amirhossein Abbaskhani

• Department of Radiology, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

autor

Vafaeezadeh Majid

• School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

Bibliografia

• 1. Global COVID-19 Tracker & Interactive Charts. Available at: https://coronavirus.1point3acres.com/en/world
• 2. Bernheim A, Mei X, Huang M, Yang Y, Fayad ZA, Zhang N, et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology. 2020;295(3): 200463. https://doi.org/10.1148/radiol.2020200463
• 3. Zumla A, Chan JF, Azhar EI, Hui DS, Yuen KY. Coronaviruses-drug discovery and therapeutic options. Nat Rev Drug Discov. 2016;15(5):327-347. https://doi.org/10.1038/nrd.2015.37
• 4. Cote A, Ternacle J, Pibarot P. Early prediction of the risk of severe coronavirus disease 2019: A key step in therapeutic decision making. eBioMedicine. 2020;59:102948. https://doi.org/10.1016/j.ebiom.2020.102948
• 5. World Health Organization. (2020). Clinical management of COVID-19: interim guidance, 27 May 2020. World Health Organization. Available at: https://apps.who.int/iris/handle/10665/332196
• 6. Hachim A, Kavian N, Cohen CA, Chin AW, Chu DK, Mok CK, et al. ORF8 and ORF3b antibodies are accurate serological markers of early and late SARS-CoV-2 infection. Nat Immunol. 2020;21(10):1293-1301. https://doi.org/10.1038/s41590-020-0773-7
• 7. MacMahon H, Naidich DP, Goo JM, Soo Lee K, Leung ANC, Mayo JR, et al. Guidelines for management of incidental pulmonary nodules detected on CT images: From the Fleischner Society 2017. Radiology. 2017;284(1):228-243. https://doi.org/10.1148/radiol.2017161659
• 8. Koo HJ, Lim S, Choe J, Choi SH, Sung H, Do KH. Radiographic and CT features of viral pneumonia. Radiographics. 2018;38(3):719-739. https://doi.org/10.1148/rg.2018170048
• 9. Franquet T. Imaging of pulmonary viral pneumonia. Radiology. 2011;260(1):18-39. https://doi.org/10.1148/radiol.11092149
• 10. Feng Z, Yu Q, Yao S, Luo L, Zhou W, Mao X, et al. Early prediction of disease progression in COVID-19 pneumonia patients with chest CT and clinical characteristics. Nat Commun. 2020;11(1):4968. https://doi.org/10.1038/s41467-020-18786-x
• 11. Francone M, Iafrate F, Masci GM, Coco S, Cilia F, Manganaro L, et al. Chest CT score in COVID-19 patients: correlation with disease severity and short-term prognosis. Eur Radiol. 2020;30(12):6808-6817. https://doi.org/10.1007/s00330-020-07033-y
• 12. Matos J, Paparo F, Mussetto I, Bacigalupo L, Veneziano A, Bernardi SP, et al. Evaluation of novel coronavirus disease (COVID-19) using quantitative lung CT and clinical data: prediction of short-term outcome. Eur Radiol Exp. 2020;4(1):39. https://doi.org/10.1186/s41747-020-00167-0
• 13. Chassagnon G, Vakalopoulou M, Battistella E, Christodoulidis S, Hoang-Thi TN, Dangeard S, et al. AI-driven quantification, staging and outcome prediction of COVID-19 pneumonia. Med Image Anal. 2021;67:101860. https://doi.org/10.1016/j.media.2020.101860
• 14. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60-88. https://doi.org/10.1016/j.media.2017.07.005
• 15. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. arXiv:1808.01974v1 [cs.LG]. 2018. https://doi.org/10.48550/arXiv.1808.01974
• 16. Asgharnezhad H, Shamsi A, Alizadehsani R, et al. Objective evaluation of deep uncertainty predictions for COVID-19 detection. Sci Rep. 2022;12:815. https://doi.org/10.1038/s41598-022-05052-x
• 17. Alizadehsani R, Sharifrazi D, N Hoseini Izadi. et al. Uncertainty-Aware Semi-Supervised Method Using Large Unlabeled and Limited Labeled COVID-19 Data. arXiv:2102.06388 [eess.IV], 2021. https://doi.org/10.48550/arXiv.2102.06388
• 18. Ayoobi N, Sharifrazi D, Alizadehsani R, et al. Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods. Results in Physics. 2021;27:104495. https://doi.org/10.1016/j.rinp.2021.104495
• 19. Khozeimeh F, Sharifrazi D, Izadi NH, et al. Combining a convolutional neural network with autoencoders to predict the survival Chance of COVID-19 patients. Sci Rep. 2021;11:15343. https://doi.org/10.1038/s41598-021-93543-8
• 20. Sharifrazi D, Alizadehsani R, Roshanzamir N, et al. Fusion of convolution neural network, support vector machine and Sobel filter for accurate detection of COVID-19 patients using X-ray images. Biomed Signal Process Control. 2021;68102622. https://doi.org/10.1016/j.bspc.2021.102622
• 21. Shoeibi A, Khodatars M, Alizadehsani R. Automated Detection and Forecasting of COVID-19 using Deep Learning Techniques: A Review. arXiv:2007.10785 [cs.LG]. 2021. https://doi.org/10.48550/arXiv.2007.10785
• 22. Wang X, Deng X, Fu Q, Zhou Q, Feng J, Ma H, et al. A weakly-supervised framework for COVID-19 classification and lesion localization from chest CT. IEEE Transact Med Imaging. 2020;39(8):2615-2625. https://doi.org/10.1109/TMI.2020.2995965
• 23. Bansal S, Singh M, Dubey RK, et al. Multi-objective Genetic Algorithm Based Deep Learning Model for Automated COVID-19 Detection Using Medical Image Data. Journal of Medical and Biological Engineering. 2021;41:678-689. https://doi.org/10.1007/s40846-021-00653-9
• 24. Almalki YE, Qayyum A, Irfan M, et al. A Novel Method for COVID-19 Diagnosis Using Artificial Intelligence in Chest X-ray Images. Healthcare (Basel). 2021;9(5):522. https://doi.org/10.3390%2Fhealthcare9050522
• 25. Irfan M, Iftikhar MA, Yasin S, et al. Role of Hybrid Deep Neural Networks (HDNNs), Computed Tomography, and Chest X-rays for the Detection of COVID-19. Int J Environ Res Public Health. 2021;18(6):3056. https://doi.org/10.3390%2Fijerph18063056
• 26. Amini N, Shalbaf A. Automatic classification of severity of COVID-19 patients using texture feature and random forest based on computed tomography images. Int J Imaging Syst Technol. 2022;32(1):102-110. https://doi.org/10.1002/ima.22679
• 27. Pan F, Ye T, Sun P, Gui S, Liang B, Li L, et al. Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia. Radiology 2020:295(3):715-721. https://doi.org/10.1148/radiol.2020200370
• 28. Huang L, Han R, Ai T, Yu P, Kang H, Tao Q, Xia L, et al. Serial quantitative chest CT assessment of COVID-19: deep-learning approach. Radiol Cardiothorac Imaging. 2020;2(2):e200075. https://doi.org/10.1148/ryct.2020200075
• 29. Shan F, Gao Y, Wang J, Shi W, Shi N, Han M, et al. Lung infection quantification of COVID-19 in CT images with deep learning. arXiv:2003.04655. 2020. https://doi.org/10.48550/arXiv.2003.04655
• 30. Ghosh B, Kumar N, Singh N, Sadhu AK, Ghosh N, Mitra P, et al. A quantitative lung computed tomography image feature for multi-center severity assessment of COVID-19. medRxiv 2020.07.13.20152231. https://doi.org/10.1101/2020.07.13.20152231
• 31. Shen C, Yu N, Cai S, Zhou J, Sheng J, Liu K, et al. Quantitative computed tomography analysis for stratifying the severity of coronavirus disease 2019. J Pharmaceut Analysis 2020;10(2):123-129. https://doi.org/10.1016/j.jpha.2020.03.004
• 32. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Sys Man Cyber. 1979;9(1):62-66. https://doi.org/10.1109/TSMC.1979.4310076
• 33. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB et al. Convolutional neural networks for medical image analysis: full training or fine tuning?. IEEE Transact Med Imaging 2016;35(5):1299-1312. https://doi.org/10.1109/TMI.2016.2535302
• 34. Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009:248-255. https://doi.org/10.1109/CVPR.2009.5206848
• 35. Khan A, Sohail A, Zahoora U, Qureshi AS, et al. A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev. 2020;53:5455-5516. https://doi.org/10.1007/s10462-020-09825-6
• 36. Tang Z, Zhao W, Xie X, Zhong Z, Shi F, Ma T, et al. Severity assessment of COVID-19 using CT image features and laboratory indices. Phys Med Biol. 2021;66(3):035015. https://doi.org/10.1088/1361-6560/abbf9e

Uwagi

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023).