Classification of lung nodule malignancy in computed tomography imaging utilising generative adversarial networks and semi-supervised transfer learning

• Autorzy: Apostolopoulos Ioannis D. , Papathanasiou Nikolaos D. , Panayiotakis George S.

Identyfikatory

DOI

10.1016/j.bbe.2021.08.006

• Języki publikacji: EN

Abstrakty

EN

The pulmonary nodules’ malignancy rating is commonly confined in patient follow-up; examining the nodule’s activity is estimated with the Positron Emission Tomography (PET) system or biopsy. However, these strategies are usually after the initial detection of the malignant nodules acquired from the Computed Tomography (CT) scan. In this study, a Deep Learning methodology to address the challenge of the automatic characterisation of Solitary Pulmonary Nodules (SPN) detected in CT scans is proposed. The research methodology is based on Convolutional Neural Networks, which have proven to be excellent automatic feature extractors for medical images. The publicly available CT dataset, called Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI), and a small CT scan dataset derived from a PET/CT system, is considered the classification target. New, realistic nodule representations are generated employing Deep Convolutional Generative Adversarial Networks to circumvent the shortage of large-scale data to train robust CNNs. Besides, a hierarchical CNN called Feature Fusion VGG19 (FF-VGG19) was developed to enhance feature extraction of the CNN proposed by the Visual Geometry Group (VGG). Moreover, the generated nodule images are separated into two classes by utilising a semi-supervised approach, called self-training, to tackle weak labelling due to DC-GAN inefficiencies. The DC-GAN can generate realistic SPNs, as the experts could only distinguish 23% of the synthetic nodule images. As a result, the classification accuracy of FF-VGG19 on the LIDCIDRI dataset increases by +7%, reaching 92.07%, while the classification accuracy on the CT dataset is increased by 5%, reaching 84,3%.

Słowa kluczowe

EN

lung nodule classification; data augmentation; generative adversarial network; medical image classification; semi-supervised learning

PL

guzki płuc; rozszerzanie danych; klasyfikacja obrazu medycznego; uczenie częściowo nadzorowane

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2021
• Tom: Vol. 41, no. 4
• Strony: 1243--1257
• Opis fizyczny: Bibliogr. 40 poz., rys., tab.

Twórcy

autor

Apostolopoulos Ioannis D.

• Department of Medical Physics, School of Medicine, University of Patras, 26504 Patras, Greece

autor

Papathanasiou Nikolaos D.

• Department of Nuclear Medicine, University Hospital of Patras, Patras, Greece

autor

Panayiotakis George S.

• Department of Medical Physics, School of Medicine, University of Patras, Patras, Greece

Bibliografia

• [1] Postmus P, Kerr K, Oudkerk M, Senan S, Waller D, Vansteenkiste J, et al. Early and locally advanced non-small-cell lung cancer (NSCLC): ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2017;28: iv1–21.
• [2] Zhuang H, Pourdehnad M, Lambright ES, Yamamoto AJ, Lanuti M, Li P, et al. Dual time point 18F-FDG PET imaging for differentiating malignant from inflammatory processes. J Nucl Med 2001;42:1412–7.
• [3] Tajbakhsh N, Suzuki K. Comparing two classes of end-to-end machine-learning models in lung nodule detection and classification: MTANNs vs CNNs. Pattern Recogn 2017;63:476–86.
• [4] Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press; 2016.
• [5] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, editors. Advances in neural information processing systems 27. Curran Associates, Inc.; 2014. p. 2672–80.
• [6] Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans: The LIDC/IDRI thoracic CT database of lung nodules. Med Phys 2011;38:915–31. https://doi.org/10.1118/1.3528204.
• [7] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:14091556 [cs] 2015.
• [8] Apostolopoulos ID, Mpesiana TA. Covid-19: automatic detection from X-ray images utilising transfer learning with convolutional neural networks. Phys Eng Sci Med 2020;43:635–40. https://doi.org/10.1007/s13246-020-00865-4.
• [9] Zhao D, Zhu D, Lu J, Luo Y, Zhang G. Synthetic medical images using F&BGAN for improved lung nodules classification by multi-scale VGG16. Symmetry 2018;10:519.
• [10] Chuquicusma MJ, Hussein S, Burt J, Bagci U. How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis. 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), IEEE; 2018, p. 240–4.
• [11] Javaid U, Lee JA. Capturing variabilities from computed tomography images with generative adversarial networks. arXiv:180511504 [cs, stat] 2018.
• [12] Jin D, Xu Z, Tang Y, Harrison AP, Mollura DJ. CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. International conference on medical image computing and computer-assisted intervention, Springer; 2018, p. 732–40.
• [13] Harrison AP, Xu Z, George K, Lu L, Summers RM, Mollura DJ. Progressive and multi-path holistically nested neural networks for pathological lung segmentation from CT images. International conference on medical image computing and computer-assisted intervention, Springer; 2017, p. 621–9.
• [14] Han C, Kitamura Y, Kudo A, Ichinose A, Rundo L, Furukawa Y, et al. Synthesising diverse lung nodules wherever massively: 3D multi-conditional GAN-based CT image augmentation for object detection. arXiv:190604962 [cs, eess] 2019.
• [15] Tang Y, Cai J, Lu L, Harrison AP, Yan K, Xiao J, et al. CT image enhancement using stacked generative adversarial networks and transfer learning for lesion segmentation improvement. arXiv:180707144 [cs] 2018.
• [16] Rother C, Kolmogorov V, Blake A. ‘‘GrabCut” interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG) 2004;23:309–14.
• [17] Xie S, Tu Z. Holistically-nested edge detection. Proceedings of the IEEE international conference on computer vision, 2015, p. 1395–403.
• [18] Yan K, Wang X, Lu L, Zhang L, Harrison AP, Bagheri M, et al. Deep lesion graphs in the wild: relationship learning and organisation of significant radiology image findings in a diverse large-scale lesion database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, p. 9261–70.
• [19] Bi L, Kim J, Kumar A, Feng D, Fulham M. Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs). In: Cardoso MJ, Arbel T, Gao F, Kainz B, van Walsum T, Shi K, editors. Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Cham: Springer International Publishing; 2017. p. 43–51. https://doi.org/10.1007/978-3-319-67564-0_5.
• [20] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Ann Hist Comput 2017:640–51.
• [21] Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, et al. Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. BioMed Res Int 2019;2019:1–9. https://doi.org/10.1155/2019/6051939.
• [22] Yang J, Liu S, Grbic S, Setio AAA, Xu Z, Gibson E, et al. Class-aware adversarial lung nodule synthesis in CT images. 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), IEEE; 2019, p. 1348–52.
• [23] Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 2013;26:1045–57. https://doi.org/10.1007/s10278-013-9622-7.
• [24] Shi W, Caballero J, Theis L, Huszar F, Aitken A, Ledig C, et al. Is the deconvolution layer the same as a convolutional layer? arXiv preprint arXiv:160907009 2016.
• [25] Ioffe S, Szegedy C. Batch normalisation: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:150203167 2015.
• [26] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15:1929–58.
• [27] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:151106434 2015.
• [28] Xu B, Wang N, Chen T, Li M. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:150500853 2015.
• [29] Kingma DP, Ba J. Adam: A method for stochastic optimisation. arXiv preprint arXiv:14126980 2014.
• [30] Yarowsky D. Unsupervised word sense disambiguation rivaling supervised methods. In: 33rd annual meeting of the association for computational linguistics. p. 189–96.
• [31] Livieris IE. A new ensemble self-labeled semi-supervised algorithm. Informatica 2019;43.
• [32] Cheng J-Z, Ni D, Chou Y-H, Qin J, Tiu C-M, Chang Y-C, et al. Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 2016;6:1–13.
• [33] Wu B, Zhou Z, Wang J, Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. arXiv:180203584 [cs] 2018.
• [34] Dey R, Lu Z, Hong Y. Diagnostic classification of lung nodules using 3D neural networks. arXiv preprint arXiv:1803071922018.
• [35] Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, et al. Multicrop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification. Pattern Recogn 2017;61:663–73. https://doi.org/10.1016/j.patcog.2016.05.029.
• [36] Song Q, Zhao L, Luo X, Dou X. Using deep learning for classification of lung nodules on computed tomography images. J Healthc Eng 2017;2017.
• [37] Causey JL, Zhang J, Ma S, Jiang B, Qualls JA, Politte DG, et al. Highly accurate model for prediction of lung nodule malignancy with CT scans. Sci Rep 2018;8:9286.
• [38] da Silva GL, da Silva Neto OP, Silva AC, de Paiva AC, Gattass M. Lung nodules diagnosis based on evolutionary convolutional neural network. Multimedia Tools Appl 2017;76:19039–55.
• [39] Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P. Infogan: Interpretable representation learning by information maximising generative adversarial nets. Adv Neural Inf Process Syst 2016:2172–80.
• [40] Dietterich TG. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 1998;10:1895–923. https://doi.org/10.1162/089976698300017197.