Attention-guided multiple instance learning for COPD identification: To combine the intensity and morphology

• Autorzy: Wu Yanan , Qi Shouliang , Feng Jie , Chang Runsheng , Pang Haowen , Hou Jie , Li Mengqi , Wang Yingxi , Xia Shuyue , Qian Wei

Identyfikatory

DOI

10.1016/j.bbe.2023.06.004

• Języki publikacji: EN

Abstrakty

EN

Chronic obstructive pulmonary disease (COPD) is a complex and multi-component respiratory disease. Computed tomography (CT) images can characterize lesions in COPD patients, but the image intensity and morphology of lung components have not been fully exploited. Two datasets (Dataset 1 and 2) comprising a total of 561 subjects were obtained from two centers. A multiple instance learning (MIL) method is proposed for COPD identification. First, randomly selected slices (instances) from CT scans and multi-view 2D snapshots of the 3D airway tree and lung field extracted from CT images are acquired. Then, three attention-guided MIL models (slice-CT, snapshot-airway, and snapshot-lung-field models) are trained. In these models, a deep convolution neural network (CNN) is utilized for feature extraction. Finally, the outputs of the above three MIL models are combined using logistic regression to produce the final prediction. For Dataset 1, the accuracy of the slice-CT MIL model with 20 instances was 88.1%. The backbone of VGG-16 outperformed Alexnet, Resnet18, Resnet26, and Mobilenet_v2 in feature extraction. The snapshotairway and snapshot-lung-field MIL models achieved accuracies of 89.4% and 90.0%, respectively. After the three models were combined, the accuracy reached 95.8%. The proposed model outperformed several state-of-the-art methods and afforded an accuracy of 83.1% for the external dataset (Dataset 2). The proposed weakly supervised MIL method is feasible for COPD identification. The effective CNN module and attention-guided MIL pooling module contribute to performance enhancement. The morphology information of the airway and lung field is beneficial for identifying COPD.

Słowa kluczowe

EN

chronic obstructive pulmonary disease; image classification; multiple instance learning; convolution neural network; computed tomography

PL

przewlekła obturacyjna choroba płuc; klasyfikacja obrazu; sieć neuronowa konwolucyjna; tomografia komputerowa

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2023
• Tom: Vol. 43, no. 3
• Strony: 568--585
• Opis fizyczny: Bibliogr. 87 poz., rys., tab., wykr.

Twórcy

autor

Wu Yanan

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China

autor

Qi Shouliang

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China

autor

Feng Jie

• School of Chemical Equipment, Shenyang University of Technology, Liaoyang, China

autor

Chang Runsheng

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China

autor

Pang Haowen

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China

autor

Hou Jie

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• Department of Radiology, General Hospital of Northern Theater Command, Shenyang, China

autor

Li Mengqi

• Graduate School, Dalian Medical University, Dalian, China
• Department of Respiratory, the Second Affiliated Hospital of Dalian Medical University, Dalian, China

autor

Wang Yingxi

• Graduate School, Dalian Medical University, Dalian, China
• Respiratory Department, Central Hospital Affiliated to Shenyang Medical College, Shenyang, China

autor

Xia Shuyue

• Respiratory Department, Central Hospital Affiliated to Shenyang Medical College, Shenyang, China

autor

Qian Wei

• College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China

Bibliografia

• [1] Global Strategy for the Diagnosis, Management, and Prevention of Chronic Obstructive Lung Disease: The GOLD Science Committee Report 2019. Eur Respir J. 2019.
• [2] Washko GR, Coxson HO, O’Donnell DE, Aaron SD. CT imaging of chronic obstructive pulmonary disease: insights, disappointments, and promise. Lancet Respir Med 2017;5(11).
• [3] Huls A, Schikowski T. Ambient particulate matter and COPD in China: a challenge for respiratory health research. Thorax 2017;72(9):771-2. https://doi.org/10.1136/thoraxjnl-2016- 209687.
• [4] Warming PE et al. Atrial fibrillation and chronic obstructive pulmonary disease: diagnostic sequence and mortality risk. Eur Heart J-Qual Care Clin Outcomes 2023;9(2):128-34.
• [5] Yu R, Kong X, Li Y. Optimizing the diagnostic algorithm for pulmonary embolism in acute COPD exacerbation using fuzzy rough sets and support vector machine. COPD: J Chron Obstruct Pulmon Dis 2023;20(1):1-8.
• [6] Kim W-D. Phenotype of chronic obstructive pulmonary disease based on computed tomography-defined underlying pathology. Tuberculosis Respirat Dis 2022;85(4):302-12.
• [7] Dhong K-R, Lee J-H, Yoon Y-R, Park H-J. Identification of TRPC6 as a novel diagnostic biomarker of PM-induced chronic obstructive pulmonary disease using machine learning models. Genes 2023;14(2):284.
• [8] Ezponda A et al. Chest CT-assessed comorbidities and allcause mortality risk in COPD patients in the BODE cohort. Respirology 2022;27(4):286-93.
• [9] Eliasson G et al. Comorbid conditions as predictors of mortality in severe COPD-an eight-year follow-up cohort study. Eur Clin Respirat J 2023;10(1):2181291.
• [10] Bouwens JD, Bischoff EW, in’t Veen JC, Schermer TR. Diagnostic differentiation between asthma and COPD in primary care using lung function testing. NPJ Primary Care Respir Med 2022;32(1):32.
• [11] Fortis S et al. Combined forced expiratory volume in 1 second and forced vital capacity bronchodilator response, exacerbations, and mortality in chronic obstructive pulmonary disease. Ann Am Thorac Soc 2019;16(7):826-35.
• [12] Friedlander AL, Lynch D, Dyar LA, Bowler RP. Phenotypes of chronic obstructive pulmonary disease. COPD: J Chron Obstruct Pulmon Dis 2007;4(4):355-84.
• [13] Flesch JD, Dine CJ. Lung volumes: measurement, clinical use, and coding. Chest 2012;142(2):506-10.
• [14] Fan L, Xia Y, Guan Y, Zhang TF, Liu SY. Characteristic features of pulmonary function test, CT volume analysis and MR perfusion imaging in COPD patients with different HRCT phenotypes. Clin Respir J 2014;8(1):45-54.
• [15] Lynch DA et al. CT-definable subtypes of chronic obstructive pulmonary disease: a statement of the Fleischner Society. Radiology 2015:192-205.
• [16] Fraiwan L, Hassanin O, Fraiwan M, Khassawneh B, Ibnian AM, Alkhodari M. Automatic identification of respiratory diseases from stethoscopic lung sound signals using ensemble classifiers. Biocybern Biomed Eng 2021;41(1):1-14.
• [17] Stokes K et al. A machine learning model for supporting symptom-based referral and diagnosis of bronchitis and pneumonia in limited resource settings. Biocybern Biomed Eng 2021;41(4):1288-302.
• [18] Bafadhel M et al. The role of CT scanning in multidimensional phenotyping of COPD. Chest 2011;140(3):634-42.
• [19] Haider NS, Behera A. Computerized lung sound based classification of asthma and chronic obstructive pulmonary disease (COPD). Biocybern Biomed Eng 2022;42(1):42-59.
• [20] Haider NS, Behera A. Respiratory sound denoising using sparsity-assisted signal smoothing algorithm. Biocybern Biomed Eng 2022;42(2):481-93.
• [21] Moslemi A et al. Quantitative CT lung imaging and machine learning improves prediction of emergency room visits and hospitalizations in COPD. Acad Radiol 2023;30(4):707-16.
• [22] Gong Y, Lv Y, Liu H, Zheng Q, Li L. Quantitative analysis of efficacy and safety of LABA/LAMA fixed-dose combinations in the treatment of stable COPD. Therap Adv Respirat Dis 2022;16 17534666211066068.
• [23] Dirksen A, Wille MMW. Computed tomography-based subclassification of chronic obstructive pulmonary disease. Ann Am Thorac Soc 2016;13(Supplement_2):S114-7.
• [24] Bodduluri S, Puliyakote AK, Nakhmani A, Charbonnier JP, Bhatt SP. CT-based airway surface area to volume ratio for phenotyping airway remodeling in chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2020.
• [25] Bhatt SP et al. Imaging advances in chronic obstructive pulmonary disease: insights from COPDGene. Am J Respir Crit Care Med 2018;199(3):286-301.
• [26] Shima H et al. Subtyping emphysematous COPD by respiratory volume change distributions on CT. Thorax 2023;78(4):344-53.
• [27] Park J et al. Phenotyping COPD patients with emphysema distribution using quantitative CT measurement; More severe airway involvement in lower dominant emphysema. Int J Chron Obstruct Pulmon Dis 2022:2013-25.
• [28] Isaac A, Nehemiah HK, Dunston SD, Kannan A. Feature selection and classification using bio-inspired algorithms for the diagnosis of pulmonary emphysema subtypes. Int J Imaging Syst Technol 2023.
• [29] Bodduluri S, Reinhardt JM, Hoffman EA, Newell JD, Bhatt SP. Recent advances in computed tomography imaging in chronic obstructive pulmonary disease. Ann Am Thorac Soc 2018;15(3):281-9.
• [30] Ostridge K, Wilkinson TMA. Present and future utility of computed tomography scanning in the assessment and management of COPD. Eur Respir J 2016;48(1):216-28.
• [31] Hofmanninger J, Prayer F, Pan J, Ro¨hrich S, Prosch H, Langs G. Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur Radiol Exp 2020;4(1):1-13.
• [32] Xu M et al. Segmentation of lung parenchyma in CT images using CNN trained with the clustering algorithm generated dataset. Biomed Eng Online 2019;18(1):1-21.
• [33] Zheng H et al. Alleviating class-wise gradient imbalance for pulmonary airway segmentation. IEEE Trans Med Imaging 2021;40(9):2452-62.
• [34] Ramalingam R, Chinnaiyan V. A comparative analysis of chronic obstructive pulmonary disease using machine learning, and deep learning. Int J Electr Comput Eng 2023;13 (1):389.
• [35] Kocks JW et al. Diagnostic performance of a machine learning algorithm (Asthma/Chronic Obstructive Pulmonary Disease [COPD] Differentiation Classification) tool versus primary care physicians and pulmonologists in asthma, COPD, and asthma/COPD overlap. J Allergy Clin Immunol. In Pract 2023.
• [36] Karthick G, Pankajavalli P. Chronic obstructive pulmonary disease prediction using Internet of things-spiro system and fuzzy-based quantum neural network classifier. Theor Comput Sci 2023;941:55-76.
• [37] Dorosti T et al. Optimizing convolutional neural networks for chronic obstructive pulmonary disease detection in clinical computed tomography imaging. arXiv preprint arXiv:2303.07189. 2023.
• [38] Yin C et al. Fractional dynamics foster deep learning of COPD stage prediction. Adv Sci 2023;10(12):2203485.
• [39] Jakkula V. Tutorial on support vector machine (svm). School EECS, Washington State Univ 2006;37(2.5):3.
• [40] Qi Y. Random forest for bioinformatics. In: Ensemble machine learning. Springer; 2012. p. 307-23.
• [41] LaValley MP. Logistic regression. Circulation 2008;117 (18):2395-9.
• [42] Yang Y et al. Lung radiomics features selection for COPD stage classification based on auto-metric graph neural network. Diagnostics 2022;12(10):2274.
• [43] Li Z et al. A novel CT-based radiomics features analysis for identification and severity staging of COPD. Acad Radiol 2022;29(5):663-73.
• [44] González G et al. Disease staging and prognosis in smokers using deep learning in chest computed tomography. Am J Respir Crit Care Med 2018;197(2):193-203.
• [45] Ho TT et al. A 3D-CNN model with CT-based parametric response mapping for classifying COPD subjects. Sci Rep 2021;11(1):1-12.
• [46] Park J et al. Subtyping COPD by using visual and quantitative CT imaging features. Chest 2020;157(1):47-60.
• [47] Tang LY, Coxson HO, Lam S, Leipsic J, Tam RC, Sin DD. Towards large-scale case-finding: training and validation of residual networks for detection of chronic obstructive pulmonary disease using low-dose CT. Lancet Digital Health 2020;2(5):e259-67.
• [48] Singla S, Gong M, Riley C, Sciurba F, Batmanghelich K. Improving clinical disease subtyping and future events prediction through a chest CT-based deep learning approach. Med Phys 2021;48(3):1168-81.
• [49] Li Z, Huang K, Liu L, Zhang Z. Early detection of COPD based on graph convolutional network and small and weakly labeled data. Med Biol Eng Comput 2022;60(8):2321-33.
• [50] Du R et al. Identification of COPD from multi-view snapshots of 3D lung airway tree via deep CNN. IEEE Access 2020;8:38907-19.
• [51] Wu Y et al. Deep CNN for COPD identification by multi-view snapshot integration of 3D airway tree and lung field. Biomed Signal Process Control 2023;79 104162.
• [52] Zhang L, Jiang B, Wisselink HJ, Vliegenthart R, Xie X. COPD identification and grading based on deep learning of lung parenchyma and bronchial wall in chest CT images. Br J Radiol 2022;95(1133):20210637.
• [53] Carbonneau M-A, Cheplygina V, Granger E, Gagnon G. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recogn 2018;77:329-53.
• [54] Tian Y, Hao W, Jin D, Chen G, Zou A. A review of latest multiinstance learning. In: 2020 4th International conference on computer science and artificial intelligence. p. 41-5.
• [55] Sudharshan P, Petitjean C, Spanhol F, Oliveira LE, Heutte L, Honeine P. Multiple instance learning for histopathological breast cancer image classification. Expert Syst Appl 2019;117:103-11.
• [56] Lippi M et al. Texture analysis and multiple-instance learning for the classification of malignant lymphomas. Comput Methods Programs Biomed 2020;185 105153.
• [57] Chang R et al. Deep multiple instance learning for predicting chemotherapy response in non-small cell lung cancer using pretreatment CT images. Sci Rep 2022;12(1):19829.
• [58] Quellec G, Cazuguel G, Cochener B, Lamard M. Multipleinstance learning for medical image and video analysis. IEEE Rev Biomed Eng 2017;10:213-34.
• [59] Li Z et al. A novel multiple instance learning framework for COVID-19 severity assessment via data augmentation and self-supervised learning. Med Image Anal 2021;69 101978.
• [60] Cheplygina V, Pena IP, Pedersen JH, Lynch DA, Sørensen L, de Bruijne M. Transfer learning for multicenter classification of chronic obstructive pulmonary disease. IEEE J Biomed Health Inform 2017;22(5):1486-96.
• [61] Cheplygina V, Sørensen L, Tax DM, Pedersen JH, Loog M, de Bruijne M. Classification of COPD with multiple instance learning. In: 2014 22nd International conference on pattern recognition. IEEE; 2014. p. 1508-13.
• [62] Xu C et al. DCT-MIL: deep CNN transferred multiple instance learning for COPD identification using CT images. Phys Med Biol 2020.
• [63] Sun J et al. Detection and staging of chronic obstructive pulmonary disease using a computed tomography-based weakly supervised deep learning approach. Eur Radiol 2022:1-11.
• [64] Xue M, Jia S, Chen L, Huang H, Yu L, Zhu W. CT-based COPD identification using multiple instance learning with two-stage attention. Comput Methods Programs Biomed 2023;230 107356.
• [65] Chen L et al. A graph convolutional multiple instance learning on a hypersphere manifold approach for diagnosing chronic obstructive pulmonary disease in CT images. IEEE J Biomed Health Inform 2022;26(12):6058-69.
• [66] Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH. nnUNet: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021;18 (2):203-11.
• [67] Mehta R et al. QU-BraTS: MICCAI BraTS 2020 challenge on quantifyinguncertainty in brain tumor segmentation-analysis of ranking scores and benchmarking results. J Mach Learn Biomed Imaging 2020;1:2022.
• [68] Van Rossum G. Python Programming language. USENIX annual technical conference 2007;41(1):1-36.
• [69] Lowekamp BC, Chen DT, Ibáñez L, Blezek D. The design of SimpleITK. Front Neuroinf 2013;7:45.
• [70] Oliphant TE. A guide to NumPy. USA: Trelgol Publishing; 2006.
• [71] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.
• [72] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 770-8.
• [73] Tajbakhsh N et al. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans Med Imaging 2016;35(5):1299-312.
• [74] Bahadori MT, Liu Y, Zhang D. A general framework for scalable transductive transfer learning. Knowl Inf Syst 2014;38(1):61-83.
• [75] Godfrey K. Simple linear regression in medical research. N Engl J Med 1985;313(26):1629-36.
• [76] Pedregosa F et al. Scikit-learn: Machine learning in Python. J Mach Learn Res 2011;12:2825-30.
• [77] Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 2012;25(2):pp.
• [78] Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 2017;284(2):574-82.
• [79] Howard AG et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. 2017.
• [80] Shin H-C et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 2016;35(5):1285-98.
• [81] Souid A, Sakli N, Sakli H. Classification and predictions of lung diseases from chest X-rays using MobileNet V2. Appl Sci 2021;11(6):2751.
• [82] Van Opbroek A, Ikram MA, Vernooij MW, De Bruijne M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans Med Imaging 2014;34 (5):1018-30.
• [83] Liu M, Shi J, Li Z, Li C, Zhu J, Liu S. Towards better analysis of deep convolutional neural networks. IEEE Trans Vis Comput Graph 2016;23(1):91-100.
• [84] Mair G et al. Computed tomographic emphysema distribution: relationship to clinical features in a cohort of smokers. Eur Respir J 2009;33(3):536-42.
• [85] Dosovitskiy A et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020.
• [86] Chen H et al. Association of inhaled corticosteroids with allcause mortality risk in patients with COPD: a meta-analysis of 60 randomized controlled trials. Chest 2023;163(1):100-14.
• [87] Stolz D et al. Towards the elimination of chronic obstructive pulmonary disease: a Lancet Commission. Lancet 2022;400 (10356):921-72.