Dual-attention EfficientNet based on multi-view feature fusion for cervical squamous intraepithelial lesions diagnosis

• Autorzy: Guo Ying , Wang Yongxiong , Yang Huimin , Zhang Jiapeng , Sun Qing

Identyfikatory

DOI

10.1016/j.bbe.2022.02.009

• Języki publikacji: EN

Abstrakty

EN

Cervicograms are widely used in cervical cancer screening but exhibit a high misdiagnosis rate. Even senior experts show only 48% specificity on clinical examinations. Most existing methods only use single-view images applied with acetic acid or Lugol’s iodine solution as their input data, ignoring the fact that non-pathological tissues may show false-positive reactions in these single-view images. This can lead to misdiagnosis in clinical diagnosis. Therefore, it is essential to extract features from multi-view colposcopy images (including the original images) as inputs, because three-view cervicograms provide complementary information. In this work, we propose an improved EfficientNet based on multi-view feature fusion for the automatic diagnosis of cervical squamous intraepithelial lesions. Specifically, EfficientNet-B0 is employed as the backbone network, and three-view images are taken as inputs by channel cascading to reduce misclassification. Additionally, we propose a dual-attention mechanism that implements the feature selection function based on Convolution Block Attention Module (CBAM) and Coordinate Attention (CA). These two attention mechanisms assist each other to enhance the feature representation of HSIL. We leverage a dataset of 3294 clinical cervigrams and obtain 90.0% accuracy with recall, specificity, and F₁-Score of 87.1%, 93.0%, and 89.7%, respectively. Experimental results prove that this method can help clinicians with precise disease classification and diagnosis, and out-performs known related works.

Słowa kluczowe

EN

cervical cancer; colposcopy; deep learning; dual attention; disease classification

PL

rak szyjki macicy; kolposkopia; uczenie głębokie; klasyfikacja choroby

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2022
• Tom: Vol. 42, no. 2
• Strony: 529--542
• Opis fizyczny: Bibliogr. 59 poz., rys., tab., wykr.

Twórcy

autor

Guo Ying

• University of Shanghai for Science and Technology, Shanghai, China

autor

Wang Yongxiong

• University of Shanghai for Science and Technology, Shanghai, China

autor

Yang Huimin

• The First Affiliated Hospital of Wan Nan Medical College, Wuhu, China

autor

Zhang Jiapeng

• University of Shanghai for Science and Technology, Shanghai, China

autor

Sun Qing

• The First Affiliated Hospital of Wan Nan Medical College, Wuhu, China

Bibliografia

• [1] Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021;71:209–49. https://doi.org/10.3322/caac.21660.
• [2] WHO. Human papillomavirus and related cancers in world. ICO Information Centre on HPV and Cancer Summary Report, 2014.
• [3] Chen T, Ma X, Liu X, WangW, Feng R, Chen J, et al. Multi-view learning with feature level fusion for cervical dysplasia diagnosis. Lect Notes Comput Sci (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11764 LNCS, 2019, p. 329–38. 10.1007/978-3-030-32239-7_37.
• [4] Xu T, Zhang H, Huang X, Zhang S, Metaxas DN. Multimodal deep learning for cervical dysplasia diagnosis. Lect Notes Comput Sci (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9901 LNCS, 2016, p. 115–23. 10.1007/978-3-319-46723-8_14.
• [5] Denkçeken T, Şimşek T, Erdoǧan G, Pesştereli E, Karaveli Ş, Özel D, et al. Elastic light single-scattering spectroscopy for the detection of cervical precancerous ex vivo. IEEE Trans Biomed Eng 2013;60:123–7. https://doi.org/10.1109/TBME.2012.2225429.
• [6] Mustafa WA, Halim A, Ab Rahman KS. A narrative review: Classification of pap smear cell image for cervical cancer diagnosis. Oncologie 2020;22:53–63.
• [7] Yang B, Chen C, Chen F, Ma C, Chen C, Zhang H, et al. Feature fusion combined with tissue Raman spectroscopy to screen cervical cancer. J Raman Spectrosc 2021;52:1830–7.
• [8] Cheng S, Liu S, Yu J, Rao G, Xiao Y, Han W, et al. Robust whole slide image analysis for cervical cancer screening using deep learning. Nat Commun 2021;12:1–10.
• [9] Guo P, Banerjee K, Stanley RJ, Long LR, Antani SK, Thoma GR, et al. Nuclei-based features for uterine cervical cancer histology image analysis with fusion-based classification. IEEE J Biomed Heal Informatics 2016;20:1595–607. https://doi.org/10.1109/JBHI.2015.2483318.
• [10] Greenspan H, Gordon S, Zimmerman G, Lotenberg S, Jeronimo J, Antani S, et al. Automatic detection of anatomical landmarks in uterine cervix images. IEEE Trans Med Imaging 2009;28:454–68. https://doi.org/10.1109/TMI.2008.2007823.
• [11] Aziz Mohamad KA, Samy Saad A, Anwar Murad AW. Visual inspection after acetic acid (via) as an alternative screening tool for cancer cervix. Gynecol Obstet 2015;5:204–7. https://doi.org/10.4172/2161-0932.1000336.
• [12] Huchko MJ, Sneden J, Zakaras JM, Smith-McCune K, Sawaya G, Maloba M, et al. A randomized trial comparing the diagnostic accuracy of visual inspection with acetic acid to visual inspection with Lugol’s iodine for cervical cancer screening in HIV infected women. PLoS ONE 2015;10. https://doi.org/10.1371/journal.pone.0118568 e0118568.
• [13] Massad LS, Jeronimo J, Katki HA, Schiffman M. The accuracy of colposcopic grading for detection of high-grade cervical intraepithelial neoplasia. J Low Genit Tract Dis 2009;13:137.
• [14] Massad LS, Jeronimo J, Schiffman M. Interobserver agreement in the assessment of components of colposcopic grading. Obstet Gynecol 2008;111:1279–84.
• [15] Jusman Y, Ng SC, Abu Osman NA. Intelligent screening systems for cervical cancer. SciWorld J 2014;2014. https://doi.org/10.1155/2014/810368.
• [16] Sato M, Horie K, Hara A, Miyamoto Y, Kurihara K, Tomio K, et al. Application of deep learning to the classification of images from colposcopy. Oncol Lett 2018;15:3518–23.
• [17] Woo S, Park J, Lee JY, Kweon IS. CBAM: Convolutional block attention module. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics), vol. 11211 LNCS, 2018, p. 3–19. 10.1007/978-3-030-01234-2_1.
• [18] Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design. Proc IEEE/CVF Conf Comput Vis Pattern Recognit 2021:13713–22.
• [19] Ma C-Y, Zhou J-Y, Xu X-T, Guo J, Han M-F, Gao Y-Z, et al. Deep learning-based auto-segmentation of clinical target volumes for radiotherapy treatment of cervical cancer. J Appl Clin Med Phys 2021.
• [20] Rigaud B, Anderson BM, Zhiqian HY, Gobeli M, Cazoulat G, Söderberg J, et al. Automatic segmentation using deep learning to enable online dose optimization during adaptive radiation therapy of cervical cancer. Int J Radiat Oncol Biol Phys 2021;109:1096–110.
• [21] Liu Z, Chen W, Guan H, Zhen H, Shen J, Liu X, et al. An adversarial deep-learning-based model for cervical cancer CTV segmentation with multicenter blinded randomized controlled validation. Front Oncol 2021:3223.
• [22] Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60–88. https://doi.org/10.1016/j.media.2017.07.005.
• [23] Xu T, Xin C, Long LR, Antani S, Xue Z, Kim E, et al. A new image data set and benchmark for cervical dysplasia classification evaluation. Lect Notes Comput Sci (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9352, 2015, p. 26–35. 10.1007/978-3-319-24888-2_4.
• [24] Ji Q, Engel J, Craine E. Classifying cervix tissue patterns with texture analysis. Pattern Recognit 2000;33:1561–73. https://doi.org/10.1016/S0031-3203(99)00123-5.
• [25] Xu T, Kim E, Huang X. Adjustable adaboost classifier and pyramid features for image-based cervical cancer diagnosis. Proc. - Int. Symp. Biomed. Imaging, vol. 2015- July, 2015, p. 281–5. 10.1109/ISBI.2015.7163868.
• [26] Acosta-Mesa HG, Cruz-Ramírez N, Hernández-Jiménez R. Aceto-white temporal pattern classification using k-NN to identify precancerous cervical lesion in colposcopic images. Comput Biol Med 2009;39:778–84. https://doi.org/10.1016/j.compbiomed.2009.06.006.
• [27] Park SY, Sargent D, Lieberman R, Gustafsson U. Domain-specific image analysis for cervical neoplasia detection based on conditional random fields. IEEE Trans Med Imaging 2011;30:867–78. https://doi.org/10.1109/TMI.2011.2106796.
• [28] Devi MA, Ravi S, Vaishnavi J, Punitha S. Classification of cervical cancer using artificial neural networks. Procedia Comput Sci 2016;89:465–72. https://doi.org/10.1016/j.procs.2016.06.105.
• [29] Catarino R, Schäfer S, Vassilakos P, Petignat P, Arbyn M. Accuracy of combinations of visual inspection using acetic acid or lugol iodine to detect cervical precancer: a meta-analysis. BJOG Int J Obstet Gynaecol 2018;125:545–53. https://doi.org/10.1111/1471-0528.14783.
• [30] Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 1998;20:1254–9. https://doi.org/10.1109/34.730558.
• [31] Liu J, Wang G, Duan LY, Abdiyeva K, Kot AC. Skeleton-based human action recognition with global context-aware attention LSTM networks. IEEE Trans Image Process 2018;27:1586–99. https://doi.org/10.1109/TIP.2017.2785279.
• [32] Chen H, Li Y. Three-stream attention-aware network for RGBD salient object detection. IEEE Trans Image Process 2019;28:2825–35. https://doi.org/10.1109/TIP.2019.2891104.
• [33] Hu C, Wang Y. An efficient convolutional neural network model based on object-level attention mechanism for casting defect detection on radiography images. IEEE Trans Ind Electron 2020;67:10922–30. https://doi.org/10.1109/TIE.2019.2962437.
• [34] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. Proc IEEE Conf Comput Vis pattern Recognit 2018:7132–41.
• [35] Erhan D, Bengio Y, Courville A, Vincent P. Visualizing higher-layer features of a deep network. Bernoulli 2009;1341:1–13.
• [36] Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, 2016, p. 2921–9. 10.1109/CVPR.2016.319.
• [37] Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis 2020;128:336–59. https://doi.org/10.1007/s11263-019-01228-7.
• [38] Tan M, Le Q V. EfficientNet: Rethinking model scaling for convolutional neural networks. In: 36th Int. Conf. Mach. Learn. ICML 2019, vol. 2019- June, 2019, p. 10691–700.
• [39] Zagoruyko S, Komodakis N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. 5th Int Conf Learn Represent ICLR - Conf Track Proc, 2017.
• [40] Müller R, Kornblith S, Hinton G. When does label smoothing help? Adv Neural Inf Process Syst 2019;32.
• [41] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016- Decem, 2016, p. 2818–26. 10.1109/CVPR.2016.308.
• [42] Pang T, Xu K, Dong Y, Du C, Chen N, Zhu J. Rethinking Softmax cross-entropy loss for adversarial robustness. ArXiv Prepr ArXiv190510626 2019.
• [43] Taylor ME, Stone P. Transfer learning for reinforcement learning domains: A survey. J Mach Learn Res 2009;10:1633–85.
• [44] Bottou L. Stochastic gradient descent tricks. Lect Notes Comput Sci (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 7700 LECTU, Springer; 2012, p. 421–36. 10.1007/978-3-642-35289-8_25.
• [45] Loshchilov I, Hutter F. SGDR: Stochastic gradient descent with warm restarts. 5th Int Conf Learn Represent ICLR - Conf Track Proc 2017.
• [46] Guo Z, Li X, Huang H, Quo N, Li Q. Medical image segmentation based on multi-modal convolutional neural network: Study on image fusion schemes. In: Proc. - Int. Symp. Biomed. Imaging, vol. 2018- April, 2018, p. 903–7. 10.1109/ISBI.2018.8363717.
• [47] Ursuleanu TF, Luca AR, Gheorghe L, Grigorovici R, Iancu S, Hlusneac M, et al. The Use of Artificial intelligence on segmental volumes, constructed from MRI and CT images, in the diagnosis and staging of cervical cancers and thyroid cancers: a study protocol for a randomized controlled trial. J Biomed Sci Eng 2021;14:300–4. https://doi.org/10.4236/jbise.2021.146025.
• [48] Chandran V, Sumithra P, Karthick A, Subramanian M. Diagnosis of cervical cancer based on ensemble deep learning network using colposcopy images. Biomed Res Int 2021.
• [49] Xue P, Man T, Qiao Y. The challenges of colposcopy for cervical cancer screening in LMICs and solutions by artificial intelligence. BMC Med 2020;18.
• [50] Zhao Y, Li Y, Xing L, Lei H, Chen D, Tang C, et al. The performance of artificial intelligence in cervical colposcopy: a retrospective data analysis. J Oncol 2022;2022:4370851. https://doi.org/10.1155/2022/4370851.
• [51] Cho BJ, Choi YJ, Lee MJ, Ju HK, Park ST. Classification of cervical neoplasms on colposcopic photography using deep learning. Sci Rep 2020;10.
• [52] Park YR, Kim YJ, Ju W, Nam K, Kim S, Kim KG. Comparison of machine and deep learning for the classification of cervical cancer based on cervicography images. Sci Rep 2021;11:1–11.
• [53] Peng G, Dong H, Liang T, Li L, Liu J. Diagnosis of cervical precancerous lesions based on multimodal feature changes. Comput Biol Med 2021;130. https://doi.org/10.1016/j.compbiomed.2021.104209 104209.
• [54] Yuan C, Yao Y, Cheng B, Cheng Y, Li Y, Li Y, et al. The application of deep learning based diagnostic system to cervical squamous intraepithelial lesions recognition in colposcopy images. Sci Rep 2020;10:1–12. https://doi.org/10.1038/s41598-020-68252-3.
• [55] Yan L, Li S, Guo Y, Ren P, Song H, Yang J, et al. Multi-state colposcopy image fusion for cervical precancerous lesion diagnosis using {BF-CNN}. Biomed Signal Process Control 2021;68. https://doi.org/10.1016/j.bspc.2021.102700 102700.
• [56] Ito Y, Miyoshi A, Ueda Y, Tanaka Y, Nakae R, Morimoto A, et al. An artificial intelligence-assisted diagnostic system improves the accuracy of image diagnosis of uterine cervical lesions. Mol Clin Oncol 2022;16:27. https://doi.org/10.3892/mco.2021.2460.
• [57] Jones HW. Colposcopic accuracy of obstetrics and gynecology residents: Commentary. Obstet Gynecol Surv 2007;62:175–6. https://doi.org/10.1097/01.ogx.0000256806.44109.bc.
• [58] Slimani O, Ben TR, Makhlouf T, Mathlouthi N, Attia L. Cytocolpo-histologic correlation: About an analytical study of 120 colposcopies. Tunisie Medicale 2016;94:616–20.
• [59] Adsul P, Manjunath N, Srinivas V, Arun A, Madhivanan P. Implementing community-based cervical cancer screening programs using visual inspection with acetic acid in India: A systematic review. Cancer Epidemiol 2017;49:161–74. https://doi.org/10.1016/j.canep.2017.06.008.