BA-Net: Brightness prior guided attention network for colonic polyp segmentation

• Autorzy: Xia Haiying , Qin Yilin , Tan Yumei , Song Shuixiang

Identyfikatory

DOI

10.1016/j.bbe.2023.08.001

• Języki publikacji: EN

Abstrakty

EN

Automatic polyp segmentation at colonoscopy plays an important role in the early diagnosis and surgery of colorectal cancer. However, the diversity of polyps in different images greatly increases the difficulty of accurately segmenting polyps. Manual segmentation of polyps in colonoscopic images is time-consuming and the rate of polyps missed remains high. In this paper, we propose a brightness prior guided attention network (BA-Net) for automatic polyp segmentation. Specifically, we first aggregate the high-level features of the last three layers of the encoder with an enhanced receptive field (ERF) module, which further fed to the decoder to obtain the initial prediction maps. Then, we introduce a brightness prior fusion (BF) module that fuses the brightness prior information into the multi-scale side-out high-level semantic features. The BF module aims to induce the network to localize salient regions, which may be potential polyps, to obtain better segmentation results. Finally, we propose a global reverse attention (GRA) module to combine the output of the BF module and the initial prediction map for obtaining long-range dependence and reverse refinement prediction results. With iterative refinement from higher-level semantics to lower-level semantics, our BA-Net can achieve more refined and accurate segmentation. Extensive experiments show that our BA-Net outperforms the stateof-the-art methods on six common polyp datasets.

Słowa kluczowe

EN

deep learning; computer aided diagnosis; colonic polyp segmentation; global reverse attention; brightness prior knowledge

PL

uczenie głębokie; diagnoza wspomagana komputerowo; polip

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2023
• Tom: Vol. 43, no. 3
• Strony: 603--615
• Opis fizyczny: Bibliogr. 48 poz., rys., tab.

Twórcy

autor

Xia Haiying

• College of Electronics Engineering, Guangxi Normal University, Guilin, China

autor

Qin Yilin

• College of Electronics Engineering, Guangxi Normal University, Guilin, China

autor

Tan Yumei

• School of Computer Science and Engineering, Guangxi Normal University, Guilin, China

autor

Song Shuixiang

• College of Electronics Engineering, Guangxi Normal University, Guilin 541004, China

Bibliografia

• [1] Silva J, Histace A, Romain O, Dray X, Granado B. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg 2014;9(2):283-93. https://doi.org/10.1007/s11548-013-0926-3.
• [2] Sánchez-Peralta LF, Bote-Curiel L, Picón A, Sánchez-Margallo FM, Pagador JB. Deep learning to find colorectal polyps in colonoscopy: A systematic literature review. Artif Intell Med 2020:101923. https://doi.org/10.1016/j.artmed.2020.101923.
• [3] Stewart BW, Kleihues P, et al., World cancer report. 2003.
• [4] Ferlay J, Colombet M, Soerjomataram I, Dyba T, Randi G, Bettio M, et al. Cancer incidence and mortality patterns in europe: Estimates for 40 countries and 25 major cancers in 2018. Eur J Cancer 2018;103:356-87. https://doi.org/10.1016/j. Ejca.2012.12.027.
• [5] Senthilkumaran N, Vaithegi S. Image segmentation by using thresholding techniques for medical images. Comput Sci Eng: Int J 2016;6(1):1-13.
• [6] Padmapriya B, Kesavamurthi T, Ferose HW. Edge based image segmentation technique for detection and estimation of the bladder wall thickness. Proc Eng 2012;30:828-35. https://doi. org/10.1016/j.proeng.2012.01.934.
• [7] Wang XQ, Wang XB, Huang XL. Image segmentation based on wavelet transform. Advanced Materials Research, vol. 225. Trans Tech Publ; 2011. p. 1041-4. https://doi.org/10.4028/ www.scientific.net/AMR.225-226.1041.
• [8] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015. p. 3431-40.
• [9] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer; 2015. p. 234-41. https://doi.org/10.1007/978-3-319-24574-4_28.
• [10] Fang Y, Chen C, Yuan Y, Tong K-Y. Selective feature aggregation network with area-boundary constraints for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. p. 302-10. https://doi.org/10.1007/978-3-030-32239-7_34.
• [11] Zhang R, Li G, Li Z, Cui S, Qian D, Yu Y. Adaptive context selection for polyp segmentation. In: International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer; 2020. p. 253-62. https://doi.org/10.1007/978-3-030-59725-2_25.
• [12] Patel K, Bur AM, Wang G. Enhanced u-net: A feature enhancement network for polyp segmentation. In: 2021 18th Conference on Robots and Vision (CRV). IEEE; 2021. p. 181-8. https://doi.org/10.1109/CRV52889.2021.00032.
• [13] Nguyen T-C, Nguyen T-P, Diep G-H, Tran-Dinh A-H, Nguyen TV, Tran M-T. Ccbanet: Cascading context and balancing attention for polyp segmentation. In: International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer; 2021. p. 633-43. https://doi. Org/10.1007/978-3-030-87193-2_60.
• [14] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p. 7132-41.
• [15] Konwar AS, Borah BS, Tuithung C. An american sign language detection system using hsv color model and edge detection. In: 2014 International Conference on Communication and Signal Processing. IEEE; 2014. p. 743-7. https://doi.org/ 10.1109/ICCSP.2014.6949942.
• [16] Mirikharaji Z, Hamarneh G. Star shape prior in fully convolutional networks for skin lesion segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2018. p. 737-45. https://doi.org/10.1007/978-3-030-00937-3_84.
• [17] Zotti C, Luo Z, Lalande A, Jodoin P-M. Convolutional neural network with shape prior applied to cardiac mri segmentation. IEEE J Biomed Health Informat 2018;23 (3):1119-28. https://doi.org/10.1109/JBHI.2018.2865450.
• [18] Grau V, Mewes A, Alcaniz M, Kikinis R, Warfield SK. Improved watershed transform for medical image segmentation using prior information. IEEE Trans Med Imag 2004;23(4):447-58. https://doi.org/10.1109/TMI.2004.824224.
• [19] Chen J, Jha AK, Frey EC. Incorporating ct prior information in the robust fuzzy c-means algorithm for qspect image segmentation. Medical Imaging 2019: Image Processing, vol. 10949. International Society for Optics and Photonics; 2019. p. 109491W. https://doi.org/10.1117/12.2506805.
• [20] Han Y, Zhang S, Geng Z, Wei Q, Ouyang Z. Level set based shape prior and deep learning for image segmentation. IET Image Proc 2020;14(1):183-91. https://doi.org/10.1049/ietipr.2018.6622.
• [21] Xi X, Shi H, Han L, Wang T, Ding HY, Zhang G, et al. Breast tumor segmentation with prior knowledge learning. Neurocomputing 2017;237:145-57. https://doi.org/10.1016/j. Neucom.2016.09.067.
• [22] Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD. Doubleu-net: A deep convolutional neural network for medical image segmentation. 2020 IEEE 33rd International Symposium on Computer-based Medical Systems (CBMS). IEEE; 2020. p. 558-64. https://doi.org/10.1109/CBMS49503.2020.00111.
• [23] Chen L-C, Papandreou G, Schroff F, Adam H. Rethinking atrous convolution for semantic image segmentation, arXiv preprint arXiv:1706.05587; 2017.
• [24] Srivastava A, Jha D, Chanda S, Pal U, Johansen HD, Johansen D, et al. Msrf-net: A multi-scale residual fusion network for biomedical image segmentation. IEEE J Biomed Health Informat 2021;26(5):2252-63. https://doi.org/10.1109/ JBHI.2021.3138024.
• [25] Srivastava A, Chanda S, Jha D, Pal U, Ali S. Gmsrf-net: An improved generalizability with global multi-scale residual fusion network for polyp segmentation. In: 2022 26th International Conference on Pattern Recognition (ICPR). IEEE; 2022. p. 4321-7. https://doi.org/10.1109/ICPR56361.2022.9956726.
• [26] Kim T, Lee H, Kim D. Uacanet: Uncertainty augmented context attention for polyp segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia; 2021. p. 2167-75, https://doi.org/10.1145/3474085.3475375.
• [27] Wei J, Hu Y, Zhang R, Li Z, Zhou SK, Cui S. Shallow attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer Assisted Intervention. Springer; 2021. p. 699-708. https://doi. Org/10.1007/978-3-030-87193-2_66.
• [28] Huang Q, Xia C, Wu C, Li S, Wang Y, Song Y, Kuo C-CJ. Semantic segmentation with reverse attention, arXiv preprint arXiv:1707.06426, 2017.
• [29] Chen S, Tan X, Wang B, Lu H, Hu X, Fu Y. Reverse attention-based residual network for salient object detection. IEEE Trans Image Process 2020;29:3763-76. https://doi.org/10.1109/ TIP.2020.2965989.
• [30] Fan D-P, Ji G-P, Zhou T, Chen G, Fu H, Shen J, et al. Pranet: Parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020. p. 263-73. https://doi.org/10.1007/978-3-030-59725-2_26.
• [31] Ates GC, Mohan P, Celik E. Dual cross-attention for medical image segmentation, arXiv preprint arXiv:2303.17696, 2023.
• [32] Gao S-H, Cheng M-M, Zhao K, Zhang X-Y, Yang M-H, Torr P. Res2net: A new multi-scale backbone architecture. IEEE Trans Pattern Anal Machine Intell 2019;43(2):652-62. https://doi.org/ 10.1109/TPAMI.2019.2938758.
• [33] Wu Z, Su L, Huang Q. Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition; 2019. p. 3907-16.
• [34] Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. Unet++: A nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer; 2018. p. 3-11. https://doi.org/10.1007/978-3-030-00889-5_1.
• [35] Wang X, Girshick R, Gupta A, He K. Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p. 7794-803.
• [36] Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F. Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph 2015;43:99-111. https://doi.org/10.1016/j.compmedimag.2015.02.007.
• [37] Vázquez D, Bernal J, Sánchez FJ, Fernández-Esparrach G, López AM, Romero A, et al. A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthcare Eng 2017. https://doi.org/10.1155/2017/4037190.
• [38] Bernal J, Sánchez J, Vilarino F. Towards automatic polyp detection with a polyp appearance model. Pattern Recogn 2012;45(9):3166-82. https://doi.org/10.1016/ j.patcog.2012.03.002.
• [39] Silva J, Histace A, Romain O, Dray X, Granado B. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comput Assisted Radiol Surg 2014;9(2):283-93. https://doi.org/10.1007/s11548-013-0926-3.
• [40] Tajbakhsh N, Gurudu SR, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Medical Imag 2015;35(2):630-44. https://doi.org/10.1109/TMI.2015.2487997.
• [41] Jha D, Smedsrud PH, Riegler MA, Halvorsen P, de Lange T, Johansen D, et al. Kvasir-seg: A segmented polyp dataset. In: International Conference on Multimedia Modeling. Springer; 2020. p. 451-62. https://doi.org/10.1007/978-3-030-37734-2_37.
• [42] Margolin R, Zelnik-Manor L, Tal A. How to evaluate foreground maps? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2014. p. 248-55.
• [43] Fan D-P, Cheng M-M, Liu Y, Li T, Borji A. Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision; 2017. p. 4548-57.
• [44] Fan D-P, Ji G-P, Qin X, Cheng M-M. Cognitive vision inspired object segmentation metric and loss function. Sci Sin Informat 2021;6(6).
• [45] Ji G-P, Chou Y-C, Fan D-P, Chen G, Fu H, Jha D, Shao L, Progressively normalized self-attention network for video polyp segmentation. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27- October 1, 2021, Proceedings, Part I 24, Springer; 2021. p. 142-152, https://doi.org/10.1007/978-3-030-87193-2_14.
• [46] Wu H, Zhong J, Wang W, Wen Z, Qin J. Precise yet efficient semantic calibration and refinement in convnets for real-time polyp segmentation from colonoscopy videos. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35; 2021. p. 2916-24, https://doi.org/10.1609/aaai.v35i4.16398.
• [47] Ji G-P, Xiao G, Chou Y-C, Fan D-P, Zhao K, Chen G, et al. Video polyp segmentation: A deep learning perspective. Mach Intell Res 2022:1-19. https://doi.org/10.1007/s11633-022-1371-y.
• [48] Duc NT, Oanh NT, Thuy NT, Triet TM, Dinh VS. Colonformer: an efficient transformer based method for colon polyp segmentation. IEEE Access 2022;10:80575-86. https://doi.org/10.1109/ACCESS.2022.3195241.