MDCF_Net: A Multi-dimensional hybrid network for liver and tumor segmentation from CT

• Autorzy: Jiang Jian , Peng Yanjun , Hou Qingfan , Wang Jiao

Identyfikatory

DOI

10.1016/j.bbe.2023.04.004

• Języki publikacji: EN

Abstrakty

EN

The segmentation of the liver and liver tumors is critical in the diagnosis of liver cancer, and the high mortality rate of liver cancer has made it one of the most popular areas for segmentation research. Some deep learning segmentation methods outperformed traditional methods in terms of segmentation results. However, they are unable to obtain satisfactory segmentation results due to blurred original image boundaries, the presence of noise, very small lesion sites, and other factors. In this paper, we propose MDCF_Net, which has dual encoding branches composed of CNN and CnnFormer and can fully utilize multidimensional image features. First, it extracts both intra-slice and inter-slice information and improves the accuracy of the network output by symmetrically using multidimensional fusion layers. In the meantime, we propose a novel feature map stacking approach that focuses on the correlation of adjacent channels of two feature maps, improving the network’s ability to perceive 3D features. Furthermore, the two coding branches collaborate to obtain both texture and edge features, and the network segmentation performance is further improved. Extensive experiments were carried out on the public datasets LiTS to determine the optimal slice thickness for this task. The superiority of the segmentation performance of our proposed MDCF_Net was confirmed by comparison with other leading methods on two public datasets, the LiTS and the 3DIRCADb.

Słowa kluczowe

EN

liver tumor segmentation; multidimensional fusion; convolutional network; deep learning

PL

guz wątroby; fuzja wielowymiarowa; sieć konwolucyjna; uczenie głębokie

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2023
• Tom: Vol. 43, no. 2
• Strony: 494--506
• Opis fizyczny: Bibliogr. 41 poz., rys., tab.

Twórcy

autor

Jiang Jian

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China

autor

Peng Yanjun

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China
• Shandong Province Key Laboratory of Wisdom Mining Information Technology, Qingdao, China

autor

Hou Qingfan

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China

autor

Wang Jiao

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China

Bibliografia

• [1] Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021;71:209-49. https://doi.org/10.3322/caac.21660.
• [2] Lin L, Yang W, Li C, Tang J, Cao X. Inference with collaborative model for interactive tumor segmentation in medical image sequences. IEEE Trans Cybern 2016;46:2796-809. https://doi. org/10.1109/TCYB.2015.2489719.
• [3] Jiang H, Shi T, Bai Z, Huang L. AHCNet: an application of attention mechanism and hybrid connection for liver tumor segmentation in CT volumes. IEEE Access 2019;7:24898-909. https://doi.org/10.1109/ACCESS.2019.2899608.
• [4] Li L, Wang J, Lu W, Tan S. Simultaneous tumor segmentation, image restoration, and blur kernel estimation in PET using multiple regularizations. Comput Vis Image Underst 2017;155:173-94. https://doi.org/10.1016/j.cviu.2016.10.002.
• [5] Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. https://doi. org/10.1016/j.media.2017.07.005.
• [6] Anter AM, Hassenian AE. CT liver tumor segmentation hybrid approach using neutrosophic sets, fast fuzzy c-means and adaptive watershed algorithm. Artif Intell Med 2019;97:105-17. https://doi.org/10.1016/j.artmed.2018.11.007.
• [7] Abd-Elaziz OF, Sayed MS, Abdullah MI. Liver tumors segmentation from abdominal CT images using region growing and morphological processing. In: 2014 Int. Conf. Eng. Technol. ICET, 2014, p. 1-6. https://doi.org/10.1109/ ICEngTechnol.2014.7016813.
• [8] Kuo C-L, Cheng S-C, Lin C-L, Hsiao K-F, Lee S-H. Texture-based treatment prediction by automatic liver tumor segmentation on computed tomography. In: 2017 Int. Conf. Comput. Inf. Telecommun. Syst. CITS, 2017, p. 128-32. https://doi.org/10.1109/CITS.2017.8035318.
• [9] Gul S, Khan MS, Bibi A, Khandakar A, Ayari MA, Chowdhury MEH. Deep learning techniques for liver and liver tumor segmentation: a review. Comput Biol Med 2022;147. https:// doi.org/10.1016/j.compbiomed.2022.105620 105620.
• [10] Shimizu A, Narihira T, Furukawa D, Kobatake H, Nawano S, Shinozaki K. Ensemble segmentation using AdaBoost with application to liver lesion extraction from a CT volume. MIDAS J 2008. https://doi.org/10.54294/wrtw01.
• [11] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Med. Image Comput. Comput.-Assist. Interv. – MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234-41. https://doi.org/10.1007/978-3-319-24574-4_28.
• [12] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is All you Need n.d.:11.
• [13] Han K, Xiao A, Wu E, Guo J, Xu C, Wang Y. Transformer in Transformer, 2021.
• [14] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: transformers for image recognition at scale, 2022.
• [15] Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, et al. TransUNet: transformers make strong encoders for medical image segmentation; 2021.
• [16] Yu W, Luo M, Zhou P, Si C, Zhou Y, Wang X, et al. IEEECVF Conf Comput Vis Pattern Recognit CVPR 2022;2022:10809-19. https://doi.org/10.1109/CVPR52688.2022.01055.
• [17] Bilic P, Christ PF, Vorontsov E, Chlebus G, Chen H, Dou Q, et al. The liver tumor segmentation benchmark (LiTS) 2019.
• [18] Soler L, Hostettler A, Agnus V, Charnoz A, Fasquel J, Moreau J, et al. 3D image reconstruction for comparison of algorithm database: a patient-specific anatomical and medical image database. IRCAD: Strasbourg. Tech Rep; 2010.
• [19] Asgari Taghanaki S, Abhishek K, Cohen JP, Cohen-Adad J, Hamarneh G. Deep semantic segmentation of natural and medical images: a review. Artif Intell Rev 2021;54:137-78. https://doi.org/10.1007/s10462-020-09854-1.
• [20] Chlebus G, Schenk A, Moltz JH, van Ginneken B, Hahn HK, Meine H. Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep 2018;8:15497. https://doi.org/10.1038/s41598-018-33860-7.
• [21] Vorontsov E, Tang A, Pal C, Kadoury S. Liver lesion segmentation informed by joint liver segmentation. In: 2018 IEEE 15th Int. Symp. Biomed. Imaging ISBI 2018. Washington, DC: IEEE; 2018. p. 1332-5. https://doi.org/10.1109/ ISBI.2018.8363817.
• [22] Kushnure DT, Talbar SN. MS-UNet: A multi-scale UNet with feature recalibration approach for automatic liver and tumor segmentation in CT images. Comput Med Imaging Graph 2021;89. https://doi.org/10.1016/j.compmedimag.2021.101885 101885.
• [23] Chen X, Zhang R, Yan P. Feature Fusion Encoder Decoder Network for Automatic Liver Lesion Segmentation. In: 2019 IEEE 16th Int. Symp. Biomed. Imaging ISBI 2019. Venice, Italy: IEEE; 2019. p. 430-3. https://doi.org/10.1109/ISBI.2019.8759555.
• [24] Dou Q, Yu L, Chen H, Jin Y, Yang X, Qin J, et al. 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal 2017;41:40-54. https://doi.org/10.1016/j.media.2017.05.001.
• [25] Yang D, Xu D, Zhou SK, Georgescu B, Chen M, Grbic S, et al. Automatic Liver Segmentation Using an Adversarial Imageto-Image Network. In: Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S, editors. Med. Image Comput. Comput. Assist. Interv. MICCAI 2017, vol. 10435, Cham: Springer International Publishing; 2017, p. 507-15. https://doi.org/10.1007/978-3-319-66179-7_58.
• [26] Abdollahi A, Pradhan B, Alamri A. VNet: an end-to-end fully convolutional neural network for road extraction from highresolution remote sensing data. IEEE Access 2020;8:179424-36. https://doi.org/10.1109/ACCESS.2020.3026658.
• [27] Li X, Chen H, Qi X, Dou Q, Fu C-W, Heng P-A. HDenseUNet: hybrid densely connected unet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging 2018;37:2663-74. https://doi.org/10.1109/TMI.2018.2845918.
• [28] Dong Z, He Y, Qi X, Chen Y, Shu H, Coatrieux J-L, et al. MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation. vol. 2, 2022, p. 870-6. https://doi.org/10.24963/ijcai.2022/122.
• [29] Jin Q, Meng Z, Sun C, Cui H, Su R. RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans. Front Bioeng Biotechnol 2020:8.
• [30] Tian J, Liu L, Shi Z, Xu F. Automatic Couinaud Segmentation from CT Volumes on Liver Using GLC-UNet. In: Suk H-I, Liu M, Yan P, Lian C, editors. Mach. Learn. Med. Imaging, Cham: Springer International Publishing; 2019, p. 274-82. https://doi.org/10.1007/978-3-030-32692-0_32.
• [31] Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, et al. SwinUnet: Unet-like Pure Transformer for Medical Image Segmentation 2021.
• [32] Di S, Zhao Y, Liao M, Zhang F, Li X. TD-Net: a hybrid end-toend network for automatic liver tumor segmentation from CT images. IEEE J Biomed Health Inform 2022:1-10. https:// doi.org/10.1109/JBHI.2022.3181974.
• [33] Li L, Ma H. RDCTrans U-Net: a hybrid variable architecture for liver CT image segmentation. Sensors 2022;22:2452. https:// doi.org/10.3390/s22072452.
• [34] Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, et al. A survey on vision transformer. IEEE Trans Pattern Anal Mach Intell 2022:1. https://doi.org/10.1109/TPAMI.2022.3152247.
• [35] Loshchilov I, Hutter F. Decoupled Weight Decay Regularization, 2022.
• [36] Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Adv. Neural Inf. Process. Syst., vol. 32, Curran Associates, Inc.; 2019.
• [37] Seo H, Huang C, Bassenne M, Xiao R, Xing L. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images. IEEE Trans Med Imaging 2020;39:1316-25. https:// doi.org/10.1109/TMI.2019.2948320.
• [38] Pang S, Du A, Orgun MA, Wang Y, Yu Z. Tumor attention networks: Better feature selection, better tumor segmentation. Neural Netw 2021;140:203-22. https://doi.org/ 10.1016/j.neunet.2021.03.006.
• [39] Hong L, Wang R, Lei T, Du X, Wan Y. Qau-Net: Quartet Attention U-Net for Liver and Liver-Tumor Segmentation. In:2021 IEEE Int. Conf. Multimed. Expo ICME, 2021, p. 1-6. https://doi.org/10.1109/ICME51207.2021.9428427.
• [40] Song L, Wang H, Wang ZJ. Bridging the gap between 2D and 3D contexts in CT volume for liver and tumor segmentation. IEEE J Biomed Health Inform 2021;25:3450-9. https://doi.org/ 10.1109/JBHI.2021.3075752.
• [41] Wang J, Lv P, Wang H, Shi C. SAR-U-Net: squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in computed tomography. Comput Methods Programs Biomed 2021;208. https://doi.org/10.1016/j.cmpb.2021.106268 106268.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023)