FRE-Net: Full-region enhanced network for nuclei segmentation in histopathology images

• Autorzy: Huang Xuping , Chen Junxi , Chen Mingzhi , Wan Yaping , Chen Lingna

Identyfikatory

DOI

10.1016/j.bbe.2023.02.002

• Języki publikacji: EN

Abstrakty

EN

Accurate nuclei segmentation is a critical step for physicians to achieve essential information about a patient’s disease through digital pathology images, enabling an effective diagnosis and evaluation of subsequent treatments. Since pathology images contain many nuclei, manual segmentation is time-consuming and error-prone. Therefore, developing a precise and automatic method for nuclei segmentation is urgent. This paper proposes a novel multi-task segmentation network that incorporates background and contour segmentation into the nuclei segmentation method and produces more accurate segmentation results. The convolution and attention modules are merged with the model to increase its global focus and enhance good segmentation results indirectly. We propose a reverse feature enhance module for contour extraction that facilitates feature integration between auxiliary tasks. The multi-feature fusion module is embedded in the final decoding branch to use different levels of features from auxiliary segmentation branches with varying concerns. We evaluate the proposed method on four challenging nuclei segmentation datasets. The proposed method achieves excellent performance on all four datasets. We found that the Dice coefficient reached 0.8563±0.0323, 0.8183±0.0383, 0.9222±0.0216, and 0.9220±0.0602 on the TNBC, MoNuSeg, KMC, and Glas. Our method produces better boundary accuracy and less sticking than other end-to-end segmentation methods. The results show that our method can perform better than other proposed state-of-the-art methods.

Słowa kluczowe

EN

histopathology image; nuclei segmentation; CNN; convolutional neural network

PL

obraz histopatologiczny; segmentacja jądra; CNN; sieć neuronowa konwolucyjna

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2023
• Tom: Vol. 43, no. 1
• Strony: 386--401
• Opis fizyczny: Bibliogr. 52 poz., rys., tab., wykr.

Twórcy

autor

Huang Xuping

• Computer School, University of South China, Hengyang, China

autor

Chen Junxi

• Affiliated Nanhua Hospital, University of South China, Hengyang, China

autor

Chen Mingzhi

• School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China

autor

Wan Yaping

• Computer School, University of South China, Hengyang, China

autor

Chen Lingna

• Computer School, University of South China, Hengyang 421001, China
• Xinjiang Institute of Technology, Xinjiang, China

Bibliografia

• [1] Nir G, Hor S, Karimi D, Fazli L, Skinnider BF, Tavassoli P, et al. Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts. Medical Image Anal 2018;50:167-80.
• [2] Pan X, Li L, Yang H, Liu Z, Yang J, Zhao L, et al. Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks. Neurocomputing 2017;229:88-99.
• [3] Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Systems Man Cybernet 1979;9(1):62-6.
• [4] Yang X, Li H, Zhou X. Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy. IEEE Trans Circuits Syst I Regul Pap 2006;53(11):2405-14.
• [5] Filipczuk P, Kowal M, Obuchowicz A. Automatic breast cancer diagnosis based on k-means clustering and adaptive thresholding hybrid segmentation. In: IP&C.
• [6] Rother C, Kolmogorov V, Blake A. grabcut interactive foreground extraction using iterated graph cuts. ACM Trans Graphics (TOG) 2004;23(3):309-14.
• [7] Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, et al. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans Med Imag 2019;38(10):2281-92.
• [8] Wang Q, Li X, Chen M, Chen L, Chen J. A regularization-driven mean teacher model based on semi-supervised learning for medical image segmentation. Phys Med Biol 2022;67(17):175010.
• [9] Xie Y, Zhang J, Shen C, Xia Y. Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer; 2021. p. 171-80.
• [10] Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recogn 2018;77:354-77.
• [11] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer; 2015. p. 234-41.
• [12] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Informat Process Syst 2017;30.
• [13] Dai Z, Liu H, Le QV, Tan M. Coatnet: Marrying convolution and attention for all data sizes. Adv Neural Informat Process Syst 2021;34:3965-77.
• [14] Cao Y, Liu S, Peng Y, Li J. Denseunet: densely connected unet for electron microscopy image segmentation. IET Image Proc 2020;14(12):2682-9.
• [15] Ali H, Cui L, Feng J, et al. Msal-net: improve accurate segmentation of nuclei in histopathology images by multiscale attention learning network. BMC Med Inform Decis Mak 2022;22(1):1-11.
• [16] Dinh TL, Kwon S-G, Lee S-H, Kwon K-R. Breast tumor cell nuclei segmentation in histopathology images using efficientunet++ and multi-organ transfer learning. J Korea Multimedia Soc 2021;24(8):1000-11.
• [17] Singh P, Cirrone J, A data-efficient deep learning framework for segmentation and classification of histopathology images, 2022. arXiv preprint arXiv:2207.06489.
• [18] Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Medical Imag 2019;39(6):1856-67.
• [19] Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, et al. Unet 3+: A full-scale connected unet for medical image segmentation. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020. p. 1055-9.
• [20] Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, et al. Transunet: Transformers make strong encoders for medical image segmentation, 2021. arXiv preprint arXiv:2102.04306.
• [21] Luo H, Changdong Y, Selvan R. Hybrid ladder transformers with efficient parallel-cross attention for medical image segmentation. In: Medical Imaging with Deep Learning, 2021.
• [22] Çayır S, Solmaz G, Kusetogullari H, Tokat F, Bozaba E, Karakaya S, et al. Mitnet: a novel dataset and a two-stage deep learning approach for mitosis recognition in whole slide images of breast cancer tissue. Neural Comput Appl 2022:1–15.
• [23] Rehman MU, Akhtar S, Zakwan M, Mahmood MH. Novel architecture with selected feature vector for effective classification of mitotic and non-mitotic cells in breast cancer histology images. Biomed Signal Process Control 2022;71:103212.
• [24] Mathew T, Ajith B, Kini JR, Rajan J. Deep learning-based automated mitosis detection in histopathology images for breast cancer grading. Int J Imag Syst Technol 2022.
• [25] Ahmad S, Ullah T, Ahmad I, Al-Sharabi A, Ullah K, Khan RA, et al. A novel hybrid deep learning model for metastatic cancer detection. Comput Intell Neurosci 2022.
• [26] Guatemala-Sanchez VR, Peregrina-Barreto H, Lopez-Armas G. Nuclei segmentation on histopathology images of breast carcinoma. 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE; 2021. p. 2622-8.
• [27] Guo P, Evans A, Bhattacharya P. Segmentation of nuclei in digital pathology images. 2016 IEEE 15th international conference on cognitive informatics & cognitive computing (ICCI* CC). IEEE; 2016. p. 547-50.
• [28] Hassan T, Javed S, Mahmood A, Qaiser T,Werghi N, Rajpoot N. Nucleus classification in histology images using message passing network. Med Image Anal 2022:102480.
• [29] Lin Y, Qu Z, Chen H, Gao Z, Li Y, Xia L, et al. Label propagation for annotation-efficient nuclei segmentation from pathology images, 2022. arXiv preprint arXiv:2202.08195.
• [30] Wetstein SC, de Jong VM, Stathonikos N, Opdam M, Dackus GM, Pluim JP, et al. Deep learning-based breast cancer grading and survival analysis on whole-slide histopathology images. Sci Rep 2022;12(1):1-12.
• [31] Khan R, Debnath R. Morphology preserving segmentation method for occluded cell nuclei from medical microscopy image. Int J Image Graph 2022;22(02):2250019.
• [32] Kalinathan L, Kathavarayan RS. Segmentation of multiple nuclei from non-overlapping immuno-histochemically stained histological hepatic images. J Digit Imaging 2022:1-9.
• [33] Hu H, Qiao S, Hao Y, Bai Y, Cheng R, Zhang W, et al. Breast cancer histopathological images recognition based on two-stage nuclei segmentation strategy. Plos One 2022;17(4): e0266973.
• [34] Mahmood F, Borders D, Chen RJ, McKay GN, Salimian KJ, Baras A, et al. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE Trans Medical Imag 2019;39(11):3257-67.
• [35] Kiran I, Raza B, Ijaz A, Khan MA. Denseres-unet: Segmentation of overlapped/clustered nuclei from multi organ histopathology images. Comput Biol Med 2022;143:105267.
• [36] Chanchal AK, Lal S, Kini J. High-resolution deep transferred asppu-net for nuclei segmentation of histopathology images. Int J Comput Assisted Radiol Surg 2021;16(12):2159-75.
• [37] Lu Y, Qin X, Fan H, Lai T, Li Z. Wbc-net: A white blood cell segmentation network based on unet++ and resnet. Appl Soft Comput 2021;101:107006.
• [38] Chanchal AK, Lal S, Kini J. Deep structured residual encoder-decoder network with a novel loss function for nuclei segmentation of kidney and breast histopathology images. Multimedia Tools Appl 2022;81(7):9201-24.
• [39] Lal S, Das D, Alabhya K, Kanfade A, Kumar A, Kini J. Nucleisegnet: robust deep learning architecture for the nuclei segmentation of liver cancer histopathology images. Comput Bol Med 2021;128:104075.
• [40] Li X, Pi J, Lou M, Qi Y, Li S, Meng J, et al. Multi-level feature fusion network for nuclei segmentation in digital histopathological images. The Visual Comput 2022:1-16.
• [41] Qu H, Yan Z, Riedlinger GM, De S, Metaxas DN. Improving nuclei/gland instance segmentation in histopathology images by full resolution neural network and spatial constrained loss. In: International conference on medical image computing and computer-assisted intervention. Springer; 2019. p. 378-86.
• [42] Aatresh AA, Yatgiri RP, Chanchal AK, Kumar A, Ravi A, Das D, et al. Efficient deep learning architecture with dimension-wise pyramid pooling for nuclei segmentation of histopathology images. Comput Med Imaging Graph 2021;93:101975.
• [43] Rahmon G, Toubal IE, Palaniappan K. Extending u-net network for improved nuclei instance segmentation accuracy in histopathology images. 2021 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). IEEE; 2021. p. 1-7.
• [44] Liu X, Guo Z, Cao J, Tang J. Mdc-net: a new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information. Comput Biol Med 2021;135:104543.
• [45] Qin J, He Y, Zhou Y, Zhao J, Ding B. Reu-net: Region-enhanced nuclei segmentation network. Comput Biol Med 2022;146:105546.
• [46] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. pp. 770-778.
• [47] Liu H, Liu F, Fan X, Huang D. Polarized self-attention: towards high-quality pixel-wise regression, 2021, arXiv preprint arXiv:2107.00782.
• [48] Milletari F, Navab N, Ahmadi S-A. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 fourth international conference on 3D vision (3DV). IEEE; 2016. p. 565-71.
• [49] Naylor P, Laé M, Reyal F, Walter T. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Medical Imag 2018;38(2):448-59.
• [50] Sirinukunwattana K, Pluim JP, Chen H, Qi X, Heng P-A, Guo YB, et al. Gland segmentation in colon histology images: The glas challenge contest. Medical Image Anal 2017;35:489-502.
• [51] Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Medical Imag 2017;36(7):1550-60.
• [52] Xiang T, Zhang C, Liu D, Song Y, Huang H, Cai W. Bio-net: learning recurrent bi-directional connections for encoder-decoder architecture. In: International conference on medical image computing and computer-assisted intervention. Springer; 2020. p. 74-84.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023)