Semi-supervised structure attentive temporal mixup coherence for medical image segmentation

• Autorzy: Pawan S.J. , Jeevan Govind , Rajan Jeny

Identyfikatory

DOI

10.1016/j.bbe.2022.09.005

• Języki publikacji: EN

Abstrakty

EN

Deep convolutional neural networks have shown eminent performance in medical image segmentation in supervised learning. However, this success is predicated on the availability of large volumes of pixel-level labeled data, making these approaches impractical when labeled data is scarce. On the other hand, semi-supervised learning utilizes pertinent information from unlabeled data along with minimal labeled data, alleviating the demand for labeled data. In this paper, we leverage the mixup-based risk minimization operator in a student-teacher-based semi-supervised paradigm along with structure-aware constraints to enforce consistency coherence among the student predictions for unlabeled samples and the teacher predictions for the corresponding mixup sample by significantly diminishing the need for labeled data. Besides, due to the intrinsic simplicity of the linear combination operation used for generating mixup samples, the proposed method stands at a computational advantage over existing consistency regularization-based SSL methods. We experimentally validate the performance of the proposed model on two public benchmark datasets, namely the Left Atrial (LA) and Automatic Cardiac Diagnosis Challenge (ACDC) datasets. Notably, on the LA dataset’s lowest labeled data set-up (5%), the proposed method significantly improved the Dice Similarity Coefficient and the Jaccard Similarity Coefficient by 1.08% and 1.46%, respectively. Furthermore, we demonstrate the efficacy of the proposed method with a consistent improvement across various labeled data proportions on the aforementioned datasets.

Słowa kluczowe

EN

convolutional neural network; semi supervised learning; consistency regularization

PL

sieć neuronowa konwolucyjna; uczenie częściowo nadzorowane; regularyzacja spójności

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2022
• Tom: Vol. 42, no. 4
• Strony: 1149--1161
• Opis fizyczny: Bibliogr. 78 poz., rys., tab.

Twórcy

autor

Pawan S.J.

• Department of Computer Science and Engineering, National Institute of Technology Karnataka, Surathkal, Mangalore - 575025, Karnataka, India

autor

Jeevan Govind

• Department of Computer Science and Engineering, National Institute of Technology Karnataka, Surathkal, Mangalore, Karnataka, India

autor

Rajan Jeny

• Department of Computer Science and Engineering, National Institute of Technology Karnataka, Surathkal, Mangalore, Karnataka, India

Bibliografia

• [1] Youm Chenyu, et al. Unsupervised wasserstein distance guided domain adaptation for 3D multi-domain liver segmentation. In Interpretable and Annotation-Efficient Learning for Medical Image Computing. Springer; 2020. pp. 155–163.
• [2] Yang Linfeng et al. NuSeT: A deep learning tool for reliably separating and analyzing crowded cells. PLoS Comput Biol 2020;16(9) e1008193.
• [3] Dong Shunjie et al. DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation. Med Image Anal 2022;78 102389.
• [4] You Chenyu, et al. CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GANCIRCLE). IEEE Trans Med Imaging 2019;39(1):188–203.
• [5] Guha Indranil et al. Deep learning based high-resolution reconstruction of trabecular bone microstructures from low-resolution CT scans using GAN-CIRCLE. Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging, vol. 11317. International Society for Optics and Photonics; 2020. p. 113170U.
• [6] Minaee Shervin, et al. Image Segmentation Using Deep Learning: A Survey. IEEE Trans Med Imaging 2021:1–1.
• [7] Aljuaid Abeer, Anwar Mohd. Survey of Supervised Learning for Medical Image Processing. SN Comput Sci 2022;3(4):1–22.
• [8] Chen Xuxin, et al. Recent advances and clinical applications of deep learning in medical image analysis. Med Image Anal 2022:102444.
• [9] Guan Hao, Liu Mingxia. Domain adaptation for medical image analysis: a survey. IEEE Trans Biomed Eng 2021;69(3):1173–85.
• [10] You Chenyu et al. Structurally-sensitive multi-scale deep neural network for low-dose CT denoising. IEEE Access 2018;6:41839–55.
• [11] You Chenyu et al. Class-aware generative adversarial transformers for medical image segmentation. In: arXiv preprint arXiv:2201.10737.
• [12] Shi Jialin, Guo Chenyi, Wu Ji. A Hybrid Robust-Learning Architecture for Medical Image Segmentation with Noisy Labels. Future Internet 2022;14(2):41.
• [13] Brahim Khawla et al. An Improved 3D Deep Learning-Based Segmentation of Left Ventricular Myocardial Diseases from Delayed-Enhancement MRI with Inclusion and Classification Prior Information U-Net (ICPIU-Net). Sensors 2022;22(6):2084.
• [14] You Chenyu, et al. Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation. arXiv preprint arXiv:2206.01369; 2022.
• [15] Varoquaux Gaël, Cheplygina Veronika. Machine learning for medical imaging: methodological failures and recommendations for the future. NPJ Digital Med 2022;5(1):1–8.
• [16] Kaissis Georgios, et al. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nat Mach Intell 2021;3(6):473–484.
• [17] Thiagarajan Jayaraman J et al. Training calibration-based counterfactual explainers for deep learning models in medical image analysis. Scientific Rep 2022;12(1):1–15.
• [18] Li Shuailin, Zhang Chuyu, He Xuming. Shape-aware semi-supervised 3d semantic segmentation for medical images. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020. pp. 552–561.
• [19] Luo Xiangde et al. Semi-supervised medical image segmentation through dual-task consistency. In: arXiv preprint arXiv:2009.04448.
• [20] Bortsova Gerda, et al. Semi-supervised medical image segmentation via learning consistency under transformations. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. pp. 810–818.
• [21] Sajjadi Mehdi, Javanmardi Mehran, Tasdizen Tolga. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Advances in neural information processing systems 2016;29:1163–71.
• [22] Shorten Connor, Khoshgoftaar Taghi M. A survey on image data augmentation for deep learning. J Big Data 2019;6(1):1–48.
• [23] Wang Shuihua et al. Advances in data preprocessing for biomedical data fusion: an overview of the methods, challenges, and prospects. Inf Fusion 2021;76:376–421.
• [24] Khalifa Nour Eldeen, Loey Mohamed, Mirjalili Seyedali. A comprehensive survey of recent trends in deep learning for digital images augmentation. Artif Intell Rev 2021:1–27.
• [25] Zhang Chunling, et al. A Deep Learning Image Data Augmentation Method for Single Tumor Segmentation. Front Oncol 2022:297.
• [26] Ghosh Atin, Thiery Alexandre H. On data-augmentation and consistency-based semi-supervised learning. In: arXiv preprint arXiv:2101.06967.
• [27] Verma Vikas et al. Interpolation consistency training for semi-supervised learning. Neural Networks 2022;145:90–106.
• [28] Miyato Takeru, et al. Virtual adversarial training: a regularization method for supervised and semisupervised learning. IEEE Trans Med Imaging 2018;41(8):1979–1993.
• [29] Ronneberger Olaf, Fischer Philipp, Brox Thomas. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer; 2015. pp. 234–241.
• [30] Milletari Fausto, Navab Nassir, Ahmadi Seyed-Ahmad. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 fourth international conference on 3D vision (3DV). IEEE 2016:565–71.
• [31] Isaksson Lars J et al. Mixup (Sample Pairing) Can Improve the Performance of Deep Segmentation Networks. J Arif Intell Soft Comput Res 2022;12(1):29–39.
• [32] Chen Jintai, et al. Flow-Mixup: Classifying multi-labeled medical images with corrupted labels. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE; 2020. pp. 534–541.
• [33] Galdran Adrian, Carneiro Gustavo, González Ballester Miguel A. Balanced-MixUp for Highly Imbalanced Medical Image Classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer. p. 323–33.
• [34] Guo Xiaoqing et al. Non-equivalent images and pixels: Confidence-aware resampling with metalearning mixup for polyp segmentation. Med Image Anal 2022;78 102394.
• [35] Lee Dong-Hyun, et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML. vol. 3; 2013. p. 896.
• [36] Li Ruizhe et al. IA generic ensemble based deep convolutional neural network for semi-supervised medical image segmentation. 2020 EEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE 2020:1168–72.
• [37] Li Yuexiang, et al. Self-loop uncertainty: A novel pseudo-label for semi-supervised medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020. pp. 614–623.
• [38] Sedai Suman, et al. Uncertainty guided semi-supervised segmentation of retinal layers in OCT images. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. pp. 282–290.
• [39] Zhang Wenqiao, et al. BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation. arXiv preprint arXiv:2203.02533; 2022.
• [40] Wu Chongshu et al. Segmentation of HE-stained meningioma pathological images based on pseudolabels. In PloS one 2022;17(2) e0263006.
• [41] Souly Nasim, Spampinato Concetto, Shah Mubarak. Semi supervised semantic segmentation using generative adversarial network. In: Proceedings of the IEEE International Conference on Computer Vision. p. 5688–96.
• [42] Ma Yuxin et al. Self-Supervised Vessel Segmentation via Adversarial Learning. In: Proceedings of the IEEE/CVFInternational Conference on Computer Vision. p. 7536–45.
• [43] Zhang Yizhe, et al. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In International conference on medical image computing and computer-assisted intervention. Springer; 2017. pp. 408–416.
• [44] Hung Wei-Chih et al. Adversarial learning for semi-supervised semantic segmentation. In: arXiv preprint arXiv:1802.07934.
• [45] Yu Lequan, et al. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. pp. 605–613.
• [46] Huynh Tri, Nibali Aiden, He Zhen. Semi-supervised learning for medical image classification using imbalanced training data. In Computer Methods and Programs in Biomedicine; 2022. p. 106628.
• [47] Calderon-Ramirez Saul, et al. A real use case of semi-supervised learning for mammogram classification in a local clinic of Costa Rica. Med Biolo Eng Comput 2022;60(4):1159–1175.
• [48] Wu Yue, et al. Lesion Localization in OCT by Semi-Supervised Object Detection. arXiv preprint arXiv:2204.11227; 2022.
• [49] Zhou Hong-Yu et al. SSMD: semi-supervised medical image detection with adaptive consistency and heterogeneous perturbation. Med Image Anal 2021;72 102117.
• [50] Ouali Yassine, Hudelot Céline, Tami Myriam. Semi-supervised semantic segmentation with crossconsistency training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 12674–84.
• [51] Chen Zailiang, et al. Dual-consistency semi-supervision combined with self-supervision for vessel segmentation in retinal OCTA images. Biomed Opt Express 2022;13(5):2824–2834.
• [52] Liu Zihang, Zhao Chunhui. Semi-supervised Medical Image Segmentation via Geometry-aware Consistency Training. arXiv preprint arXiv:2202.06104; 2022.
• [53] Chaitanya Krishna, et al. Contrastive learning of global and local features for medical image segmentation with limited annotations. In Advances in Neural Information Processing Systems. Ed. by H. Larochelle et al. Vol. 33. Curran Associates Inc; 2020. pp. 12546–12558. URL: https://proceedings.neurips.cc/paper/2020/file/949686ecef4ee20a62d16b4a2d7ccca3-Paper.pdf.
• [54] Yu Hua et al. Uncertainty-Guided Voxel-Level Supervised Contrastive Learning for Semi-Supervised Medical Image Segmentation. Int J Neural Syst 2022;32(04):2250016.
• [55] You Chenyu, et al. Simcvd: Simple contrastive voxel-wise representation distillation for semi-supervised medical image segmentation. IEEE Trans Med Imag 2022.
• [56] You Chenyu et al. Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation. In: arXiv preprint arXiv:2105.07059.
• [57] You Chenyu, et al. Bootstrapping Semi-supervised Medical Image Segmentation with Anatomicalaware Contrastive Distillation. arXiv preprint arXiv:2206.02307; 2022.
• [58] Luo Xiangde, et al. Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer. arXiv preprint arXiv:2112.04894; 2021.
• [59] Bdair Tariq, et al. ROAM: Random layer mixup for semi-supervised learning in medical images. In IET Image Processing; 2022.
• [60] Wang Kaiping et al. Semi-supervised medical image segmentation via a tripled-uncertainty guided mean teacher model with contrastive learning. Med Image Anal 2022;79102447.
• [61] Zhang Yuhan et al. Twin self-supervision based semi-supervised learning (TS-SSL): Retinal anomaly classification in SD-OCT images. Neurocomputing 2021;462:491–505.
• [62] Huang Yifei, et al. Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data. arXiv preprint arXiv:2112.01034; 2021.
• [63] Geirhos Robert, et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net; 2019.
• [64] Ma Jun, et al. How Distance Transform Maps Boost Segmentation CNNs: An Empirical Study. In Medical Imaging with Deep Learning. Ed. by Tal Arbel et al. Vol. 121. Proceedings of Machine Learning Research. PMLR; 2020. pp. 479–492.
• [65] Tarvainen Antti, Valpola Harri. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems. Ed. by I. Guyon et al. Vol. 30. Curran Associates Inc; 2017.
• [66] Laine Samuli, Aila Timo. Temporal Ensembling for Semi-Supervised Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. OpenReview.net; 2017.
• [67] Zhang Hongyi, et al. mixup: Beyond Empirical Risk Minimization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30-May 3, 2018, Conference Track Proceedings. OpenReview.net; 2018.
• [68] Tobon-Gomez Catalina et al. Benchmark for Algorithms Segmenting the Left Atrium From 3D CT and MRI Datasets. IEEE Trans Med Imag 2015;34(7):1460–73.
• [69] Bernard Olivier, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE Trans Med Imag 2018;37(11):2514–2525.
• [70] Bai Wenjia, et al. Semi-supervised learning for network-based cardiac MR image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2017. pp. 253–260.
• [71] Xiangde Luo. SSL4MIS. https://github.com/HiLab-git/SSL4MIS; 2020.
• [72] Dice Lee R. Measures of the amount of ecologic association between species. Ecology 1945;26(3):297–302.
• [73] Navarro Fernando, et al. Shape-Aware Complementary-Task Learning for Multi-organ Segmentation. Oct. 2019. pp. 620–627. isbn: 978-3-030-32691-3. doi: 10.1007/978-3-030-32692-0_71.
• [74] Kervadec Hoel, et al. Boundary loss for highly unbalanced segmentation. In Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning. Ed. by M. Jorge Cardoso et al. Vol. 102. Proceedings of Machine Learning Research. PMLR; 2019. pp. 285–296. URL: https://proceedings.mlr.press/v102/kervadec19a.html.
• [75] Karimi Davood, Salcudean Septimiu E. Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks. IEEE Trans Med Imag 2020;39(2):499–513. doi: 10.1109/TMI.2019.2930068.
• [76] Vu T, et al. ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, 2019, pp. 2512–2521. doi: 10.1109/CVPR.2019. 00262.
• [77] Li Shuailin, Zhang Chuyu, He Xuming. Shape-Aware Semi-supervised 3D Semantic Segmentation for Medical Images. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Ed. by Anne L. Martel et al. Cham: Springer International Publishing; 2020. pp. 552–561. isbn: 978-3-030-59710-8.
• [78] Luo Xiangde, et al. Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2021. pp. 318–329.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023)