A semi-automated annotation algorithm based on weakly supervised learning for medical images

• Autorzy: Li Hailiang , Zhang Bin , Zhang Yu , Liu Weiwei , Mao Yijun , Huang Jiacheng , Wei Linfeng

Identyfikatory

DOI

10.1016/j.bbe.2020.03.005

• Języki publikacji: EN

Abstrakty

EN

For medical image recognition, deep learning requires a massive training set, while anno-tation work is a tedious and time-consuming process because of the high technical thresh-old. Furthermore, it is difficult to guarantee annotation accuracy due to the knowledge, skills, and status of the annotator. In this research, we propose a semi-automated annota- tion model based on weakly supervised learning. Moreover, a target-level annotation method is proposed based on weakly supervised learning that is guided by machine learning. The machine learning method is used to screen the regions of interest (RoIs), whose semantic feature vectors are extracted by the deep learning method. Then, the machine learning method is used to cluster them, and the RoIs are finally classified and labeled by a distance comparison. Therefore, this model achieves target-level semi-auto- mated annotation by only using image-level annotations. We applied this method to ultrasound imaging of thyroid papillary carcinoma. The experiments demonstrate the potential of this new methodology to reduce the workload for pathologists and increase the objectivity of diagnoses. We find that 89.8% of papillary thyroid carcinoma regions can be detected automatically, while 82.6% of benign and normal tissue can be excluded without the use of any additional immunohistochemical markers or human intervention.

Słowa kluczowe

EN

deep learning; machine learning; target level annotation; weakly supervised learning; thyroid carcinoma

PL

uczenie głębokie; uczenie maszynowe; uczenie nadzorowane; nowotwór tarczycy

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2020
• Tom: Vol. 40, no. 2
• Strony: 787--802
• Opis fizyczny: Bibliogr. 38 poz., rys., tab., wykr.

Twórcy

autor

Li Hailiang

• College of Information Science and Technology/College of Cyber Security, Jinan University, Guangzhou, Guangdong, China

autor

Zhang Bin

• College of Information Science and Technology/College of Cyber Security, Jinan University, Guangzhou, Guangdong, China

autor

Zhang Yu

• College of Information Science and Technology/College of Cyber Security, Jinan University, Guangzhou, Guangdong, China

autor

Liu Weiwei

• Sun Yat-sen University Cancer Center, Guanzhou, China

autor

Mao Yijun

• College of Mathematics and Informatics, South China Agricultural University, Guanzhou, China

autor

Huang Jiacheng

• College of Mathematics and Informatics, South China Agricultural University, Guanzhou, China

autor

Wei Linfeng

• College of Information Science and Technology/College of Cyber Security, Jinan University, No.601, West Huangpu Avenue, Guangzhou, Guangdong, China

Bibliografia

• [1] Randle RW, Balentine CJ, Leverson GE, Havlena JA, Sippel RS, Schneider DF, et al. Trends in the presentation, treatment, and survival of patients with medullary thyroid cancer over the past 30 years. Surgery 2017;161(1):137–46.
• [2] Cameron A. The metabolic syndrome: prevalence in worldwide populations. endocrinology and metabolism clinics of North America. Metabol Syndr 2004;1.
• [3] Brander MDA, Viikinkoski MDP, Tuuhea MDJ, Voutilainen MDL, Kivisaari MDL. Clinical versus ultrasound examination of the thyroid gland in common clinical practice. J Clin Ultrasound 1992;20(1):37–42.
• [4] Ahuja AT, Chow L, Chick W, King W, Metreweli C. Metastatic cervical nodes in papillary carcinoma of the thyroid: ultrasound and histological correlation. Clin Radiol 1995;50(4):229–31.
• [5] Zhao R, Yan R, Chen Z, Mao K, Wang P, Gao RX. Deep learning and its applications to machine health monitoring. Mech Syst Signal Process 2019;115:213–37.
• [6] Faust O, Acharya UR, Meiburger KM, Molinari F, Ng KH. Comparative assessment of texture features for the identification of cancer in ultrasound images: a review. Biocybern Biomed Eng 2018;38(2).
• [7] Li H, Weng J, Shi Y, Gu W, Mao Y, Wang Y, et al. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci Rep 2018;8(1):1–12.
• [8] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436.
• [9] Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, et al. A guide to deep learning in healthcare. Nat Med 2019;25(1):24.
• [10] Razavian AS, Azizpour H, Sullivan J, Carlsson S. CNN features off-the-shelf: an astounding baseline for recognition. Comput Vision Pattern Recogn 2014;512–9.
• [11] Li H, Huang Y, Zhang Z. An improved faster R-CNN for same object retrieval. IEEE Access 2017;5:13665–76.
• [12] Lin D, Dai J, Jia J, He K, Sun J, Scribblesup:. Scribble-supervised convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition 2016;3159–67.
• [13] Pathak D, Krahenbuhl P, Darrell T. Constrained convolutional neural networks for weakly supervised segmentation. IEEE International Conference on Computer Vision 2015;1796–804.
• [14] Papandreou G, Chen LC, Murphy K, Yuille AL. Weakly- and semi-supervised learning of a DCNN for semantic image segmentation. Comput Vision Pattern Recogn 2015;1742–50.
• [15] Xu J, Schwing AG, Urtasun R. Learning to segment under various forms of weak supervision. IEEE Conference on Computer Vision and Pattern Recognition 2015;3781–90.
• [16] Kwak S, Hong S, Han B. Weakly supervised semantic segmentation using superpixel pooling network. AAAI 2017;4111–7.
• [17] Oquab M, Bottou L, Laptev I, Sivic J. Weakly supervised object recognition with convolutional neural networks. Vision Par Ordinateur Et Reconnaissance De Formes 2014.
• [18] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 2014;PP(99):1.
• [19] Souza RPW, Srgio DF, Bicalho Luciano MD, Gatti AMF, Purisch Saulo MD, Lanza PE, et al. Ultrasonographic differentiation between metastatic and benign lymph nodes in patients with papillary thyroid carcinoma. J Ultrasound Med 2005;24(10):1385–9.
• [20] Chan BK, Desser TS, Mcdougall IR, Weigel RJ. Common and uncommon sonographic features of papillary thyroid carcinoma. J Ultrasound Med Off J Am Inst Ultrasound Med 2003;22(10):1083–90.
• [21] Choi JS, Kim J, Kwak JY, Kim MJ, Chang HS, Kim EK. Preoperative staging of papillary thyroid carcinoma: comparison of ultrasound imaging and CT. AJR Am J Roentgenol 2009;193(3):871–8.
• [22] Kim E, Park JS, Son KR, Kim JH, Jeon SJ, Na DG. Preoperative diagnosis of cervical metastatic lymph nodes in papillary thyroid carcinoma: comparison of ultrasound, computed tomography, and combined ultrasound with computed tomography. Thyroid Off J Am Thyroid Assoc 2008;18(4):411.
• [23] Frates MC, Benson CB, Charboneau JW, Cibas ES, Clark OH, Coleman BG, et al. Management of thyroid nodules detected at us: society of radiologists in ultrasound consensus conference statement. Radiology 2005;237 (3):794.
• [24] Crandall DJ, Huttenlocher DP. Weakly supervised learning of part-based spatial models for visual object recognition. European Conference on Computer Vision 2006;16–29.
• [25] Fergus R, Perona P, Zisserman A. Weakly supervised scale-invariant learning of models for visual recognition. Int J Comput Vis 2007;71(3):273–303.
• [26] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. International Conference on Neural Information Processing Systems 2015;91–9.
• [27] Edelkamp S, Schrdl S. Selective search; 2012.
• [28] Bahmani B, Moseley B, Vattani A, Kumar R, Vassilvitskii S. Scalable k-means++. Proc VLDB Endowment 2012;5 (7):622–33.
• [29] Ng AY, Jordan MI, Weiss Y. On spectral clustering: analysis and an algorithm. Proc Nips 2001;14:849–56.
• [30] Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Proceedings of the IEEE International Conference on Computer Vision 2015;1440–8.
• [31] Exhaustive Search. Encyclopedia of genetics genomics proteomics & informatics; 2008;653.
• [32] Gabrilovich E, Markovitch S. Computing semantic relatedness using wikipedia-based explicit semantic analysis. Proceedings of the International Joint Conference on Artificial Intelligence 2016;1606–11.
• [33] Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, et al. Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 2016;35(5):1299–312.
• [34] Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. Imagenet: a large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition 2009;248–55.
• [35] Hartigan JA, Wong MA. Algorithm as 136: a k-means clustering algorithm. J R Stat Soc Ser C Appl Stat 1979;28 (1):100–8.
• [36] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems 2012.
• [37] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. MIT Press; 1988.
• [38] Girshick R. Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision 2015;1440–8.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).