Simultaneous detection and diagnosis of mammogram mass using bilateral analysis and soft label based metric learning

• Autorzy: Ma Yingran , Peng Yanjun

Identyfikatory

DOI

10.1016/j.bbe.2022.01.001

• Języki publikacji: EN

Abstrakty

EN

In the clinics, mammogram masses appear as asymmetric structures between the left and right breasts. In this paper, we design a bilateral image analysis method based on convolutional neural network which can detect and classify breast mass regions simultaneously. It mainly consists of three parts: a feature similarity based region matching technique, mass region of interest (ROI) selection step and a deep metric learning based classifier. Firstly, discriminative score maps are calculated relied on the deep features extracted from bilateral left and right mammograms respectively in global or local spatial image domain. The contralateral correspondences are determined by minimum discriminative scores. Secondly, to select the mass candidate ROIs and further remove false positive mass-tonormal pairs, we propose a dynamic histogram weighting mechanism with three new constrains imposed on the distribution of discriminative score histogram. In addition, a novel soft label based deep metric learning regularization is designed for mass ROI classifier to tackle the large variation of masses in shape, size, texture and breast density. We apply it to the open dataset Digital Database for Screening Mammography. Compared with other state-of-the-art approaches, the proposed scheme gives competitive results in classification and localization tasks for mammographic lesions.

Słowa kluczowe

EN

mammogram mass detection; bilateral analysis; convolutional neural network; deep metric learning

PL

mammografia; analiza bilateralna; sieć neuronowa konwolucyjna; uczenie głębokie

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2022
• Tom: Vol. 42, no. 1
• Strony: 215--232
• Opis fizyczny: Bibliogr. 65 poz., rys., tab., wykr.

Twórcy

autor

Ma Yingran

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, Shandong Province, PR China

autor

Peng Yanjun

• College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, Shandong Province, PR China; Shandong Province Key Laboratory of Wisdom Mining Information Technology, Shandong University of Science and Technology, Qingdao, Shandong Province, PR China

Bibliografia

• [1] Jiao Z, Gao X, Wang Y, Li J. A parasitic metric learning net for breast mass classification based on mammography. Pattern Recogn 2018;75:292–301.
• [2] Nahid A-A, Mehrabi MA, Kong Y. Histopathological breast cancer image classification by deep neural network techniques guided by local clustering. Biomed Res Int 2018;2018:1–20.
• [3] Xu Q, Wang X, Jiang H. Convolutional neural network for breast cancer diagnosis using diffuse optical tomography. Visual Comput Ind Biomed Art 2019;2(1):1.
• [4] Li Y, Chen H, Yang Y, Yang N. Pectoral muscle segmentation in mammograms based on homogenous texture and intensity deviation. Pattern Recogn 2013;46(3):681–91.
• [5] Ertosun MG, Rubin DL. Probabilistic visual search for masses within mammography images using deep learning. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine 2015: 1310-1315.
• [6] Kopans DB. Breast imaging. Philadelphia: Lippincott Williams & Wilkins; 2006.
• [7] Whitman GJ, Haygood TM. Mammographic interpretation: a practical approach. New York: Cambridge University Press; 2012.
• [8] Li Y, Zhang L, Chen H, Cheng L. Mass detection in mammograms by bilateral analysis using convolution neural network. Comput Methods Programs Biomed 2020;195:105518.
• [9] Scutt D, Lancaster GA, Manning JT. Breast asymmetry and predisposition to breast cancer. Breast Cancer Res 2006;8 (2):1–7.
• [10] Li Y, Chen H, Yang Y, Cheng L, Cao L. A bilateral analysis scheme for false positive reduction in mammogram mass detection. Comput Biol Med 2015;57:84–95.
• [11] Casti P, Mencattini A, Salmeri M, Rangayyan RM. Analysis of structural similarity in mammograms for detection of bilateral asymmetry. IEEE Trans Med Imaging 2015;34 (2):662–71.
• [12] Šerifović-Trbalić A, Demirović D, Prljača N, Szekely G, Cattin PC. Intensity-based elastic registration incorporating anisotropic landmark errors and rotational information. Int J Comput Assist Radiol Surg 2009;4(5):463–8.
• [13] Kalra PK, Kumar N. An automatic image registration scheme using Tsallis entropy. Biomed Signal Process Control 2010;5 (4):328–35.
• [14] Celaya-Padilla J, Martinez-Torteya A, Rodriguez-Rojas J, Galvan-Tejada J, Treviño V, Tamez-Peña J. Bilateral image subtraction and multivariate models for the automated triaging of screening mammograms. Biomed Res Int 2015;2015:1–12.
• [15] Bandeira Diniz JO, Bandeira Diniz PH, Azevedo Valente TL, Corrêa Silva A, de Paiva AC, Gattass M. Detection of mass regions in mammograms by bilateral analysis adapted to breast density using similarity indexes and convolutional neural networks. Comput Methods Programs Biomed 2018;156:191–207.
• [16] Xu S, Liu H, Xu X, et al. Bilateral asymmetry detection in mammograms using non-rigid registration and pseudo-color coding. In: IEEE International Conference on Electrical and Control Engineering; 2010: 544-547.
• [17] Brandt SS, Karemore G, Karssemeijer N, Nielsen M. An anatomically oriented breast coordinate system for mammogram analysis. IEEE Trans Med Imaging 2011;30 (10):1841–51.
• [18] Méndez AJ, Tahoces PG, Lado MJ, Souto M, Vidal JJ. Computeraided diagnosis: automatic detection of malignant masses in digitized mammograms. Med Phys 1998;25(6):957–64.
• [19] Stamatakis EA, Ricketts IW, Cairns AY, Walker C, Preece PE. Detecting abnormalities on mammograms by bilateral comparison. IEE colloquium on digital mammography, 1996.
• [20] Wirth MA, Choi C, Jennings A. A nonrigid-body approach to matching mammograms. In: 7th International Conference on Image Processing and its Applications; 1999: 484-488.
• [21] Han X, Leung T, Jia Y, et al. MatchNet: Unifying feature and metric learning for patch-based matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 3279-3286.
• [22] Georgsson F. Differential analysis of bilateral mammograms. Int J Pattern Recognit Artif Intell 2003;17(07):1207–26.
• [23] Bovis K, Singh S. Detection of masses in mammograms using texture features. In: Proceedings 15th International Conference on Pattern Recognition; 2000, 2: 267-270.
• [24] Maitra IK, Nag S. Bilateral breast asymmetry detection using intensity histogram and landmark based registration technique. Int J Emerg Sci 2012;2(3):383–404.
• [25] Wu Y-T, Wei J, Hadjiiski LM, Sahiner B, Zhou C, Ge J, et al. Bilateral analysis based false positive reduction for computer-aided mass detection. Med Phys 2007;34 (8):3334–44.
• [26] Shiotsuki K, Matsunobu Y, Yabuuchi H, et al. Usefulness of histogram analysis of spatial frequency components for exploring the similarity and bilateral asymmetry in mammograms. Medical imaging 2015: computer-aided diagnosis, international society for optics and photonics, 2015.
• [27] Dhahbi S, Barhoumi W, Zagrouba E. Multi-view score fusion for content-based mammogram retrieval. 8th International Conference on Machine Vision, International Society for Optics and Photonics, 2015.
• [28] Kim DH, Kim ST, Ro YM. Latent feature representation with 3-D multi-view deep convolutional neural network for bilateral analysis in digital breast tomosynthesis. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.
• [29] Wang X, Girshick R, Gupta A, et al. Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7794-7803.
• [30] Rocco I, Arandjelovic R, Sivic J. Convolutional neural network architecture for geometric matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017: 6148-6157.
• [31] Tian Y, Fan B, Wu F. L2-net: Deep learning of discriminative patch descriptor in Euclidean space. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017: 661-669.
• [32] Ustinova E, Lempitsky V. Learning deep embeddings with histogram loss. arXiv preprint arXiv:1611.00822; 2016.
• [33] Wang Z, Li H, Ouyang W, et al. Learnable histogram: Statistical context features for deep neural networks. In: European Conference on Computer Vision; 2016: 246-262.
• [34] Jiao Z, Gao X, Wang Y, Li J. A deep feature based framework for breast masses classification. Neurocomputing 2016;197:221–31.
• [35] Cheng G, Yang C, Yao X, Guo L, Han J. When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Trans Geosci Remote Sens 2018;56(5):2811–21.
• [36] Tzeng E, Hoffman J, Darrell T, et al. Simultaneous deep transfer across domains and tasks. In: Proceedings of the IEEE International Conference on Computer Vision; 2015: 4068-4076.
• [37] Bakalo R, Goldberger J, Ben-Ari R. A dual branch deep neural network for classification and detection in mammograms. arXiv preprint arXiv:1904.12589, 2019.
• [38] Wu CY, Manmatha R, Smola AJ, et al. Sampling matters in deep embedding learning. In: Proceedings of the IEEE International Conference on Computer Vision; 2017: 2840-2848.
• [39] Schroff F, Kalenichenko D, Philbin J. Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 815-823.
• [40] Hariharan B, Arbeláez P, Girshick R, et al. Hypercolumns for object segmentation and fine-grained localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 447-456.
• [41] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 3431-3440.
• [42] Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39 (12):2481–95.
• [43] Yu R, Saito S, Li H, et al. Learning dense facial correspondences in unconstrained images. In: Proceedings of the IEEE International Conference on Computer Vision; 2017: 4723-4732.
• [44] Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 2018;40(4):834–48.
• [45] Bowyer K, Kopans D, Kegelmeyer WP, et al. The digital database for screening mammography. Third international workshop on digital mammography, 1996.
• [46] Gallego-Posada J, Montoya-Zapata DA, Quintero-Montoya OL, et al. Detection and diagnosis of breast tumors using deep convolutional neural networks. Conference Proceedings of XVII Latin American Conference in Automatic Control, 2016.
• [47] Geras KJ, Wolfson S, Shen Y, et al. High-resolution breast cancer screening with multi-view deep convolutional neural networks. arXiv preprint arXiv:1703.07047, 2017.
• [48] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Systems 2012;25:1097–105.
• [49] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
• [50] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 2818-2826.
• [51] Carneiro G, Nascimento J, Bradley AP. Automated analysis of unregistered multi-view mammograms with deep learning. IEEE Trans Med Imaging 2017;36(11):2355–65.
• [52] Agarwal R, Diaz O, Lladó X, Yap MH, Martí R. Automatic mass detection in mammograms using deep convolutional neural networks. J Med Imaging 2019.
• [53] Van der Maaten L, Hinton G. Visualizing data using t-SNE. J Machine Learn Res 2008;9(11):2579–605.
• [54] Dhungel N, Carneiro G, Bradley AP. Automated mass detection in mammograms using cascaded deep learning and random forests. In: International Conference on Digital Image Computing: Techniques and Applications. p. 1–8.
• [55] Wang Y, Choi J, Morariu V, et al. Mining discriminative triplets of patches for fine-grained classification. In: In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. p. 1163–72.
• [56] Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision. p. 1520–8.
• [57] Wu J, Zhou B, Peck D, et al. Deepminer: Discovering interpretable representations for mammogram classification and explanation. arXiv preprint arXiv:1805.12323, 2018.
• [58] Al-masni MA, Al-antari MA, Park J-M, Gi G, Kim TY, Rivera P, et al. Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. Comput Methods Programs Biomed 2018;157:85–94.
• [59] Shen L, Margolies LR, Rothstein JH, Fluder E, McBride R, Sieh W. Deep learning to improve breast cancer detection on screening mammography. Sci Rep 2019;9(1):1–12.
• [60] Beller M, Stotzka R, Müller TO, et al. An example-based system to support the segmentation of stellate lesions. Berlin: Springer; 2005.
• [61] Sampat MP, Bovik AC, Whitman GJ, et al. A model-based framework for the detection of spiculated masses on mammography. Med Phys 2008;35(5):2110–23.
• [62] Sun L, Sun H, Wang J, Wu S, Zhao Y, Xu Y. Breast mass detection in mammography based on image template matching and CNN. Sensors 2021;21(8):2855.
• [63] Braz Junior G, da Rocha SV, de Almeida JDS, de Paiva AC, Silva AC, Gattass M. Breast cancer detection in mammography using spatial diversity, geostatistics, and concave geometry. Multimedia Tools Appl 2019;78(10):13005–31.
• [64] Liu Y, Zhang F, Zhang Q, et al. Cross-view correspondence reasoning based on bipartite graph convolutional network for mammogram mass detection. In: Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition; 2020: 3812-3822.
• [65] Cao H, Pu S, Tan W, Tong J. Breast mass detection in digital mammography based on anchor-free architecture. Comput Methods Programs Biomed 2021;205:106033. https://doi.org/ 10.1016/j.cmpb.2021.106033.