Multi-View Attention-based Late Fusion (MVALF) CADx system for breast cancer using deep learning

• Autorzy: Iftikhar Hina , Shahid Ahmad Raza , Raza Basit , Khan Hasan

Identyfikatory

DOI

10.22630/MGV.2020.29.1.4

• Języki publikacji: EN

Abstrakty

EN

Breast cancer is a leading cause of death among women. Early detection can significantly reduce the mortality rate among women and improve their prognosis. Mammography is the first line procedure for early diagnosis. In the early era, conventional Computer-Aided Diagnosis (CADx) systems for breast lesion diagnosis were based on just single view information. The last decade evidence the use of two views mammogram: Medio-Lateral Oblique (MLO) and Cranio-Caudal (CC) view for the CADx systems. Most recent studies show the effectiveness of four views of mammogram to train CADx system with feature fusion strategy for classification task. In this paper, we proposed an end-to-end Multi-View Attention-based Late Fusion (MVALF) CADx system that fused the obtained predictions of four view models, which is trained for each view separately. These separate models have different predictive ability for each class. The appropriate fusion of multi-view models can achieve better diagnosis performance. So, it is necessary to assign the proper weights to the multi-view classification models. To resolve this issue, attention-based weighting mechanism is adopted to assign the proper weights to trained models for fusion strategy. The proposed methodology is used for the classification of mammogram into normal, mass, calcification, malignant masses and benign masses. The publicly available datasets CBIS-DDSM and mini-MIAS are used for the experimentation. The results show that our proposed system achieved 0.996 AUC for normal vs. abnormal, 0.922 for mass vs. calcification and 0.896 for malignant vs. benign masses. Superior results are seen for the classification of malignant vs benign masses with our proposed approach, which is higher than the results using single view, two views and four views early fusion-based systems. The overall results of each level show the potential of multi-view late fusion with transfer learning in the diagnosis of breast cancer.

Słowa kluczowe

EN

breast cancer; mammogram; four-view mammogram; information fusion; late fusion; transfer learning

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2020
• Tom: Vol. 29, No. 1/4
• Strony: 55--78
• Opis fizyczny: Bibliogr. 51 poz., fot., schem., tab., wykr.

Twórcy

autor

Iftikhar Hina

• 1) Medical Imaging and Diagnostics Laboratory (MID), National Centre of Artificial Intelligence (NCAI), Islamabad, Pakistan
• 2) Department of Computer Science, COMSATS University Islamabad (CUI), Pakistan

autor

Shahid Ahmad Raza

• 1) Medical Imaging and Diagnostics Laboratory (MID), National Centre of Artificial Intelligence (NCAI), Islamabad, Pakistan
• 2) Department of Computer Science, COMSATS University Islamabad (CUI), Pakistan

autor

Raza Basit

• 1) Medical Imaging and Diagnostics Laboratory (MID), National Centre of Artificial Intelligence (NCAI), Islamabad, Pakistan
• 2) Department of Computer Science, COMSATS University Islamabad (CUI), Pakistan

autor

Khan Hasan

• 1) Medical Imaging and Diagnostics Laboratory (MID), National Centre of Artificial Intelligence (NCAI), Islamabad, Pakistan
• 2) Department of Computer Science, COMSATS University Islamabad (CUI), Pakistan

Bibliografia

• [1] Large Scale Visual Recognition Challenge 2015 (ILSVRC2015), 2015. http://www.image-net.org/challenges/LSVRC/2015/results. [Aceessed Jun 2020].
• [2] R. Agarwal, O. Diaz, X. Lladó, et al. Automatic mass detection in mammograms using deep convolutional neural networks. Journal of Medical Imaging, 6(3):31409, 2019. doi:10.1117/1.JMI.6.3.031409.
• [3] A. Akselrod-Ballin, L. Karlinsky, A. Hazan, et al. Deep learning for automatic detection of ab-normal findings in breast mammography. In M. J. Cardoso, T. Arbel, G. Carneiro, et al., editors, Proc. Int. Workshops on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support DLMIA, ML-CDS, in conjunction with MICCAI 2017, volume 10553 of Lecture Notes in Computer Science, pages 321–329. Springer, Québec City, QC, Canada, 14 Sep 2017. doi:10.1007/978-3-319-67558-937.
• [4] M. A. Al-masni, M. A. Al-antari, J.-M. Park, et al. Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. Computer methods programs in biomedicine, 157:85–94, 2018. doi:10.1016/j.cmpb.2018.01.017.
• [5] J. Arevalo, F. A. González, Ramos-Pollán, et al. Representation learning for mammography mass lesion classification with convolutional neural networks. Computer methods programs in biomedicine,127:248–257, 2016. doi:10.1016/j.cmpb.2015.12.014.
• [6] L. W. Bassett, I. A. Hirbawi, N. DeBruhl, and M. K. Hayes. Mammographic positioning: evaluation from the view box. Radiology, 188(3):803–806, 1993. doi:10.1148/radiology.188.3.8351351.
• [7] M. Bator and M. Nieniewski. Detection of cancerous masses in mammograms by template matching: Optimization of template brightness distribution by means of evolutionary algorithm. Journal of Digital Imaging, 25(1):162–172, 2012. doi:10.1007/s10278-011-9402-1.
• [8] G. Carneiro, J. Nascimento, and A. P. Bradley. Unregistered multiview mammogram analysis with pre-trained deep learning models. In N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, editors, Proc. Int. Conf. Medical Image Computing and Computer-Assisted Intervention MICCAI 2015, volume 9351 of Lecture Notes in Computer Science, pages 652–660, Munich, Germany, 5-9 Oct 2015. Springer. doi:10.1007/978-3-319-24574-478.
• [9] G. Carneiro, J. Nascimento, and A. P. Bradley. Automated analysis of unregistered multi-view mammograms with deep learning. IEEE Transactions on Medical Imaging, 36(11):2355–2365, 2017. doi:10.1109/TMI.2017.2751523.
• [10] J. Chakraborty, A. Midya, and R. Rabidas. Computer-aided detection and diagnosis of mammographic masses using multi-resolution analysis of oriented tissue patterns. Expert Systems with Applications, 99:168–179, 2018. doi:10.1016/j.eswa.2018.01.010.
• [11] H. Chen, D. Ni, J. Qin, et al. Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE Journal of Biomedical Health Informatics, 19(5):1627–1636, 2015. doi:10.1109/JBHI.2015.2425041.
• [12] H. Chougrad, H. Zouaki, and O. Alheyane. Deep convolutional neural networks for breast cancer screening. Computer Methods Programs in Biomedicine, 157:19–30, 2018. doi:10.1016/j.cmpb.2018.01.011.
• [13] F. Ciompi, B. de Hoop, S. J. van Riel, et al. Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Medical Image Analysis, 26(1):195–202, 2015. doi:10.1016/j.media.2015.08.001.
• [14] A. F. Clark. The mini-MIAS database of mammograms, 2012. http://peipa.essex.ac.uk/info/mias.html [Accessed Jun 2020].
• [15] K. Clark, B. Vendt, K. Smith, et al. The cancer imaging archive (TCIA): Maintaining and operating a public information repository. Journal of Digital Imaging, 26(6):1045–1057, 2013. doi:10.1007/s10278-013-9622-7.
• [16] J. Deng, W. Dong, R. Socher, et al. ImageNet: A large-scale hierarchical image database. In Proc. IEEE Conf. Computer Vision and Pattern Recognition CVPR 2009, pages 248–255, Miami, FL,USA, 20-25 Jun 2009. IEEE. doi:10.1109/CVPR.2009.5206848.
• [17] S. Dhahbi, W. Barhoumi, and E. Zagrouba. Multi-view score fusion for content-based mammogram retrieval. In A. Verikas, P. Radeva, and D. Nikolaev, editors, Proc. 8th Int. Conf. Machine Vision ICMV 2015, volume 9875 of Proc. SPIE, page 987515, Barcelona, Spain, 8 Dec 2015. doi:10.1117/12.2228614.
• [18] N. Dhungel, G. Carneiro, and A. P. Bradley. A deep learning approach for the analysis of masses in mammograms with minimal user intervention. Medical Image Analysis, 37:114–128, 2017. doi:10.1016/j.media.2017.01.009.
• [19] G. W. Eklund. The art of mammographic positioning. In M. Friedrich and E. A. Sickles, editors, Radiological Diagnosis of Breast Diseases, pages 75–88. Springer, 2000. doi:10.1007/978-3-642-60919-06.
• [20] M. Heath, D. Bowyer, R. Kopans, et al. The digital data base for screening Mammography. In M. J. Yaffe, editor, Proc. 5th Int. Workshop on Digital Mammography, pages 212–218, Toronto, Canada, 11-14 Jun 2000. Medical Physics Publishing, Madison, WI, USA. http://www.eng.usf.edu/cvprg/Mammography/Database.html.
• [21] M. Heath, K. Bowyer, D. Kopans, et al. Current status of the digital database for screening mammography. In N. Karssemeijer, M. Thijssen, J. Hendriks, and L. van Erning, editors, Digital Mammography, pages 457–460. Springer Netherlands, Dordrecht, 1998. doi:10.1007/978-94-011-5318-875.
• [22] B. Q. Huynh, H. Li, and M. L. Giger. Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. Journal of Medical Imaging, 3(3):34501, 2016. doi:10.1117/1.JMI.3.3.034501.
• [23] Z. Jiao, X. Gao, Y. Wang, and J. Li. A deep feature based framework for breast masses classification. Neurocomputing, 197:221–231, 2016. doi:10.1016/j.neucom.2016.02.060.
• [24] A. Jouirou, A. Baâzaoui, and W. Barhoumi. Multi-view information fusion in mammograms: A comprehensive overview. Information Fusion, 52:308–321, 2019. doi:10.1016/j.inffus.2019.05.001.
• [25] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 53(8):5455–5516, 2020. doi:10.1007/s10462-020-09825-6.
• [26] H. N. Khan, A. R. Shahid, B. Raza, A. H. Dar, and H. Alquhayz. Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access, 7:165724–165733, 2019. doi:10.1109/ACCESS.2019.2953318.
• [27] E. Kozegar, M. Soryani, B. Minaei, and I. Domingues. Assessment of a novel mass detection algorithm in mammograms. Journal of Cancer Research and Therapeutics, 9(4):592, 2013. doi:10.4103/0973-1482.126453.
• [28] R. S. Lee, F. Gimenez, A. Hoogi, et al. A curated mammography data set for use in computer-aided detection and diagnosis research. Scientific Data, 4:170177, 2017. doi:10.1038/sdata.2017.177.
• [29] H. Li, S. Zhuang, D.-A. Li, J. Zhao, and Y. Ma. Benign and malignant classification of mammogram images based on deep learning. Biomedical Signal Processing Control, 51:347–354, 2019. doi:10.1016/j.bspc.2019.02.017.
• [30] Y. Li, H. Chen, Y. Yang, et al. A bilateral analysis scheme for false positive reduction in mammogram mass detection. Computers in Biology and Medicine, 57:84–95, 2015. doi:10.1016/j.compbiomed.2014.12.007.
• [31] X. Liu, T. Zhu, L. Zhai, and J. Liu. Improvement of mass detection in mammogram using multi-view information. In C. M. Falco and X. Jiang, editors, Proc. 8th Int. Conf. Digital Image Processing ICDIP 2016, volume 10033 of Proc. SPIE, page 100334M, Chengdu, China, 29 Aug 2016. doi:10.1117/12.2244627.
• [32] W. Lotter, G. Sorensen, and D. Cox. A multi-scale CNN and curriculum learning strategy for mammogram classification. In M. J. Cardoso, T. Arbel, G. Carneiro, et al., editors, Proc. Int. Workshops on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support DLMIA, ML-CDS, in conjunction with MICCAI 2017, volume 10553 of Lecture Notes in Computer Science, pages 169–177. Springer, Québec City, QC, Canada, 14 Sep 2017. doi:10.1007/978-3-319-67558-920.
• [33] W. Peng, R. V. Mayorga, and E. M. A. Hussein. An automated confirmatory system for analysis of mammograms. Computer Methods Programs in Biomedicine, 125:134–144, 2016. doi:10.1016/j.cmpb.2015.09.019.
• [34] D. Ribli, A. Horváth, Z. Unger, et al. Detecting and classifying lesions in mammograms with deep learning. Scientific Reports, 8(1):4165, 2018. doi:10.1038/s41598-018-22437-z.
• [35] O. Russakovsky, J. Deng, H. Su, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015. doi:10.1007/s11263-015-0816-y.
• [36] H.-C. Shin, H. R. Roth, M. Gao, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5):1285–1298, 2016. doi:10.1109/TMI.2016.2528162.
• [37] R. L. Siegel, K. D. Miller, and A. Jemal. Cancer statistics, 2019.CA: A Cancer Journal for Clinicians, 69(1):7–34, 2019. doi:10.3322/caac.21551.
• [38] K. Simonyan and A Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint, 2014. arXiv:1409.1556v6.
• [39] K. Smith, J. Kirby, D. Runbin, et al. CBIS-DDSM – Curated Breast Imaging Subset of DDSM. In TCIA Team [44]. [Accessed Jun 2020]. https://wiki.cancerimagingarchive.net/display/Public/CBIS-DDSM.
• [40] J. Suckling, J. Parker, D. Dance, et al. The Mammographic Images Analysis Society digital mammogram database. In A. G. Gale, S. M. Astley, D. R. Dance, and A. Y. Cairns, editors, Digital Mammography, volume 1069 of Exerpta Medica International Congress Series, pages 375–378. Elsevier, 1994.http://www.wiau.man.ac.uk/services/MIAS/ [Inoperative].
• [41] L. Sun, L. Li, W. Xu, et al. A novel classification scheme for breast masses based on multi-view information fusion. In Proc. 4th Int. Conf. Bioinformatics and Biomedical Engineering iCBBE 2010, pages 1–4, Chengdu, China, 18-20 Jun 2010. IEEE. doi:10.1109/iCBBE.2010.5517742.
• [42] C. Szegedy, W. Liu, Y. Jia, et al. Going deeper with convolutions. In Proc. IEEE Conf. Computer Vision and Pattern Recognition CVPR 2015, pages 1–9, Boston, MA, USA, 7-12 Jun 2015. doi:10.1109/CVPR.2015.7298594. https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Szegedy_Going_Deeper_With_2015_CVPR_paper.html.
• [43] M. Tan, J. Pu, S. Cheng, et al. Assessment of a four-view mammographic image feature based fusion model to predict near-term breast cancer risk. Annals of Biomedical Engineering, 43(10):2416–2428, 2015. doi:10.1007/s10439-015-1316-5.
• [44] TCIA Team, editors. The Cancer Imaging Archive, 2021. [Accessed Jun 2020]. https://www.cancerimagingarchive.net/.
• [45] P. D. Trieu, P. C. Brennan, W. Lee, E. Ryan, et al. The value of the craniocaudal mammographic view in breast cancer detection: a preliminary study. In C. K. Abbey and C. R. Mello-Thoms, editors, Proc. Conf. SPIE Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, volume 8673 of Proc. SPIE, page 86731J, Lake Buena Vista, FL, United States, 28 Mar 2013. doi:10.1117/12.2006821.
• [46] C. J. Vyborny and R. A. Schmidt. Mammography as a radiographic examination: an overview. Radio Graphics, 9(4):723–764, 1989. doi:10.1148/radiographics.9.4.2667052.
• [47] WebMD. Breast Calcifications, 2020. https://www.webmd.com/women/guide/breast-calcification-symptoms-causes-treatments. [Accessed Jun 2020].
• [48] J. Wei, H.-P. Chan, C. Zhou, et al. Computer-aided detection of breast masses: Four-view strategy for screening mammography. Medical Physics, 38(4):1867–1876, 2011. doi:10.1118/1.3560462.
• [49] World Health Organization. Cancer. Facts sheet, 2018. https://www.who.int/news-room/fact-sheets/detail/cancer. [Aceessed Jun 2020].
• [50] N. I. R. Yassin, S. Omran, E. M. F. El Houby, and H. Allam. Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review. Computer Methods and Programs in Biomedicine, 156:25–45, 2018. doi:10.1016/j.cmpb.2017.12.012.
• [51] H. Zhao, W. Xu, L. Li, and J. Zhang. Classification of breast masses based on multi-view information fusion using multi-agent method. In Proc. 5th Int. Conf. Bioinformatics and Biomedical Engineering iCBBE 2011, pages 1–4, Wuhan, China, 10-12 May 2011. IEEE. doi:10.1109/iCBBE.2011.5780304