Review on approaches to concept detection in medical images

• Autorzy: Miranda Diana , Thenkanidiyoor Veena , Dinesh Dileep Aroor

Identyfikatory

DOI

10.1016/j.bbe.2022.02.012

• Języki publikacji: EN

Abstrakty

EN

Concept detection in medical images involves the identification of various biomedical semantic entities in the images. This is a non-trivial task due to the highly heterogeneous nature of medical images. This heterogeneity is caused due to the different body parts, presence of abnormalities, and imaging techniques used to capture the image. In this paper, a thorough survey on important approaches to concept detection in medical images is presented. Methods such as multi-label classification, sequence-to-sequence learning, detecting concepts from captions, and similarity search-based approaches to concept detection in medical images are reviewed. This paper also highlights the challenges associated with the medical image concept detection task. Possible avenues for further research are also discussed.

Słowa kluczowe

EN

concept detection; medical image; biomedical concept unique identifiers; CUIs

PL

wykrywanie koncepcji; obraz medyczny; CUIs

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2022
• Tom: Vol. 42, no. 2
• Strony: 453--462
• Opis fizyczny: Bibliogr. 54 poz., rys., tab., wykr.

Twórcy

autor

Miranda Diana

• National Institute of Technology Goa, Ponda, India

autor

Thenkanidiyoor Veena

• National Institute of Technology Goa, Ponda, India

autor

Dinesh Dileep Aroor

• Indian Institute of Technology Mandi, India

Bibliografia

• [1] Li Z, Yao L, Chang X, Zhan K, Sun J, Zhang H. Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recogn 2019;88:595–603.
• [2] Pesce E, Withey SJ, Ypsilantis P-P, Bakewell R, Goh V, Montana G. Learning to detect chest radiographs containing pulmonary lesions using visual attention networks. Med Image Anal 2019;53:26–38.
• [3] Karatzas B, Pavlopoulos J, Kougia V, Androutsopoulos I. Aueb nlp group at imageclefmed caption 2020. CLEF 2020 Working Notes, 22–25. Thessaloniki, Greece; September 2020. p. 2020.
• [4] A.G.S. de Herrera, F.P. Andrade, L. Bentley, A.A. Compean, Essex at image-clefcaption 2020 task, in: CLEF2020 Working Notes. CEUR Workshop Proceedings, CEUR-WS. org, Thessaloniki, Greece (September 22–25 2020), 2020.
• [5] Arul SD, Srinivasan K. Imageclef 2020: Image caption prediction using multilabel convolutional neural network. Ultrasound 2020;8629(2134):502.
• [6] R. Sonker, A. Mishra, P. Bansal, A. Pattnaik, Techniques for medical concept detection from multi-modal images, in: CLEF2020 Working Notes. CEUR Workshop Proceedings, CEUR-WS. org, Thessaloniki, Greece (September 22–25 2020), 2020.
• [7] Xu J, Liu W, Liu C, Wang Y, Chi Y, Xie X, Hua X-S. Concept detection based on multi-label classification and image captioning approach-DAMO at ImageCLEF 2019. CLEF (Working Notes) 2019.
• [8] Lyndon D, Kumar A, Kim J. Neural captioning for the ImageCLEF 2017 medical image challenges. CLEF (Working Notes) 2017.
• [9] Eickhoff C, Schwall I. A. Garcia Seco De Herrera, H. Müller, Overview of ImageCLEFcaption 2017–image caption prediction and concept detection for biomedical images. CEUR Workshop Proceedings 2017.
• [10] A. García Seco de Herrera, C. Eickhoff, V. Andrearczyk, H. Müller, Overview of the ImageCLEF 2018 caption prediction tasks, in: CLEF2018 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org , Avignon, France, 2018.
• [11] Pelka O, Friedrich CM, Seco de Herrera AG, Müller H. Overview of the ImageCLEFmed 2019 concept detection task. CEUR Workshop Proceedings 2019.
• [12] Pelka O, Friedrich CM. A. García Seco de Herrera, H. Müller, Overview of the imageclefmed 2020 concept prediction task: Medical image understanding. In: CLEF2020 Working Notes, CEUR Workshop Proceedings. Thessaloniki: CEUR-WS. Org; 2020.
• [13] Singh S, Karimi S, Ho-Shon K, Hamey L. Biomedical concept detection in medical images: MQ-CSIRO at 2019 ImageCLEFmed caption task. CLEF (Working Notes) 2019.
• [14] Sinha P, Purkayastha S, Gichoya J. Full training versus fine tuning for radiology images concept detection task for the ImageCLEF 2019 challenge. CLEF (Working Notes) 2019.
• [15] Wang X, Liu N. AI600 Lab at ImageCLEF 2019 concept detection task. CLEF (Working Notes) 2019.
• [16] Guo Z, Wang X, Zhang Y, Li J. ImageSem at ImageCLEFmed caption 2019 task: a two-stage medical concept detection strategy. CLEF (Working Notes) 2019.
• [17] X. Wang, Z. Guo, Y. Zhang, J. Li, Medical image labelling and semantic understanding for clinical applications, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2019, pp. 260–270.
• [18] Tatusch M. Approaches for the improvement of the multilabel multiclass classification with a huge number of classes. Grundlagen von Datenbanken 2018:65–70.
• [19] Y. Zhang, X. Wang, Z. Guo, J. Li, ImageSem at ImageCLEF 2018 caption task: Image retrieval and transfer learning, in: CLEF CEUR Workshop, Avignon, France, 2018.
• [20] Wang X, Zhang Y, Guo Z, Li J. Identifying concepts from medical images via transfer learning and image retrieval. Math Biosci Eng 2019;16(4):1978–91.
• [21] E. Pinho, J.F. Silva, J.M. Silva, C. Costa, Towards representation learning for biomedical concept detection in medical images: UA. PT Bioinformatics in ImageCLEF 2017, in: CLEF (Working Notes), 2017.
• [22] Pinho E, Costa C. Unsupervised learning for concept detection in medical images: A comparative analysis. Appl Sci 2018;8(8):1213.
• [23] Rahman M, Lagree T, Taylor M. A cross-modal concept detection and caption prediction approach in ImageCLEFcaption track of ImageCLEF 2017. CLEF (Working Notes) 2017(8).
• [24] Gonçalves AJ, Pinho E, Costa C. Informative and intriguing visual features: UA. PT Bioinformatics in ImageCLEF caption 2019. CLEF (Working Notes) 2019.
• [25] L. Valavanis, S. Stathopoulos, IPL at ImageCLEF 2017 concept detection task, in: CLEF (Working Notes), 2017.
• [26] A.B. Abacha, A. Seco De Herrera, S. Gayen, D. DemnerFushman, S. Antani, NLM at ImageCLEF 2017 caption task, CEUR Workshop Proceedings, 2017.
• [27] L. Valavanis, T. Kalamboukis, IPL at ImageCLEF 2018: A knnbased concept detection approach, in: CLEF (Working Notes), 2018.
• [28] E. Pinho, C. Costa, Feature learning with adversarial networks for concept detection in medical images: UA. PT Bioinformatics at ImageCLEF 2018, in: CLEF (Working Notes), 2018.
• [29] Kougia V, Pavlopoulos J, Androutsopoulos I. AUEB NLP group at ImageCLEFmed caption 2019. CLEF (Working Notes) 2019.
• [30] Hasan SA, Ling Y, Liu J, Sreenivasan R, Anand S, Arora TR, Datla VV, Lee K, Qadir A, Swisher C, et al. PRNA at ImageCLEF 2017 caption prediction and concept detection tasks. CLEF (Working Notes) 2017.
• [31] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 770–8.
• [32] Kalimuthu M, Nunnari F, Sonntag D. A competitive deep neural network approach for the imageclefmed, 2020. caption 2020 task, arXiv preprint arXiv:2007.14226.
• [33] O. Layode, M.M. Rahman, Multi-label and cross-modal based concept detection in biomedical images by morgan cs at imageclef2020.
• [34] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 2818–26.
• [35] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 1–9.
• [36] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014).
• [37] S.A. Hasan, Y. Ling, J. Liu, R. Sreenivasan, S. Anand, T.R. Arora, V. Datla, K. Lee, A. Qadir, C. Swisher, et al., Attention-based medical caption generation with image modality classification and clinical concept mapping, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2018, pp. 224–230.
• [38] Dimitris K, Ergina K. Concept detection on medical images using deep residual learning network. In: Working notes of conference and labs of the evaluation forum. Dublin, Ireland: Springer; 2017.
• [39] L.-D. Stefan, B. Ionescu, H. Müller, Generating captions for medical images with a deep learning multi-hypothesis approach: MedGIFT–UPB participation in the ImageCLEF 2017 caption task.
• [40] N. Udas, F. Beuth, D. Kowerko, Concept detection in medical images using xception models-tuc mc at imageclefmed 2020 (2020).
• [41] Ny Hoavy N, Mothe J, Randrianarivony MI. IRIT & MISA at image clef 2017-multi label classification. CEUR Workshop Proceedings 2017.
• [42] Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database, in, IEEE conference on computer vision and pattern recognition. IEEE 2009;2009:248–55.
• [43] Elman JL. Finding structure in time. Cognit. Sci. 1990;14 (2):179–211.
• [44] F.A. Gers, J. Schmidhuber, F. Cummins, Learning to forget: Continual prediction with lstm (1999).
• [45] Tharwat A. Classification assessment methods. Appl Comput Inform 2018.
• [46] Yang S, Niu J, Wu J, Liu X. Automatic medical image report generation with multi-view and multi-modal attention mechanism. In: International Conference on Algorithms and Architectures for Parallel Processing Springer. p. 687–99.
• [47] Demner-Fushman D, Kohli MD, Rosenman MB, Shooshan SE, Rodriguez L, Antani S, Thoma GR, McDonald CJ. Preparing a collection of radiology examinations for distribution and retrieval. J Am Med Inform Assoc 2016;23(2):304–10.
• [48] A.E. Johnson, T.J. Pollard, N.R. Greenbaum, M.P. Lungren, C.-Y. Deng, Y. Peng, Z. Lu, R.G. Mark, S.J. Berkowitz, S. Horng, Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs, arXiv preprint arXiv:1901.07042 (2019).
• [49] Y. Zhang, X. Wang, Z. Xu, Q. Yu, A. Yuille, D. Xu, When radiology report generation meets knowledge graph, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 12910–12917.
• [50] Zhou Y, Huang L, Zhou T, Fu H, Shao L. Visual-textual attentive semantic consistency for medical report generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 3985–94.
• [51] Yin C, Qian B, Wei J, Li X, Zhang X, Li Y, Zheng Q. Automatic generation of medical imaging diagnostic report with hierarchical recurrent neural network, in, IEEE international conference on data mining (ICDM). IEEE 2019;2019:728–37.
• [52] P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. Langlotz, K. Shpanskaya, et al., Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning, arXiv preprint arXiv:1711.05225 (2017).
• [53] Li Z, Nie F, Chang X, Nie L, Zhang H, Yang Y. Rank-constrained spectral clustering with flexible embedding. IEEE Trans Neural Networks Learn Syst 2018;29(12):6073–82.
• [54] Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N. Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Networks Learn Syst 2018;29(12):6323–32.