Fusing fine-tuned deep features for recognizing different tympanic membranes

• Autorzy: Cömert Zafer

Identyfikatory

DOI

10.1016/j.bbe.2019.11.001

• Języki publikacji: EN

Abstrakty

EN

Otitis media (OM) refers to a group of inflammatory diseases regarding the middle ear. Although there are a wide variety of disease types regarding OM, the most commonly seen disorders are acute otitis media (AOM), otitis media with effusion (OME) and chronic suppurative otitis media (CSOM). The examination of OM in the clinics is realized subjec-tively. This subjective examination is error-prone and leads to a limited variability among specialist. For these reasons, computer-aided systems are in demand. In this study, we focus on recognizing normal, AOM, CSOM, and earwax tympanic membrane (TM) conditions using fused fine-tuned deep features provided by pre-trained deep convolutional neural networks (DCNNs). These features are applied as the input to several networks, such as an artificial neural network (ANN), k-nearest neighbor (k NN), decision tree (DT) and support vector machine (SVM). Moreover, we release a new publicly available TM data set consisting of totally 956 otoscope images. As a result, the DCNNs yielded promising results. Especially, the most efficient results were provided by VGG-16 with an accuracy of 93.05 %. The fused fine-tuned deep features improved the overall classification success. Finally, the proposed model yielded promising results with an accuracy of 99.47 %, sensitivity of 99.35 %, and specificity of 99.77 % using the combination of the fused fine-tuned deep features and SVM model. Consequently, this study shows that fused fine-tuned deep features are rather useful in recognizing different TMs and these features can provide a fully automated model with high sensitivity.

Słowa kluczowe

EN

biomedical signal processing; computer-aided diagnosis system; image processing; otitis media; deep learning; deep features

PL

przetwarzanie sygnałów biomedycznych; komputerowe wspomaganie diagnostyki; przetwarzanie obrazu; zapalenie ucha środkowego; uczenie głębokie

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2020
• Tom: Vol. 40, no. 1
• Strony: 40--51
• Opis fizyczny: Bibliogr. 53 poz., rys., tab., wykr.

Twórcy

autor

Cömert Zafer

• Samsun University, Department of Software Engineering, Canik Yerleskesi Gürgenyatak Mahallesi Merkez Sokak No: 40-2/1 55080, Canik, Samsun, Turkey

Bibliografia

• [1] Shah-Becker S, Carr MM. Current management and referral patterns of pediatricians for acute otitis media. Int J Pediatr Otorhinolaryngol 2018;113:19–21. http://dx.doi.org/10.1016/j.ijporl.2018.06.036.
• [2] Marom T, Kraus O, Habashi N, Tamir SO. Emerging technologies for the diagnosis of otitis media. Otolaryngol Neck Surg 2019;160:447–56. http://dx.doi.org/10.1177/0194599818809337.
• [3] Schilder AGM, Chonmaitree T, Cripps AW, Rosenfeld RM, Casselbrant ML, Haggard MP, et al. Otitis media. Nat Rev Dis Prim 2016;2:16063.
• [4] Diagnosis and management of acute otitis media. Pediatrics 2004;113:1451–65.
• [5] van Uum RT, Venekamp RP, Sjoukes A, van de Pol AC, de Wit GA, Schilder AGM, et al. Optimising pain management in children with acute otitis media through a primary care-based multifaceted educational intervention: study protocol for a cluster randomised controlled trial. Trials 2018;19:501. http://dx.doi.org/10.1186/s13063-018-2880-4.
• [6] Thomas JP, Berner R, Zahnert T, Dazert S. Acute otitis media-a structured approach. Dtsch Arztebl Int 2014;111:151–60. http://dx.doi.org/10.3238/arztebl.2014.0151.
• [7] Guo H, Li M, Xu L-J. Apigetrin treatment attenuates LPS-induced acute otitis media though suppressing inflammation and oxidative stress. Biomed Pharmacother 2019;109:1978–87. http://dx.doi.org/10.1016/j.biopha.2018.07.022.
• [8] Davidoss NH, Varsak YK, Santa Maria PL. Animal models of acute otitis media – a review with practical implications for laboratory research. Eur Ann Otorhinolaryngol Head Neck Dis 2018;135:183–90. http://dx.doi.org/10.1016/j.anorl.2017.06.013.
• [9] Mousseau S, Lapointe A, Gravel J. Diagnosing acute otitis media using a smartphone otoscope; A randomized controlled trial. Am J Emerg Med 2018;36:1796–801. http://dx.doi.org/10.1016/j.ajem.2018.01.093.
• [10] Cetinkaya EA, Ciftci O, Alan S, Oztanir MN, Basak N. The efficacy of hesperidin for treatment of acute otitis media. Auris Nasus Larynx 2019;46:172–7. http://dx.doi.org/10.1016/j.anl.2018.07.005.
• [11] Basaran E, Cömert Z, Çelik Y. Convolutional neural network approach for automatic tympanic membrane detection and classification. Biomed Signal Process Control 2020;56:101734. http://dx.doi.org/10.1016/j.bspc.2019.101734.
• [12] Singh GB, Solo M, Kaur R, Arora R, Kumar S. Mycology of chronic suppurative otitis media-cholesteatoma disease: an evaluative study. Am J Otolaryngol 2018;39:157–61. http://dx.doi.org/10.1016/j.amjoto.2017.12.001.
• [13] Lefebvre M-A, Quach C, Daniel SJ. Chronic suppurative otitis media due to nontuberculous mycobacteria: a case of successful treatment with topical boric acid. Int J Pediatr Otorhinolaryngol 2015;79:1158–60. http://dx.doi.org/10.1016/j.ijporl.2015.04.043.
• [14] Ahmad S. Antibiotics in chronic suppurative otitis media: a bacteriologic study. Egypt J Ear Nose Throat Allied Sci 2013;14:191–4. http://dx.doi.org/10.1016/j.ejenta.2013.06.001.
• [15] Nakagawa H, Toyoda Y, Albrecht T, Tsukamoto M, Praetorius M, Ishikawa T, et al. Are human ATP-binding cassette transporter C11 and earwax associated with the incidence of cholesteatoma? Med Hypotheses 2018;114:19–22. http://dx.doi.org/10.1016/j.mehy.2018.02.030.
• [16] Sanna M, Russo A, Caruso A, Taibah A, Piras G. Color atlas of endo-otoscopy. Thieme; 2017.
• [17] Núñez-Batalla F, Jáudenes-Casaubón C, Sequí-Canet JM, Vivanco-Allende A, Zubicaray-Ugarteche J. Diagnosis and treatment of otitis media with effusion: CODEPEH recommendations. Acta Otorrinolaringol (English Ed) 2019;70:36–46. http://dx.doi.org/10.1016/j.otoeng.2017.07.004.
• [18] Goggin LS, Eikelboom RH, Atlas MD. Clinical decision support systems and computer-aided diagnosis in otology. Otolaryngol Neck Surg 2007;136:Error: FPage (s21) is higher than LPage (6)!. http://dx.doi.org/10.1016/j.otohns.2007.01.028.
• [19] Junior H, Comunello E, Costa S, Dornelles CC. Computational techniques for accompaniment and measuring of otology pathologies. Twent. IEEE Int. Symp. Comput. Med. Syst., Maribor, Slovenia; 2007.
• [20] Cheng L, Liu J, Roehm CE, Valdez TA. Enhanced video images for tympanic membrane characterization. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS 2011;4002–5. http://dx.doi.org/10.1109/IEMBS.2011.6090994.
• [21] Mironica I, Vertan C, Gheorghe DC. Automatic pediatric otitis detection by classification of global image features. 2011 E-Health Bioeng. Conf.. 2011. pp. 1–4.
• [22] Vertan C, Gheorghe DC, Ionescu B. Eardrum color content analysis in video-otoscopy images for the diagnosis support of pediatric otitis. ISSCS 2011 - Int Symp Signals, Circuits Syst Proc. 2011. pp. 129–32. http://dx.doi.org/10.1109/ISSCS.2011.5978676.
• [23] Kuruvilla A, Shaikh N, Hoberman A, Kovacevic´ J. Automated diagnosis of otitis media: vocabulary and grammar. J Biomed Imaging 2013;2013:27.
• [24] Shie CK, Chang HT, Fan FC, Chen CJ, Fang TY, Wang PC. A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media. 2014 36th Annu Int Conf IEEE Eng Med Biol Soc EMBC 2014;2014:4655–8. http://dx.doi.org/10.1109/EMBC.2014.6944662.
• [25] Myburgh HC, van Zijl WH, Swanepoel D, Hellström S, Laurent C. Otitis media diagnosis for developing countries using tympanic membrane image-analysis. EBioMedicine 2016;5:156–60. http://dx.doi.org/10.1016/J.EBIOM.2016.02.017.
• [26] Huang YK, Huang CP. A depth-first search algorithm based otoscope application for real-time otitis media image interpretation. Parallel Distrib Comput Appl Technol PDCAT Proc 2018;2017(Decem):170–5. http://dx.doi.org/10.1109/PDCAT.2017.00036.
• [27] Myburgh HC, Jose S, Swanepoel DW, Laurent C. Towards low cost automated smartphone- and cloud-based otitis media diagnosis. Biomed Signal Process Control 2018;39:34–52. http://dx.doi.org/10.1016/j.bspc.2017.07.015.
• [28] Lee JY, Choi S-H, Chung JW. Automated classification of the tympanic membrane using a convolutional neural network. Appl Sci 2019;9. http://dx.doi.org/10.3390/app9091827.
• [29] Monroy GL, Won J, Dsouza R, Pande P, Hill MC, Porter RG, et al. Automated classification platform for the identification of otitis media using optical coherence tomography. Npj Digit Med 2019;2:22. http://dx.doi.org/10.1038/s41746-019-0094-0.
• [30] Cha D, Pae C, Seong S-B, Choi JY, Park H-J. Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database. EBioMedicine 2019;45:606–14. http://dx.doi.org/10.1016/j.ebiom.2019.06.050.
• [31] Krizhevsky A, Sutskever I, Hinton GE. Machine learning and computer vision group deep learning with tensorflow; 2012.
• [32] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ArXiv Prepr ArXiv14091556 2014.
• [33] Szegedy C, Liu Wei, Jia Yangqing, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conf. Comput. Vis. Pattern Recognit.; 2015. pp. 1–9. http://dx.doi.org/10.1109/CVPR.2015.7298594.
• [34] Lin M, Chen Q, Yan S. Network in network. ArXiv Prepr ArXiv13124400 2013.
• [35] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proc. IEEE Conf. Comput. Vis. Pattern Recognit.; 2016. pp. 770–8. http://dx.doi.org/10.1109/CVPR.2016.90.
• [36] Cömert Z, Kocamaz AF. Fetal hypoxia detection based on deep convolutional neural network with transfer learning approach, vol. 763. 2019. http://dx.doi.org/10.1007/978-3-319-91186-1_25.
• [37] Cibuk M, Budak U, Guo Y, Ince MC, Sengur A. Efficient deep features selections and classification for flower species recognition. Measurement 2019;137:7–13. http://dx.doi.org/10.1016/j.measurement.2019.01.041.
• [38] Zhao Z, Zhang Y, Comert Z, Deng Y. Computer-aided diagnosis system of fetal hypoxia incorporating recurrence plot with convolutional neural network. Front Physiol 2019;10:255. http://dx.doi.org/10.3389/fphys.2019.00255.
• [39] Sarigül M, Ozyildirim BM, Avci M. Differential convolutional neural network. Neural Networks 2019;116:279–87. http://dx.doi.org/10.1016/j.neunet.2019.04.025.
• [40] Mesut T, Burhan E. Biyomedikal görüntülerde derin Ögrenme ile mevcut yöntemlerin kiyaslanmasi. Firat Üniversitesi Mühendislik Bilim Derg 2019;31:109–21.
• [41] Altuntas Y, Cömert Z, Kocamaz AF. Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach. Comput Electron Agric 2019;163:104874. http://dx.doi.org/10.1016/j.compag.2019.104874.
• [42] Cömert Z, Kocamaz AF. Fetal hypoxia detection based on deep convolutional neural network with transfer learning approach. In: Silhavy R, editor. Softw. Eng. Algorithms intell. Syst.. Cham: Springer International Publishing; 2019. p. 239–48. http://dx.doi.org/10.1007/978-3-319-91186-1_25.
• [43] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Proc 25th int Conf Neural inf Process Syst, Vol. 1. USA: Curran Associates, Inc.; 2012. p. 1097–105.
• [44] Sahin H, Subasi A. Classification of the cardiotocogram data for anticipation of fetal risks using machine learning techniques. Appl Soft Comput 2015;33:231–8. http://dx.doi.org/10.1016/j.asoc.2015.04.038.
• [45] Akbulut Y, Sengur A, Guo Y, Smarandache F. NS-k-NN: neutrosophic set-based k-nearest neighbors classifier. Symmetry (Basel) 2017;9.
• [46] Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. Springer; 2009. http://dx.doi.org/10.1007/b94608.
• [47] Altuntas Y, Kocamaz AF, Cömert Z, Cengiz R, Esmeray M. Identification of haploid maize seeds using gray level Co-occurrence matrix and machine learning techniques. 2018 Int. Conf. Artif. Intell. Data Process.; 2018. pp. 1–5. http://dx.doi.org/10.1109/IDAP.2018.8620740.
• [48] Huang M-L, Yung-Yan H. Fetal distress prediction using discriminant analysis, decision tree, and artificial neural network. J Biomed Sci Eng 2012;05:526–33. http://dx.doi.org/10.4236/jbise.2012.59065.
• [49] Diker A, Cömert Z, Avci E, Velappan S. Intelligent system based on genetic algorithm and support vector machine for detection of myocardial infarction from ECG signals. 2018 26th Signal Process. Commun. Appl. Conf.. 2018. pp. 1–4. http://dx.doi.org/10.1109/SIU.2018.8404299.
• [50] Diker A, Avci E, Cömert Z, Avci D, Kaçar E, Serhatlioglu I. Classification of ECG signal by using machine learning methods. 2018 26th Signal Process. Commun. Appl. Conf.. 2018. pp. 1–4. http://dx.doi.org/10.1109/SIU.2018.8404298.
• [51] Cömert Z, Kocamaz AF, Subha V. Prognostic model based on image-based time-frequency features and genetic algorithm for fetal hypoxia assessment. Comput Biol Med 2018. http://dx.doi.org/10.1016/j.compbiomed.2018.06.003.
• [52] Kentala E, Auramo Y, Pyykkö I, Juhola M. Otoneurological expert system. Ann Otol Rhinol Laryngol 1996;105:654–8. http://dx.doi.org/10.1177/000348949610500812.
• [53] Cömert Z, Kocamaz AF. Open-access software for analysis of fetal heart rate signals. Biomed Signal Process Control 2018;45:98–108. http://dx.doi.org/10.1016/j.bspc.2018.05.016.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).