Fourier transform-based data augmentation in deep learning for diabetic foot thermograph classification

• Autorzy: Anaya-Isaza Andrés , Zequera-Diaz Martha

Identyfikatory

DOI

10.1016/j.bbe.2022.03.001

• Języki publikacji: EN

Abstrakty

EN

Diabetes Mellitus (DM) belongs to the ten diseases group with the highest mortality rate globally, with an estimated 578 million cases by 2030, according to the World Health Organization (WHO). The disease manifests itself through different disorders, where vasculopathy shows a chronic relationship with diabetic ulceration events in distal extremities, being temperature a biomarker that can quantify the risk scale. According to the above, an analysis is performed with standing thermography images, finding temperature patterns that do not follow a particular distribution in patients with DM. Therefore, the modern medical literature has taken the use of Computer-Aided Diagnosis (CAD) systems as a plausible option to increase medical analysis capabilities. In this sense, we proposed to study three state-of-the-art deep learning (DL) architectures, experimenting with convolutional, residual, and attention (Transformers) approaches to classify subjects with DM from diabetic foot thermography images. The models were trained under three conditions of data augmentation. A novel method based on modifying the images through the change of the amplitude in the Fourier Transform is proposed, being the first work to perform such synergy in the characterization of risk in ulcers through thermographies. The results show that the proposed method allowed reaching the highest values, reaching a perfect classification through the convolutional neural network ResNet50v2, promising for limited data sets in thermal pattern classification problems.

Słowa kluczowe

EN

artificial intelligence; computational intelligence; diabetes mellitus; image classification; machine learning; medical diagnostic imaging

PL

sztuczna inteligencja; inteligencja obliczeniowa; cukrzyca; klasyfikacja obrazu; uczenie maszynowe; diagnostyczne obrazowanie medyczne

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2022
• Tom: Vol. 42, no. 2
• Strony: 437--452
• Opis fizyczny: Bibliogr. 63 poz., rys., tab., wykr.

Twórcy

autor

Anaya-Isaza Andrés

• Pontificia Universidad Javeriana, 110231 Bogotá, Colombia
• INDIGO Research, Bogotá, Colombia

autor

Zequera-Diaz Martha

• Electronics Department, BASPI - FootLab, Pontificia Universidad Javeriana, Bogotá, Colombia

Bibliografia

• [1] Federación-Internacional-de-Diabetes. Atlas de la diabetes 2019. https://www.diabetesatlas.org/upload/ resources/material/20200302_133352_2406-IDF-ATLAS-SPANBOOK.pdf (accessed October 8, 2021).
• [2] World Health Organization. Diabetes n.d. https://www.who. int/news-room/fact-sheets/detail/diabetes (accessed October 8, 2021).
• [3] Cho NH, Shaw JE, Karuranga S, Huang Y, da Rocha Fernandes JD, Ohlrogge AW, et al. IDF Diabetes Atlas: Global estimates of diabetes prevalence for 2017 and projections for 2045. Diabetes Res Clin Pract 2018;138:271–81.
• [4] Ingaramo RA. Obesity, diabetes, and other cardiovascular risk factors in native populations of South America. Curr Hypertens Rep 2016;18:9. https://doi.org/10.1007/s11906-015-0613-6.
• [5] Egan AM, Dinneen SF. What is diabetes? Med (United Kingdom) 2014;42:679–81. https://doi.org/10.1016/j. Mpmed.2014.09.005.
• [6] French EK, Donihi AC, Korytkowski MT. Diabetic ketoacidosis and hyperosmolar hyperglycemic syndrome: Review of acute decompensated diabetes in adult patients. BMJ 2019;365. https://doi.org/10.1136/bmj.l1114.
• [7] Einarson TR, Acs A, Ludwig C, Panton UH. Prevalence of cardiovascular disease in type 2 diabetes: a systematic literature review of scientific evidence from across the world in 2007–2017. Cardiovasc Diabetol 2018;17:83. https://doi.org/10.1186/s12933-018-0728-6.
• [8] Gerstein HC. Diabetes: Dysglycaemia as a cause of cardiovascular outcomes. Nat Rev Endocrinol 2015;11:508–10. https://doi.org/10.1038/nrendo.2015.118.
• [9] Harding JL, Pavkov ME, Magliano DJ, Shaw JE, Gregg EW. Global trends in diabetes complications: a review of current evidence. Diabetologia 2019;62:3–16. https://doi.org/10.1007/ s00125-018-4711-2.
• [10] Bonaca MP, Nault P, Giugliano RP, Keech AC, Pineda AL, Kanevsky E, et al. Low-density lipoprotein cholesterol lowering with evolocumab and outcomes in patients with peripheral artery disease: insights from the FOURIER trial (further cardiovascular outcomes research with PCSK9 inhibition in subjects with elevated risk). Circulation 2018;137:338–50.
• [11] Woody J. Overview of diabetic foot care for the nurse practitioner. J Nurse Pract 2020;16:28–33. https://doi.org/ 10.1016/j.nurpra.2019.08.011.
• [12] Bandyk DF. The diabetic foot: pathophysiology, evaluation, and treatment. Semin Vasc Surg 2018;31:43–8. https://doi.org/ 10.1053/j.semvascsurg.2019.02.001.
• [13] Petrova NL, Donaldson NK, Tang W, MacDonald A, Allen J, Lomas C, et al. Infrared thermography and ulcer prevention in the high-risk diabetic foot: data from a single-blind multicentre controlled clinical trial. Diabet Med 2020;37 (1):95–104.
• [14] Chastain CA, Klopfenstein N, Serezani CH, Aronoff DM. A clinical review of diabetic foot infections. Clin Podiatr Med Surg 2019;36:381–95. https://doi.org/10.1016/j.cpm.2019.02.004.
• [15] Rossboth S, Lechleitner M, Oberaigner W. Risk factors for diabetic foot complications in type 2 diabetes—A systematic review. Endocrinol Diabetes Metab 2021;4. https://doi.org/ 10.1002/edm2.175.
• [16] Polk C, Sampson MM, Roshdy D, Davidson LE. Skin and soft tissue infections in patients with diabetes mellitus. Infect Dis Clin North Am 2021;35:183–97. https://doi.org/10.1016/j. Idc.2020.10.007.
• [17] Ananian CE, Dhillon YS, Van Gils CC, Lindsey DC, Otto RJ, Dove CR, et al. A multicenter, randomized, single-blind trial comparing the efficacy of viable cryopreserved placental membrane to human fibroblast-derived dermal substitute for the treatment of chronic diabetic foot ulcers. Wound Repair Regen 2018;26(3):274–83.
• [18] Peter-Riesch B. The diabetic foot: the never-ending challenge, n.d., p. 108–34. 10.1159/000439409.
• [19] Ingelfinger JR, Armstrong DG, Boulton AJM, Bus SA. Diabetic foot ulcers and their recurrence. N Engl J Med 2017;376:2367–75. https://doi.org/10.1056/NEJMra1615439.
• [20] Bus SA, Waaijman R, Arts M, de Haart M, Busch-Westbroek T, van Baal J, et al. Effect of custom-made footwear on foot ulcer recurrence in diabetes: a multicenter randomized controlled trial. Diabetes Care 2013;36:4109–16. https://doi.org/10.2337/dc13-0996.
• [21] Veves A, Murray HJ, Young MJ, Boulton AJ. The risk of foot ulceration in diabetic patients with high foot pressure: a prospective study. Diabetologia 1992;35:660–3. https://doi. org/10.1007/BF00400259.
• [22] Bus SA, Armstrong DG, van Deursen RW, Lewis JEA, Caravaggi CF, Cavanagh PR. IWGDF guidance on footwear and offloading interventions to prevent and heal foot ulcers in patients with diabetes. Diabetes Metab Res Rev 2016;32(Suppl 1):25–36. https://doi.org/10.1002/dmrr.2697.
• [23] Agrawal K, Chauhan N. Pressure ulcers: back to the basics. Indian J Plast Surg 2012;45:244–54. https://doi.org/10.4103/ 0970-0358.101287.
• [24] Coleman S, Nixon J, Keen J, Wilson L, McGinnis E, Dealey C, et al. A new pressure ulcer conceptual framework. J Adv Nurs 2014;70(10):2222–34. https://doi.org/10.1111/jan.12405.
• [25] Dealey C, Brindle CT, Black J, Alves P, Santamaria N, Call E, et al. Challenges in pressure ulcer prevention. Int Wound J 2015;12:309–12. https://doi.org/10.1111/iwj.12107.
• [26] Pieper B. Pressure ulcers: impact, etiology, and classification, 2010.
• [27] Yavuz M, Ersen A, Hartos J, Lavery LA, Wukich DK, Hirschman GB, et al. Temperature as a causative factor in diabetic foot ulcers: A call to revisit ulceration pathomechanics. J Am Podiatr Med Assoc 2019;109:345–50. https://doi.org/10.7547/17-131.
• [28] Bharara M, Cobb JE, Claremont DJ. Thermography and thermometry in the assessment of diabetic neuropathic foot: A case for furthering the role of thermal techniques. Int J Low Extrem Wounds 2006;5:250–60. https://doi.org/10.1177/ 1534734606293481.
• [29] Ilo A, Romsi P, Mäkelä J. Infrared thermography and vascular disorders in diabetic feet. J Diabetes Technol 2019;14:28–36. https://doi.org/10.1177/1932296819871270.
• [30] Bouzida N, Bendada A, Maldague XP. Visualization of body thermoregulation by infrared imaging. J Therm Biol 2009;34:120–6. https://doi.org/10.1016/j.jtherbio.2008.11.008.
• [31] Ilo A, Romsi P, Mäkelä J. Infrared thermography as a diagnostic tool for peripheral artery disease. Adv Skin Wound Care 2020;33:482–8.
• [32] Lahiri BB, Bagavathiappan S, Jayakumar T, Philip J. Medical applications of infrared thermography: A review. Infrared Phys Technol 2012;55:221–35. https://doi.org/10.1016/j. Infrared.2012.03.007.
• [33] Bandalakunta Gururajarao S, Venkatappa U, Shivaram JM, Sikkandar MY, Al Amoudi A. Chapter 4 - Infrared thermography and soft computing for diabetic foot assessment. In: Dey N, Borra S, Ashour AS, Shi FBT-ML in BSA and DI, editors., Academic Press; 2019, p. 73–97. 10.1016/B978-0-12-816086-2.00004-7.
• [34] Fujita H. AI-based computer-aided diagnosis (AI-CAD): the latest review to read first. Radiol Phys Technol 2020;13:6–19. https://doi.org/10.1007/s12194-019-00552-4.
• [35] Ravi D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, et al. Deep learning for health informatics. IEEE J Biomed Heal Informatics 2017;21(1):4–21.
• [36] Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing 2017;234:11–26. https://doi.org/10.1016/j. Neucom.2016.12.038.
• [37] He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proc. IEEE Int. Conf. Comput. Vis., vol. 2015 Inter, 2015, p. 1026–34. 10.1109/ICCV.2015.123.
• [38] Castiglioni I, Rundo L, Codari M, Di Leo G, Salvatore C, Interlenghi M, et al. AI applications to medical images: From machine learning to deep learning. Phys Medica 2021;83:9–24.
• [39] Faust O, Rajendra Acharya U, Ng EYK, Hong TJ, Yu W. Application of infrared thermography in computer aided diagnosis. Infrared Phys Technol 2014;66:160–75. https://doi. org/10.1016/j.infrared.2014.06.001.
• [40] Suissa S, Ernst P. Optical illusions from visual data analysis: Example of the New Zealand asthma mortality epidemic. J Clin Epidemiol 1997;50:1079–88. https://doi.org/10.1016/S0895-4356(97)00158-3.
• [41] Adam M, Ng EYK, Tan JH, Heng ML, Tong JWK, Acharya UR. Computer aided diagnosis of diabetic foot using infrared thermography: A review. Comput Biol Med 2017;91:326–36. https://doi.org/10.1016/j.compbiomed.2017.10.030.
• [42] Mori T, Nagase T, Takehara K, Oe M, Ohashi Y, Amemiya A, et al. Morphological pattern classification system for plantar thermography of patients with diabetes. J Diabetes Technol 2013;7(5):1102–12.
• [43] Liu C, van Netten JJ, van Baal JG, Bus SA, van der Heijden F. Automatic detection of diabetic foot complications with infrared thermography by asymmetric analysis. J Biomed Opt 2015;20:026003. 10.1117/1.jbo.20.2.026003.
• [44] Adam M, Ng EYK, Oh SL, Heng ML, Hagiwara Y, Tan JH, et al. Automated characterization of diabetic foot using nonlinear features extracted from thermograms. Infrared Phys Technol 2018;89:325–37.
• [45] Carlos Padierna L, Fabián Amador-Medina L, Olivia Murillo- Ortiz B, Villaseñor-Mora C. Classification method of peripheral arterial disease in patients with type 2 diabetes mellitus by infrared thermography and machine learning. Infrared Phys Technol 2020;111. https://doi.org/10.1016/j.infrared.2020.103531 103531.
• [46] J JS, Prakash KVMV, Sai GA, Vinod B, Chellappan K. Classification of foot thermograms using texture features and support vector machine. In: Second Int. Conf. Electron. Sustain. Commun. Syst., IEEE; 2021, pp. 1445–9. 10.1109/ ICESC51422.2021.9532777.
• [47] Hernandez-Contreras D, Peregrina-Barreto H, Rangel-Magdaleno J, Ramirez-Cortes J, Renero-Carrillo F. Automatic classification of thermal patterns in diabetic foot based on morphological pattern spectrum. Infrared Phys Technol 2015;73:149–57. https://doi.org/10.1016/j.infrared.2015.09.022.
• [48] Cruz-Vega I, Hernandez-Contreras D, Peregrina-Barreto H, de J, Rangel-Magdaleno, Ramirez-Cortes JM. Deep learning classification for diabetic foot thermograms. Sensors 2020;20. https://doi.org/10.3390/s20061762.
• [49] Khandakar A, Chowdhury MEH, Ibne Reaz MB, Md Ali SH, Hasan MA, Kiranyaz S, et al. A machine learning model for early detection of diabetic foot using thermogram images. Comput Biol Med 2021;137:104838.
• [50] Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data 2019;6:60. https:// doi.org/10.1186/s40537-019-0197-0.
• [51] Hernandez-Contreras DA, Peregrina-Barreto H, Rangel-Magdaleno JdeJ, Renero-Carrillo FJ. Plantar thermogram database for the study of diabetic foot complications. IEEE Access 2019;7:161296–307.
• [52] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44. https://doi.org/10.1038/nature14539.
• [53] Erickson BJ. Basic artificial intelligence techniques. Radiol Clin North Am 2021;59:933–40. https://doi.org/10.1016/j. Rcl.2021.06.004.
• [54] Kuo C-C-J. Understanding convolutional neural networks with a mathematical model. J Vis Commun Image Represent 2016;41:406–13. https://doi.org/10.1016/j.jvcir.2016.11.003.
• [55] Chlap P, Min H, Vandenberg N, Dowling J, Holloway L, Haworth A. A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 2021;65:545–63. https://doi.org/10.1111/1754- 9485.13261.
• [56] Haykin S. Multilayer perceptrons. Neural Networks Learn Mach 2009:122–5.
• [57] Karhunen J, Raiko T, Cho K. Unsupervised deep learning. Adv Indep Compon Anal Learn Mach 2015:125–42. https://doi.org/ 10.1016/B978-0-12-802806-3.00007-5.
• [58] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86:2278–324. https://doi.org/10.1109/5.726791.
• [59] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Adv. Neural Inf. Process. Syst., vol. 25, Curran Associates, Inc.; 2012.
• [60] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 IEEE Conf. Comput. Vis. Pattern Recognit., IEEE; 2016, p. 770–8. 10.1109/CVPR.2016.90.
• [61] He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. Comput Vis – ECCV 2016 2016:630–45. 10.1007/978-3-319-46493-0_38.
• [62] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need 2017.
• [63] Nagase T, Sanada H, Takehara K, Oe M, Iizaka S, Ohashi Y, et al. Variations of plantar thermographic patterns in normal controls and non-ulcer diabetic patients: Novel classification using angiosome concept. J Plast Reconstr Aesthetic Surg 2011;64:860–6. https://doi.org/10.1016/j.bjps.2010.12.003.