EO-CNN: Equilibrium Optimization-Based hyperparameter tuning for enhanced pneumonia and COVID-19 detection using AlexNet and DarkNet19

• Autorzy: Kiziloluk Soner , Sert Eser , Hammad Mohamed , Tadeusiewicz Ryszard , Pławiak Paweł

Identyfikatory

DOI

10.1016/j.bbe.2024.06.006

• Języki publikacji: EN

Abstrakty

EN

Convolutional neural networks (CNN) have been increasingly popular in image categorization in recent years. Hyperparameter optimization is a critical stage in enhancing the effectiveness of CNNs and achieving better results. Properly tuning hyperparameters allows the model to exhibit improved performance and facilitates faster learning. Misconfigured hyperparameters can prolong the training time or lead to the model not learning at all. Manually tuning hyperparameters is a time-consuming and challenging process. Automatically adjusting hyperparameters helps save time and resources. This study aims to propose an approach that shows higher classification performance than unoptimized convolutional neural network models, even at low epoch values, by automatically optimizing the hyperparameters of AlexNet and DarkNet19 with equilibrium optimization, the newest metaheuristic algorithm. In this respect, the proposed approach optimizes the number and size of filters in the first five convolutional layers in AlexNet and DarkNet19 using an equilibrium optimization algorithm. To evaluate the efficacy of the suggested method, experimental analyses were conducted on the pneumonia and COVID-19 datasets. An important advantage of this approach is its ability to accurately classify medical images. The testing process suggests that utilizing the proposed approach to optimize hyperparameters for AlexNet and DarkNet19 led to a 7% and 4.07% improvement, respectively, in image classification accuracy compared to non-optimized versions of the same networks. Furthermore, the approach displayed superior classification performance even in a few epochs compared to AlexNet, ShuffleNet, DarkNet19, GoogleNet, MobileNet-V2, VGG-16, VGG-19, ResNet18, and Inceptionv3. As a result, automatic tuning of the hyperparameters of AlexNet and DarkNet-19 with EO enabled the performance of these two models to increase significantly.

Słowa kluczowe

EN

AlexNet; DarkNet19; convolutional neural networks; CNN; equilibrium optimisation; EO; image classification

PL

AlexNet; DarkNet19; sieć neuronowa konwolucyjna; klasyfikacja obrazu

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2024
• Tom: Vol. 44, no. 3
• Strony: 635--650
• Opis fizyczny: Bibliogr. 74 poz., rys., tab., wykr.

Twórcy

autor

Kiziloluk Soner

• Department of Computer Engineering, Malatya Turgut Özal University, Malatya, Turkey

autor

Sert Eser

• Department of Computer Engineering, Malatya Turgut Özal University, Malatya, Turkey

autor

Hammad Mohamed

• EIAS Data Science Lab, College of Computer and Information Sciences, Prince Sultan University, Riyadh 11586, Saudi Arabia
• Department of Information Technology, Faculty of Computers and Information, Menoufia University, Shibin El Kom 32511, Egypt

autor

Tadeusiewicz Ryszard

• AGH University of Science and Technology, Department of Biocybernetics and Biomedical Engineering, Krakow, Poland

autor

Pławiak Paweł

• Department of Computer Science, Faculty of Computer Science and Telecommunications, Cracow University of Technology, Krakow, Poland
• Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Gliwice, Poland

Bibliografia

• [1] Wang Y, Zhang H, Zhang G. cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks. Swarm Evol Comput 2019;49:114-23. https://doi.org/10.1016/j.swevo.2019.06.002.
• [2] Kızıloluk S, Sert E. Hurricane-Faster R-CNN-JS: Hurricane detection with faster RCNN using artificial Jellyfish Search (JS) optimizer. Multimed Tools Appl 2022;81: 37981-99. https://doi.org/10.1007/s11042-022-13156-9.
• [3] Albani RAS, Albani VVL, Silva Neto AJ. Source characterization of airborne pollutant emissions by hybrid metaheuristic/gradient-based optimization techniques. Environ Pollut 2020;267:115618. https://doi.org/10.1016/j.envpol.2020.115618.
• [4] Kiziloluk S, Sert E. COVID-CCD-Net: COVID-19 and colon cancer diagnosis system with optimized CNN hyperparameters using gradient-based optimizer. Med Biol Eng Comput 2022;60:1595-612. https://doi.org/10.1007/s11517-022-02553-9.
• [5] Murugan R, Goel T, Mirjalili S, Chakrabartty DK. WOANet: Whale optimized deep neural network for the classification of COVID-19 from radiography images. Biocybern Biomed Eng 2021;41:1702-18. https://doi.org/10.1016/j.bbe.2021.10.004.
• [6] Zhou L, Zhao C, Liu N, Yao X, Cheng Z. Improved LSTM-based deep learning model for COVID-19 prediction using optimized approach. Eng Appl Artif Intell 2023;122: 106157. https://doi.org/10.1016/j.engappai.2023.106157.
• [7] Nalluri S, Sasikala R. Pneumonia screening on chest X-rays with optimized ensemble model. Expert Syst Appl 2024;242:122705. https://doi.org/10.1016/j.eswa.2023.122705.
• [8] Faramarzi A, Heidarinejad M, Stephens B, Mirjalili S. Equilibrium optimizer: A novel optimization algorithm. Knowl-Based Syst 2020;191:105190. https://doi.org/10.1016/j.knosys.2019.105190.
• [9] Sharma P, Dinkar SK. An intelligent deep neural network with Opposition based Laplacian Equilibrium Optimizer to improve feature extraction using ECG signals. Biomed Signal Process Control 2024;87:105415. https://doi.org/10.1016/j.bspc.2023.105415.
• [10] Seleem SI, Hasanien HM, El-Fergany AA. Equilibrium optimizer for parameter extraction of a fuel cell dynamic model. Renew Energy 2021;169:117-28. https://doi.org/10.1016/j.renene.2020.12.131.
• [11] Wang J, Yang B, Li D, Zeng C, Chen Y, Guo Z, et al. Photovoltaic cell parameter estimation based on improved equilibrium optimizer algorithm. Energy Convers Manag 2021;236:114051. https://doi.org/10.1016/j.enconman.2021.114051.
• [12] Mohakud R, Dash R. Designing a grey wolf optimization based hyper-parameter optimized convolutional neural network classifier for skin cancer detection. J King Saud Univ - Comput Inf Sci 2022;34:6280-91. https://doi.org/10.1016/j.jksuci.2021.05.012.
• [13] Gülmez B. A novel deep neural network model based Xception and genetic algorithm for detection of COVID-19 from X-ray images. Annals of Operations Research 2023;328:617-41. https://doi.org/10.1007/s10479-022-05151-y.
• [14] Kaya M. Feature fusion-based ensemble CNN learning optimization for automated detection of pediatric pneumonia. Biomed Signal Process Control 2024;87:105472. https://doi.org/10.1016/j.bspc.2023.105472.
• [15] Pradhan AK, Mishra D, Das K, Obaidat MS, Kumar M. A COVID-19 X-ray image classification model based on an enhanced convolutional neural network and hill climbing algorithms. Multimed Tools Appl 2023;82:14219-37. https://doi.org/10.1007/s11042-022-13826-8.
• [16] Pradhan AK, Das K, Mishra D, Chithaluru P. Optimizing CNN-LSTM hybrid classifier using HCA for biomedical image classification. Expert Syst 2023;40: e13235.
• [17] Balmuri KR, Konda S, Mamidala KK, Gunda M, Rani BS. Automated and reliable detection of multi-diseases on chest X-ray images using optimized ensemble transfer learning. Expert Syst Appl 2024;246:122810. https://doi.org/10.1016/j.eswa.2023.122810.
• [18] Parthasarathy V, Saravanan S. Chaotic Sea Horse Optimization with Deep Learning Model for lung disease pneumonia detection and classification on chest X-ray images. Multimed Tools Appl 2024. https://doi.org/10.1007/s11042-024-18301-0.
• [19] Mahdaddi A, Meshoul S, Belguidoum M. EA-based hyperparameter optimization of hybrid deep learning models for effective drug-target interactions prediction. Expert Syst Appl 2021;185:115525. https://doi.org/10.1016/j.eswa.2021.115525.
• [20] Fatyanosa TN, Aritsugi M. An Automatic Convolutional Neural Network Optimization Using a Diversity-Guided Genetic Algorithm. IEEE Access 2021;9: 91410-26. https://doi.org/10.1109/ACCESS.2021.3091729.
• [21] Hussein AM, Sharifai AG, Alia OM, Abualigah L, Almotairi KH, Abujayyab SKM, et al. Auto-detection of the coronavirus disease by using deep convolutional neural networks and X-ray photographs. Sci Rep 2024;14:534. https://doi.org/10.1038/s41598-023-47038-3.
• [22] Kaur BP, Singh H, Hans R, Sharma SK, Kaushal C, Hassan MM, et al. An augmentation aided concise CNN based architecture for COVID-19 diagnosis in real time. Sci Rep 2024;14:1136. https://doi.org/10.1038/s41598-024-51317-y.
• [23] Yi R, Tang L, Tian Y, Liu J, Wu Z. Identification and classification of pneumonia disease using a deep learning-based intelligent computational framework. Neural Comput Appl 2023;35:14473-86. https://doi.org/10.1007/s00521-021-06102-7.
• [24] Singh S, Kumar M, Kumar A, Verma BK, Shitharth S. Pneumonia detection with QCSA network on chest X-ray. Sci Rep 2023;13:9025. https://doi.org/10.1038/s41598-023-35922-x.
• [25] Mann PS, Panchal SD, Singh S, Saggu GS, Gupta K. A hybrid deep convolutional neural network model for improved diagnosis of pneumonia. Neural Comput Appl 2024;36:1791-804. https://doi.org/10.1007/s00521-023-09147-y.
• [26] Erdogan Yildirim A, Canayaz M. A novel deep learning-based approach for prediction of neonatal respiratory disorders from chest X-ray images. Biocybern Biomed Eng 2023;43:635-55. https://doi.org/10.1016/j.bbe.2023.08.004.
• [27] Szepesi P, Szilágyi L. Detection of pneumonia using convolutional neural networks and deep learning. Biocybern Biomed Eng 2022;42:1012-22. https://doi.org/10.1016/j.bbe.2022.08.001.
• [28] Nahiduzzaman Md, Faruq Goni MO, Robiul Islam Md, Sayeed A, Shamim Anower Md, Ahsan M, et al. Detection of various lung diseases including COVID-19 using extreme learning machine algorithm based on the features extracted from a lightweight CNN architecture. Biocybern. Biomed Eng 2023;43:528-50. https://doi.org/10.1016/j.bbe.2023.06.003.
• [29] Gour M, Jain S. Automated COVID-19 detection from X-ray and CT images with stacked ensemble convolutional neural network. Biocybern Biomed Eng 2022;42: 27-41. https://doi.org/10.1016/j.bbe.2021.12.001.
• [30] Argho AG, Maswood MMS, Mahmood MI, Mondol N. EfficientCovNet: A CNNbased approach to detect various pulmonary diseases including COVID-19 using modified EfficientNet. Intell Syst Appl 2024;21:200315. https://doi.org/10.1016/j.iswa.2023.200315.
• [31] Abdullah M, Ftsum Berhe A, Kedir B, Takore TT. A Hybrid Deep Learning CNN model for COVID-19 detection from chest X-rays. Heliyon 2024:e26938.
• [32] Abdel-Basset M, Chang V, Mohamed R. A novel equilibrium optimization algorithm for multi-thresholding image segmentation problems. Neural Comput Appl 2021; 33:10685-718. https://doi.org/10.1007/s00521-020-04820-y.
• [33] Shaheen AM, Elsayed AM, El-Sehiemy RA, Abdelaziz AY. Equilibrium optimization algorithm for network reconfiguration and distributed generation allocation in power systems. Appl Soft Comput 2021;98:106867. https://doi.org/10.1016/j.asoc.2020.106867.
• [34] Chattopadhyay S, Dey A, Singh PK, Ahmadian A, Sarkar R. A feature selection model for speech emotion recognition using clustering-based population generation with hybrid of equilibrium optimizer and atom search optimization algorithm. Multimed Tools Appl 2023;82:9693-726. https://doi.org/10.1007/s11042-021-11839-3.
• [35] Yang E, Shankar K, Kumar S, Seo C, Moon I. Equilibrium Optimization Algorithm with Deep Learning Enabled Prostate Cancer Detection on MRI Images. Biomedicines 2023;11:3200. https://doi.org/10.3390/biomedicines11123200.
• [36] Rizk-Allah RM, Hassanien AE. A hybrid equilibrium algorithm and pattern search technique for wind farm layout optimization problem. ISA Trans 2023;132:402-18. https://doi.org/10.1016/j.isatra.2022.06.014.
• [37] Guo T, Dong J, Li H, Gao Y. Simple convolutional neural network on image classification. In: 2017 IEEE 2nd Int. Conf. Big Data Anal. ICBDA. Beijing, China: IEEE; 2017. p. 721-4. https://doi.org/10.1109/ICBDA.2017.8078730.
• [38] Nhat-Duc H, Nguyen Q-L, Tran V-D. Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network. Autom Constr 2018;94:203-13. https://doi.org/10.1016/j.autcon.2018.07.008.
• [39] Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights Imaging 2018;9:611-29. https://doi.org/10.1007/s13244-018-0639-9.
• [40] Sinha T, Haidar A, Verma B. Particle Swarm Optimization Based Approach for Finding Optimal Values of Convolutional Neural Network Parameters. In: 2018 IEEE Congr. Evol. Comput. CEC. Rio de Janeiro: IEEE; 2018. p. 1-6. https://doi.org/10.1109/CEC.2018.8477728.
• [41] Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira F, Burges CJ, Bottou L, Weinberger KQ, editors. Adv Neural Inf Process Syst, vol. 25. Curran Associates, Inc.; 2012.
• [42] Redmon, J. Darknet: Open source neural networks in C, http://pjreddie.com/darknet/, 2013-2016 n.d.
• [43] Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In: 2018 IEEECVF Conf. Comput. Vis. Pattern Recognit. Salt Lake City, UT: IEEE; 2018. p. 4510-20. https://doi.org/10.1109/CVPR.2018.00474.
• [44] Zhang X, Zhou X, Lin M, Sun J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In: 2018 IEEECVF Conf. Comput. Vis. Pattern Recognit. Salt Lake City, UT: IEEE; 2018. p. 6848-56. https://doi.org/10.1109/CVPR.2018.00716.
• [45] Szegedy C, Wei Liu, Yangqing Jia, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conf. Comput. Vis. Pattern Recognit. CVPR, Boston, MA, USA: IEEE; 2015, p. 1-9. Doi: 10.1109/CVPR.2015.7298594.
• [46] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition 2015.
• [47] He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. CVPR: Proc. IEEE Conf. Comput. Vis. Pattern Recognit; 2016.
• [48] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the Inception Architecture for Computer Vision. CVPR: Proc. IEEE Conf. Comput. Vis. Pattern Recognit; 2016.
• [49] Lee W-Y, Park S-M, Sim K-B. Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm. Optik 2018;172:359-67. https://doi.org/10.1016/j.ijleo.2018.07.044.
• [50] Agrawal A, Mittal N. Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Vis Comput 2020;36: 405-12. https://doi.org/10.1007/s00371-019-01630-9.
• [51] Naseri H, Mehrdad V. Novel CNN with investigation on accuracy by modifying stride, padding, kernel size and filter numbers. Multimed Tools Appl 2023;82: 23673-91. https://doi.org/10.1007/s11042-023-14603-x.
• [52] Kim S-H, Geem ZW, Han G-T. Hyperparameter Optimization Method Based on Harmony Search Algorithm to Improve Performance of 1D CNN Human Respiration Pattern Recognition System. Sensors 2020;20:3697. https://doi.org/10.3390/s20133697.
• [53] Ghosh A, Jana ND, Das S, Mallipeddi R. Two-Phase Evolutionary Convolutional Neural Network Architecture Search for Medical Image Classification. IEEE Access 2023;11:115280-305. https://doi.org/10.1109/ACCESS.2023.3323705.
• [54] Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, et al. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell 2018;172:1122-1131.e9. https://doi.org/10.1016/j.cell.2018.02.010.
• [55] Kermany D. Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification 2018. Doi: 10.17632/RSCBJBR9SJ.2.
• [56] Rahman T, Chowdhury ME, Khandakar A. COVID-19 chest X-ray database. Kaggle Data 2020;v3. https://www.kaggle.com/tawsifurrahman/covid19-radiography-database. n.d.
• [57] Reddy BB, Sudhakar MV, Reddy PR, Reddy PR. Ensemble deep honey architecture for COVID-19 prediction using CT scan and chest X-ray images. Multimed Syst 2023;29:2009-35. https://doi.org/10.1007/s00530-023-01072-3.
• [58] Subramanian M, Sathishkumar VE, Cho J, Shanmugavadivel K. Learning without forgetting by leveraging transfer learning for detecting COVID-19 infection from CT images. Sci Rep 2023;13:8516. https://doi.org/10.1038/s41598-023-34908-z.
• [59] Ragab M, W. Al-Rabia M, Saeed Binyamin S, A. Aldarmahi A. Intelligent Firefly Algorithm Deep Transfer Learning Based COVID-19 Monitoring System. Comput Mater Contin 2023;74:2889-903. https://doi.org/10.32604/cmc.2023.032192.
• [60] Almutairi SA. A multimodal AI-based non-invasive COVID-19 grading framework powered by deep learning, manta ray, and fuzzy inference system from multimedia vital signs. Heliyon 2023;9:e16552.
• [61] Najaran MHT. A genetic programming-based convolutional deep learning algorithm for identifying COVID-19 cases via X-ray images. Artif Intell Med 2023; 142:102571. https://doi.org/10.1016/j.artmed.2023.102571.
• [62] Patro KK, Allam JP, Hammad M, Tadeusiewicz R, Pławiak P. SCovNet: A skip connection-based feature union deep learning technique with statistical approach analysis for the detection of COVID-19. Biocybern Biomed Eng 2023;43:352-68. https://doi.org/10.1016/j.bbe.2023.01.005.
• [63] Bhosale RD, Yadav DM. Customized convolutional neural network for pulmonary multi-disease classification using chest x-ray images. Multimed Tools Appl 2023; 83:18537-71. https://doi.org/10.1007/s11042-023-16297-7.
• [64] Reis HC, Turk V, Khoshelham K, Kaya S. MediNet: transfer learning approach with MediNet medical visual database. Multimed Tools Appl 2023;82:39211-54. https://doi.org/10.1007/s11042-023-14831-1.
• [65] AbouEl-Magd LM, Darwish A, Snasel V, Hassanien AE. A pre-trained convolutional neural network with optimized capsule networks for chest X-rays COVID-19 diagnosis. Clust Comput 2023;26:1389-403. https://doi.org/10.1007/s10586-022-03703-2.
• [66] Shibu George G, Raj Mishra P, Sinha P, Ranjan PM. COVID-19 detection on chest Xray images using Homomorphic Transformation and VGG inspired deep convolutional neural network. Biocybern Biomed Eng 2023;43:1-16. https://doi.org/10.1016/j.bbe.2022.11.003.
• [67] Abubeker KM, Baskar S. B2-Net: an artificial intelligence powered machine learning framework for the classification of pneumonia in chest x-ray images. Mach Learn Sci Technol 2023;4:015036. https://doi.org/10.1088/2632-2153/acc30f.
• [68] An Q, Chen W, Shao W. A Deep Convolutional Neural Network for Pneumonia Detection in X-ray Images with Attention Ensemble. Diagnostics 2024;14:390. https://doi.org/10.3390/diagnostics14040390.
• [69] Saifullah S, Yuwono B, Rustamaji HC, Saputra B, Dwiyanto FA, Dreżewski R. Detection of Chest X-ray Abnormalities Using CNN Based on Hyperparameter Optimization. 4th Int. Electron Conf Appl Sci, MDPI 2023:223. https://doi.org/10.3390/ASEC2023-16260.
• [70] Guesmi T. Detecting Pneumonia with a Deep Learning Model and Random Data Augmentation Techniques. Int J Adv Comput Sci Appl 2023:14. https://doi.org/10.14569/IJACSA.2023.01405122.
• [71] Qiao Z, Li L, Zhao X, Liu L, Zhang Q, Shili H, et al. An enhanced Runge Kutta boosted machine learning framework for medical diagnosis. Comput Biol Med 2023;160:106949. https://doi.org/10.1016/j.compbiomed.2023.106949.
• [72] Nalluri S, Sasikala R. A deep neural architecture for SOTA pneumonia detection from chest X-rays. Int J Syst Assur Eng Manag 2024;15:489-502. https://doi.org/10.1007/s13198-022-01788-x.
• [73] Çelik A, Demirel S. Enhanced Pneumonia Diagnosis Using Chest X-Ray Image Features and Multilayer Perceptron and k-NN Machine Learning Algorithms. Trait Signal 2023;40:1015-23. https://doi.org/10.18280/ts.400317.
• [74] Lafraxo S, El Ansari M, Koutti L. A new hybrid approach for pneumonia detection using chest X-rays based on ACNN-LSTM and attention mechanism. Multimed Tools Appl 2024. https://doi.org/10.1007/s11042-024-18401-x.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).