Adaptive Differential Evolution with Elite Opposition-Based Learning and its Application to Training Artificial Neural Networks

• Autorzy: Choi Tae Jong , Lee Jong-Hyun , Youn Hee Yong , Ahn Chang Wook

Identyfikatory

DOI

10.3233/FI-2019-1764

• Języki publikacji: EN

Abstrakty

EN

Differential Evolution (DE) algorithm is one of the popular evolutionary algorithms that is designed to find a global optimum on multi-dimensional continuous problems. In this paper, we propose a new variant of DE algorithm by combining a self-adaptive DE algorithm called dynNP-DE with Elite Opposition-Based Learning (EOBL) scheme. Since dynNP-DE algorithm uses a small number of population size in the later of the search process, the population diversity becomes low, and therefore premature convergence may occur. We have therefore extended an OBL scheme to dynNP-DE algorithm to overcome this shortcoming and improve the optimization performance. By combining EOBL scheme to dynNP-DE algorithm, the population diversity can be supplemented because not only the information of individuals but also their opposition information can be utilized. We measured the optimization performance of the proposed algorithm on CEC 2005 benchmark problems and breast cancer detection, which is a research field that has recently attracted a lot of attention. It was verified that the proposed algorithm could find better solutions than five state-of-the-art DE algorithms.

Słowa kluczowe

EN

artificial neural networks; differential evolution algorithm; feed-forward neural network; neural network training; opposition-based learning

• Wydawca: IOS Press
• Czasopismo: Fundamenta Informaticae
• Rocznik: 2019
• Tom: Vol. 164, nr 2-3
• Strony: 227--242
• Opis fizyczny: Bibliogr. 39 poz., tab.

Twórcy

autor

Choi Tae Jong

• College of Software, Sungkyunkwan University (SKKU), 2066 Seobu-ro Jangan-gu Suwon-si Gyeonggi-do, Republic of Korea

autor

Lee Jong-Hyun

• College of Software, Sungkyunkwan University (SKKU), 2066 Seobu-ro Jangan-gu Suwon-si Gyeonggi-do, Republic of Korea

autor

Youn Hee Yong

• College of Software, Sungkyunkwan University (SKKU), 2066 Seobu-ro Jangan-gu Suwon-si Gyeonggi-do, Republic of Korea

autor

Ahn Chang Wook

• School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), 123 Cheomdangwagi-ro Buk-gu Gwangju, Republic of Korea

Bibliografia

• [1] Storn R, Price K. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization. 1997;11(4):341-359. doi:10.1023/A:1008202821328.
• [2] Das S, Suganthan PN. Differential evolution: a survey of the state-of-the-art. IEEE transactions on evolutionary computation. 2011;15(1):4-31.
• [3] Das S, Mullick SS, Suganthan PN. Recent advances in differential evolution-an updated survey. Swarm and Evolutionary Computation. 2016;27:1-30. URL https://doi.org/10.1016/j.swevo.2016.01.004.
• [4] Choi TJ, Ahn CW. Artificial life based on boids model and evolutionary chaotic neural networks for creating artworks. Swarm and Evolutionary Computation. 2017, pp. 1-30. URL https://doi.org/10.1016/j.swevo.2017.09.003.
• [5] Brest J, Maučec MS. Population size reduction for the differential evolution algorithm. Applied Intelligence. 2008;29(3):228-247. doi:10.1007/s10489-007-0091-x.
• [6] Zhou X, Wu Z, Wang H. Elite opposition-based differential evolution for solving large-scale optimization problems and its implementation on GPU. In: Parallel and Distributed Computing, Applications and Technologies (PDCAT), 2012 13th International Conference on. IEEE; 2012. p. 727-732. doi:10.1109/PDCAT.2012.70.
• [7] Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, et al. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report. 2005;2005005:2005.
• [8] Lichman M. UCI machine learning repository; 2013.
• [9] Brest J, Greiner S, Boskovic B, Mernik M, Zumer V. Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE transactions on evolutionary computation. 2006;10(6):646-657. doi:10.1109/TEVC.2006.872133.
• [10] Zhang J, Sanderson AC. JADE: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation. 2009;13(5):945-958. doi:10.1109/TEVC.2009.2014613.
• [11] Tanabe R, Fukunaga A. Success-history based parameter adaptation for differential evolution. In: Evolutionary Computation (CEC), 2013 IEEE Congress on. IEEE; 2013. pp. 71-78. doi:0.1109/CEC.2013.6557555.
• [12] Choi TJ, Ahn CW, An J. An adaptive cauchy differential evolution algorithm for global numerical optimization. The Scientific World Journal. 2013;2013:969734. URL http://dx.doi.org/10.1155/2013/969734.
• [13] Choi TJ, Ahn CW. An adaptive differential evolution algorithm with automatic population resizing for global numerical optimization. In: Bio-Inspired Computing-Theories and Applications. Springer; 2014. pp. 68-72. doi:10.1007/978-3-662-45049-9_11.
• [14] Choi TJ, Ahn CW. An adaptive population resizing scheme for differential evolution in numerical optimization. Journal of Computational and Theoretical Nanoscience. 2015;12(7):1336-1350. URL https://doi.org/10.1166/jctn.2015.3895.
• [15] Choi TJ, Ahn CW. An adaptive cauchy differential evolution algorithm with population size reduction and modified multiple mutation strategies. In: Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems-Volume 2. Springer; 2015. pp. 13-26. doi:10.1007/978-3-319-13356-0_2.
• [16] Choi TJ, Ahn CW. Adaptive α-stable differential evolution in numerical optimization. Natural Computing. 2017;16(4):637-657. doi:10.1007/s11047-016-9579-9.
• [17] Al-Dabbagh RD, Neri F, Idris N, Baba MS. Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy. Swarm and Evolutionary Computation. 2018. URL https://doi.org/10.1016/j.swevo.2018.03.008.
• [18] Qin AK, Huang VL, Suganthan PN. Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE transactions on Evolutionary Computation. 2009;13(2):398-417. doi:10.1109/TEVC.2008.927706.
• [19] Piotrowski AP. Review of differential evolution population size. Swarm and Evolutionary Computation. 2017;32:1-24. URL https://doi.org/10.1016/j.swevo.2016.05.003.
• [20] Choi TJ, Lee Y. Asynchronous differential evolution with selfadaptive parameter control for global numerical optimization. In: MATEC Web of Conferences. vol. 189. EDP Sciences; 2018. p. 03020. URL https://doi.org/10.1051/matecconf/201818903020.
• [21] Choi TJ, Ahn CW. An Adaptive Cauchy Differential Evolution Algorithm with Bias Strategy Adaptation Mechanism for Global Numerical Optimization. JCP. 2014;9(9):2139-2145. doi:10.4304/jcp.9.9.2139-2145.
• [22] Zhabitskaya E, Zhabitsky M. Asynchronous differential evolution. In: Mathematical Modeling and Computational Science. Springer; 2012. pp. 328-333. doi:10.1007/978-3-642-28212-6_41.
• [23] Ali M, Pant M. Improving the performance of differential evolution algorithm using Cauchy mutation. Soft Computing. 2011;15(5):991-1007. doi:10.1007/s00500-010-0655-2.
• [24] Choi TJ, Ahn CW. Accelerating differential evolution using multiple exponential cauchy mutation. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM; 2018. p. 207-208.
• [25] Tizhoosh HR. Opposition-based learning: a new scheme for machine intelligence. In: Computational intelligence for modelling, control and automation, 2005 and international conference on intelligent agents, web technologies and internet commerce, international conference on. vol. 1. IEEE; 2005. p. 695-701. doi:10.1109/CIMCA.2005.1631345.
• [26] Rahnamayan S, Tizhoosh HR, Salama MM. Opposition-based differential evolution. IEEE Transactions on Evolutionary computation. 2008;12(1):64-79. doi:10.1109/TEVC.2007.894200.
• [27] Mahdavi S, Rahnamayan S, Deb K. Opposition based learning: A literature review. Swarm and evolutionary computation. 2018;39:1-23. URL https://doi.org/10.1016/j.swevo.2017.09.010.
• [28] Rahnamayan S, Tizhoosh HR, Salama MM. Opposition-based differential evolution (ODE) with variable jumping rate. In: Foundations of Computational Intelligence, 2007. FOCI 2007. IEEE Symposium on. IEEE; 2007. pp. 81-88. doi:10.1109/FOCI.2007.372151.
• [29] Rahnamayan S, Tizhoosh HR, Salama MM. Quasi-oppositional differential evolution. In: Evolutionary Computation, 2007. CEC 2007. IEEE Congress on. IEEE; 2007. pp. 2229-2236. doi:10.1109/CEC.2007.4424748.
• [30] Xu Q, Wang L, He B, Wang N. Modified opposition-based differential evolution for function optimization. Journal of Computational Information Systems. 2011;7(5):1582-1591. URL http://www.Jofcis.com.
• [31] Esmailzadeh A, Rahnamayan S. Enhanced differential evolution using center-based sampling. In: Evolutionary Computation (CEC), 2011 IEEE Congress on. IEEE; 2011. pp. 2641-2648. doi:10.1109/CEC.2011.5949948.
• [32] Esmailzadeh A, Rahnamayan S. Opposition-based differential evolution with protective generation jumping. In: Differential Evolution (SDE), 2011 IEEE Symposium on. IEEE; 2011. pp. 1-8. doi:10.1109/SDE.2011.5952059.
• [33] Wang H, Wu Z, Rahnamayan S. Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems. Soft Computing. 2011;15(11):2127-2140. doi:10.1007/s00500-010-0642-7.
• [34] Park SY, Lee JJ. Stochastic opposition-based learning using a beta distribution in differential evolution. IEEE transactions on cybernetics. 2016;46(10):2184-2194. doi:10.1109/TCYB.2015.2469722.
• [35] Ilonen J, Kamarainen JK, Lampinen J. Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters. 2003;17(1):93-105. doi:10.1023/A:1022995128597.
• [36] Slowik A. Application of an adaptive differential evolution algorithm with multiple trial vectors to artificial neural network training. IEEE Transactions on Industrial Electronics. 2011;58(8):3160-3167. doi:10.1109/TIE.2010.2062474.
• [37] Piotrowski AP. Differential evolution algorithms applied to neural network training suffer from stagnation. Applied Soft Computing. 2014;21:382-406. doi:https://doi.org/10.1016/j.asoc.2014.03.039.
• [38] Choi TJ, Ahn CW. An Improved Differential Evolution Algorithm and Its Application to Large-Scale Artificial Neural Networks. In: Journal of Physics: Conference Series. IOP Publishing, 2007;806(1):012010. doi:10.1088/1742-6596/806/1/012010.
• [39] Choi TJ, Ahn CW. Adaptive Cauchy Differential Evolution with Strategy Adaptation and Its Application to Training Large-Scale Artificial Neural Networks. In: International Conference on Bio-Inspired Computing: Theories and Applications. Springer; 2017. pp. 502-510. doi:10.1007/978-981-10-7179-9_39.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).