Wyniki wyszukiwania - BazTech

1

An enhanced differential evolution algorithmwith adaptive weight bounds for efficient training ofneural networks

Limtrakul Saithip, Wetweerapong Jeerayut

Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska

|

2023

|

T. 13, nr 1

4--13

EN

Artificial neural networks are essential intelligent tools for various learning tasks. Training them is challenging due to the nature of the data set, many training weights, and their dependency, which gives rise to a complicated high-dimensional error function for minimization. Thus, global optimization methods have become an alternative approach. Many variants of differential evolution (DE) have been applied as training methods to approximate the weights of a neural network. However, empirical studies show that they suffer from generally fixed weight bounds. In this research, we propose an enhanced differential evolution algorithm with adaptive weight bound adjustment (DEAW) for the efficient training of neural networks. The DEAW algorithm uses small initial weight bounds and adaptive adjustment in the mutation process. It gradually extends the bounds when a component of a mutant vector reaches its limits. We also experiment with using several scales of an activation function with the DEAW algorithm. Then, we apply the proposed method with its suitable setting to solve function approximation problems. DEAW can achieve satisfactory results compared to exact solutions.

PL

Sztuczne sieci neuronowe są niezbędnymi inteligentnymi narzędziami do realizacji różnych zadań uczenia się. Ich szkolenie stanowi wyzwanie ze względu na charakter zbioru danych, wiele wag treningowych i ich zależności, co powoduje powstanie skomplikowanej, wielowymiarowej funkcji błędu do minimalizacji. Dlatego alternatywnym podejściem stały się metody optymalizacji globalnej. Wiele wariantów ewolucji różnicowej (DE) zostało zastosowanych jako metody treningowe do aproksymacji wag sieci neuronowej. Jednak badania empiryczne pokazują, że cierpią one z powodu ogólnie ustalonych granic wag. W tym badaniu proponujemy ulepszony algorytm ewolucji różnicowej z adaptacyjnym dopasowaniem granic wag (DEAW) dla efektywnego szkolenia sieci neuronowych. Algorytm DEAW wykorzystuje małe początkowe granice wag i adaptacyjne dostosowanie w procesie mutacji. Stopniowo rozszerza on granice, gdy składowa wektora mutacji osiąga swoje granice. Eksperymentujemy również z wykorzystaniem kilku skal funkcji aktywacji z algorytmem DEAW. Następnie, stosujemy proponowaną metodę z jej odpowiednim ustawieniem do rozwiązywania problemów aproksymacji funkcji. DEAW może osiągnąć zadowalające rezultaty w porównaniu z rozwiązaniami dokładnymi.

2

Adaptive differential evolution algorithm with a pheromone-based learning strategy for global continuous optimization

Singsathid Pirapong, Puphasuk Pikul, Wetweerapong Jeerayut

Foundations of Computing and Decision Sciences

|

2023

|

Vol. 48, No. 2

243--266

EN

Differential evolution algorithm (DE) is a well-known population-based method for solving continuous optimization problems. It has a simple structure and is easy to adapt to a wide range of applications. However, with suitable population sizes, its performance depends on the two main control parameters: scaling factor (F) and crossover rate (CR). The classical DE method can achieve high performance by a time-consuming tunning process or a sophisticated adaptive control implementation. We propose in this paper an adaptive differential evolution algorithm with a pheromone-based learning strategy (ADE-PS) inspired by ant colony optimization (ACO). The ADE-PS embeds a pheromone-based mechanism that manages the prob- abilities associated with the partition values of F and CR. It also introduces a resetting strategy to reset the pheromone at a specific time to unlearn and relearn the progressing search. The preliminary experiments find a suitable number of subintervals (ns) for partitioning the control parameter ranges and the reset period (rs) for resetting the pheromone. Then the comparison experiments evaluate ADE-PS using the suitable ns and rs against some adaptive DE methods in the literature. The results show that ADE-PS is more reliable and outperforms several well-known methods in the literature.

3

"An enhanced differential evolution algorithm with adaptation of switching crossover strategy for continuous optimization"

Puphasuk Pikul, Wetweerapong Jeerayut

Foundations of Computing and Decision Sciences

|

2020

|

Vol. 45, No. 2

97--124

EN

Designing an efficient optimization method which also has a simple structure is generally required by users for its applications to a wide range of practical problems. In this research, an enhanced differential evolution algorithm with adaptation of switching crossover strategy (DEASC) is proposed as a general-purpose population-based optimization method for continuous optimization problems. DEASC extends the solving ability of a basic differential evolution algorithm (DE) whose performance significantly depends on user selection of the control parameters: scaling factor, crossover rate and population size. Like the original DE, the proposed method is aimed at efficiency, simplicity and robustness. The appropriate population size is selected to work in accordance with good choices of the scaling factors. Then, the switching crossover strategy of using low or high crossover rates are incorporated and adapted to suit the problem being solved. In this manner, the adaptation strategy is just a convenient add-on mechanism. To verify the performance of DEASC, it is tested on several benchmark problems of various types and difficulties, and compared with some well-known methods in the literature. It is also applied to solve some practical systems of nonlinear equations. Despite its much simpler algorithmic structure, the experimental results show that DEASC greatly enhances the basic DE. It is able to solve all the test problems with fast convergence speed and overall outperforms the compared methods which have more complicated structures. In addition, DEASC also shows promising results on high dimensional test functions.