Wyniki wyszukiwania - BazTech

Ograniczanie wyników

Znaleziono wyników: 3

Liczba wyników na stronie

Wyniki wyszukiwania

Wyszukiwano:
w słowach kluczowych: gradient descent

Sortuj według:

Ogranicz wyniki do:

Traffic Signal Settings Optimization Using Gradient Descent

Możejko Marcin, Brzeski Maciej, Mądry Łukasz, Skowronek Łukasz, Gora Paweł

Schedae Informaticae

2018

Vol. 27

19--30

We investigate performance of a gradient descent optimization (GR) applied to the traffic signal setting problem and compare it to genetic algorithms. We used neural networks as metamodels evaluating quality of signal settings and discovered that both optimization methods produce similar results, e.g., in both cases the accuracy of neural networks close to local optima depends on an activation function (e.g., TANH activation makes optimization process converge to different minima than ReLU activation).

lossgrad : Automatic Learning Rate in Gradient Descent

Wójcik Bartosz, Maziarka Łukasz, Tabor Jacek

Schedae Informaticae

2018

Vol. 27

47--57

In this paper, we propose a simple, fast and easy to implement algorithm lossgrad (locally optimal step-size in gradient descent), which au- tomatically modifies the step-size in gradient descent during neural networks training. Given a function f, a point x, and the gradient rxf of f, we aim to nd the step-size h which is (locally) optimal, i.e. satisfies: h = arg min t0 f(x 􀀀 trxf): Making use of quadratic approximation, we show that the algorithm satisfies the above assumption. We experimentally show that our method is insensitive to the choice of initial learning rate while achieving results comparable to other methods.

Convergence Analysis of Inverse Iterative Neural Networks with L₂ Penalty

Wen Y., Wang J., Huang B., Zurada J. M.

Journal of Applied Computer Science Methods

2016

Vol. 8 No. 2

85--98

The iterative inversion of neural networks has been used in solving problems of adaptive control due to its good performance of information processing. In this paper an iterative inversion neural network with L₂ penalty term has been presented trained by using the classical gradient descent method. We mainly focus on the theoretical analysis of this proposed algorithm such as monotonicity of error function, boundedness of input sequences and weak (strong) convergence behavior. For bounded property of inputs, we rigorously proved that the feasible solutions of input are restricted in a measurable field. The weak convergence means that the gradient of error function with respect to input tends to zero as the iterations go to infinity while the strong convergence stands for the iterative sequence of input vectors convergence to a fixed optimal point.