Search-based approach to multilayer perceptron training

• Autorzy: Kordos M.
• Języki publikacji: EN

Abstrakty

EN

The paper presents two search-based algorithms for MLP training; numerical gradient and yariable step search algorithm. The advantages of the methods comparing to analytical gradient-based algorithms include Iow memory reąuirements, the algorithm simplicity and determining morę optimal next step direction by direct access to the influence of hidden layer weights on nerwork error.

PL

Artykuł przedstawia dwie metody uczenia sieci MLP oparte na . przeszukiwaniu: gradient numeryczny i metodę zmiennego kroku przeszukiwania (VSS). Zaletą tych metod są małe wymagania pamięciowe, prosta budowa algorytmu t v oraz dokładniejsze określenie kierunku następnego kroku przez bezpos'rednie określanie wpływu wartości wag warstwy ukrytej na błąd sieci.

Słowa kluczowe

EN

MLP; neural networks; search algorithms

PL

MLP; sieci neuronowe; algorytmy przeszukiwań

• Wydawca: Wydawnictwo Politechniki Śląskiej
• Czasopismo: Studia Informatica
• Rocznik: 2005
• Tom: Vol. 26, nr 1
• Strony: 29--43
• Opis fizyczny: Bibliogr. 20 poz., rys.

Twórcy

autor

Kordos M.

• Politechnika Śląska, Instytut Informatyki, ul. Akademicka 16, 44-101 Gliwice, Polska,

Bibliografia

• Rumelhart, Hinton G. E., Williams R. J.: Learning Internal Representations by Error Propagation. Parallel Data Processing, Vol.1, Chapter 8, the M.I.T. Press, Cambridge, 1986, pp. 318-362.
• Riedmiller M., Braun H.: RPROP – a Fast Adaptive Learning Algorithm. Technical Report, University Karlsruhe, 1992.
• Fahlman S. E.: Faster Learning Variations of Backpropagation: an Empirical Study. Connectionist Models Summer School, Morgan Kaufmann, 1998, pp. 38-51.
• Battiti R.: First and second order methods for learning: Between steepest descent and Newton's method. Neural Computation, vol. 4, no. 2, 1992, pp. 141-166.
• Marquardt D.: An Flgorithm for Least-squares Estimation of Nonlinear Parameters. SIAM J. Appl. Math., 1963, Vol.11, pp. 431-441.
• Möller M. F.: A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning. Neural Networks, vol. 6, 1993, pp. 525-533.
• Bilski J., Rutkowski L.: Numerically Robust Learning Algorithms for Feed Forward Neural Networks. Th 6th International Conference on Neural Networks and Soft Computing, Zakopane, Poland, June 2002, pp. 149-154.
• Montana D. J.: Neural Network Weight Selection Using Genetic Algorithms. Intelligent Hybrid Systems, Wiley, New York, 1995, pp. 85-104.
• Engel J.: Teaching Feed-forward Neural Networks by Simulated Annealing. Complex Systems 2, 1988, pp. 641-648.
• Unnikrishnan K. P., Venugopal K. P.: Alopex: A Correlation-Based Learning Algorithm for Feed-Forward and Recurrent Neural Networks. Neural Computations, 6, 1994, pp. 469-490.
• Koosh V. F.: Analog Computation and Learning in VLSI. PhD Thesis, Caltech, Pasadena, CA, 2001.
• Shang Y., Wah B. W.: Global Optimization for Neural Network Training. IEEE Computer, 29, 1996, pp. 45-54.
• Battiti R., Tecchiolli G.: Training Neural Nets with the Reactive Tabu Search, Transactions on Neural Networks, vol.6, 1995, , pp. 1185-1200.
• Gupta H. V., Hsu K., Sorooshian S.: Superior Training of Artificial Neural Networks Using Weight-Space Partitioning. International Conference on Neural Networks. Houston, USA, 1997, pp. 1919-1923.
• Gallagher M.: Multi-layer Perceptron Error Surfaces: Visualization, Structure and Modeling. PhD Thesis, University of Queensland, 2000.
• Kordos M., Duch W.: Search-based Training for Logical Rule Extraction by Multilayer Perceptron. International Conference on Artificial Neural Networks (ICANN) Istanbul, Turkey, 2003, pp. 126-129.
• Kordos M., Duch W.: Multilayer Perceptron Trained with Numerical Gradient. International Conference on Artificial Neural Networks (ICANN), Istanbul, Turkey, 2003, pp. 106-109.
• Kordos M., Duch W.: Variable Step Search Algorithm for MLP Training. The 8th IASTED International Conference on Artificial Intelligence and Soft Computing, Marbella, Spain, 2004, pp. 215-220.
• Kordos M., Duch W.: On Some Factors Influencing MLP Error Surface. The 7th International Conference on Artificial Intelligence and Soft Computing (ICAISC), Zakopane, Poland, 2004, pp. 217-222.
• Mertz C. J., Murphy P.M.: UCI repository of machine learning databases, http://www.ics.uci.edu/~mlearn/MLRepository.html