Neural network structure optimization algorithm

• Autorzy: Nowakowski G. , Dorogyy Y. , Doroga-Ivaniuk O.

Identyfikatory

DOI

10.14313/JAMRIS_1-2018/1

• Języki publikacji: EN

Abstrakty

EN

This paper presents a deep analysis of literature on the problems of optimization of parameters and structure of the neural networks and the basic disadvantages that are present in the observed algorithms and methods. As a result, there is suggested a new algorithm for neural network structure optimization, which is free of the major shortcomings of other algorithms. The paper describes a detailed description of the algorithm, its implementation and application for recognition problems.

Słowa kluczowe

EN

structure optimization; neural network; ReLU; SGD

• Wydawca: Łukasiewicz Industrial Research Institute for Automation and Measurements PIAP
• Czasopismo: Journal of Automation Mobile Robotics and Intelligent Systems
• Rocznik: 2018
• Tom: Vol. 12, No. 1
• Strony: 5--13
• Opis fizyczny: Bibliogr. 35 poz., rys.

Twórcy

autor

Nowakowski G.

• Cracow University of Technology ul. Warszawska 24, 31-155 Cracow, Poland

autor

Dorogyy Y.

• National Technical University of Ukraine “Igor Sikorsky Kyiv Politechnic Institute” av. Victory 37, Kyiv, Ukraine.

autor

Doroga-Ivaniuk O.

• National Technical University of Ukraine “Igor Sikorsky Kyiv Politechnic Institute” av. Victory 37, Kyiv, Ukraine

Bibliografia

•  [1] G. Nowakowski et al., “The Realisation of Neural Network Structural Optimization Algorithm”, In: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, 2017, 1365–1371. DOI: 10.15439/2017F448.
•  [2] Q. Xiao, W. Shi, X. Xian, X. Yan, “An image restoration method based on genetic algorithm BP neural network”. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, 2008, 7653–7656.
•  [3] W. Wu, W. Guozhi, Z. Yuanmin, W. Hongling, “Genetic Algorithm Optimizing Neural Network for Short-Term Load Forecasting”. In: International Forum on Information Technology and Applications, 2009, 583–585. DOI: 10.1109/IFITA.2009.326.
•  [4] S. Zeng, J. Li, L. Cui, “Cell Status Diagnosis for the Aluminum Production on BP Neural Network with Genetic Algorithm”, Communications in Computer and Information Science, vol. 175, 2011, 146-152. DOI: 10.1007/978-3-642-21783-8_24.
•  [5] W. Yinghua, X. Chang, “Using Genetic Artificial Neural Network to Model Dam Monitoring Data”. In: Second International Conference on Computer Modeling and Simulation, 2010, 3–7. DOI: 10.1109/ICCMS.2010.80.
•  [6] R. Sulej, K. Zaremba, K. Kurek, R. Rondio, Application of the Neural Networks in Events Classification in the Measurement of the Spin Structure of the Deuteron, Warsaw University of Technology, Poland, 2007.
•  [7] S. A. Harp, T. Samad, “Genetic Synthesis of Neural Network Architecture”, Handbook of Genetic Algorithms, 1991, 202–221.
•  [8] D. Whitley, T. Starkweather, C. Bogart, “Genetic Algorithms and Neural Networks: Optimizing Connections and Connectivity”, Parallel Computing, vol. 14, no. 3, 1990, 347–61. DOI: 10.1016/0167-8191(90)90086-O.
•  [9] V. Bevilacqua, G. Mastronardi, F. Menolascina,P. Pannarale, A. Pedone, “A Novel Multi-Objective Genetic Algorithm Approach to Artificial Neural Network Topology Optimisation: The Breast Cancer Classification Problem”, International Joint Conference on Neural Networks, 1958–1965, 2006.
• [10] Y. Du, Y. Li, “Sonar array azimuth control system based on genetic neural network”. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, 2008, 6123–6127.
• [11] S. Nie, B. Ye, “The Application of BP Neural Network Model of DNA-Based Genetic Algorithm to Monitor Cutting Tool Wear”. In: International Conference on Measuring Technology and Mechatronics Automation, 2009, 338–341. DOI: 10.1109/ICMTMA.2009.160.
• [12] C. Tang, Y. He, L. Yuan, “A Fault Diagnosis Method of Switch Current Based on Genetic Algorithm to Optimize the BP Neural Network”. In: International Conference on Electric and Electronics, vol. 99, chapter 122, 2011, 943–950. DOI: 10.1007/978-3-642-21747-0_122.
• [13] Y. Du, Y. Li, “Sonar array azimuth control system based on genetic neural network”. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, 2008, 6123–6127.
• [14] L. Jinru, L. Yibing, Y. Keguo, “Fault diagnosis of piston compressor based on Wavelet Neural Network and Genetic Algorithm”. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, 2008, 6006–6010. DOI: 10.1109/WCICA.2008.4592852.
• [15] D. Dasgupta, D. R. McGregor, “Designing Application-Specific Neural Networks using the Structured Genetic Algorithm”. In: Proceedings of International Workshop on Combinations of Genetic Algorithms and Neural Networks, 1992, 87–96. DOI: 10.1109/COGANN.1992.273946.
• [16] G. G. Yen, H. Lu, “Hierarchical Genetic Algorithm Based Neural Network Design”, IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, 2000, 168–175. DOI:10.1109/ECNN.2000.886232.
• [17] P. Koehn, Combining Genetic Algorithms and Neural Networks: The Encoding Problem, University of Tennessee, Knoxville, 1994.
• [18] Z. Chen, “Optimization of Neural Network Based on Improved Genetic Algorithm”. In: International Conference on Computational Intelligence and Software Engineering, 2009, 1–3. DOI: 10.1109/CISE.2009.5365287.
• [19] P. W. Munro, “Genetic Search for Optimal Representation in Neural Networks”. In: Proceedings of the International Joint Conference on Neural Networks and Genetic Algorithms, chapter 91,1993, 675–682. DOI: 10.1007/978-3-7091-7533-0_91.
• [20] X. Fu, P.E.R. Dale, S. Zhang, “Evolving Neural Network Using Variable String Genetic Algorithms (VGA) for Color Infrared Aerial Image Classification”, Chinese Geographical Science, vol. 18(2), 2008, 162–170.
• [21] J. M. Bishop, M. J. Bushnell, “Genetic Optimization of Neural Network Architectures for Colour Recipe Prediction”. In: Proceedings of the International Joint Conference on Neural Networks and Genetic Algorithms, 719–725, 1993.
• [22] M. Mezard, J.P. Nadal, “Learning in feedforward layered networks: The Tiling algorithm”, Journal of Physics, 1989, V. A22, P. 2191 – 2203.
• [23] M. Frean, “The Upstart Algorithm: A Method for Constructing and Training Feed-Forward Neural Networks”, Tech. Rep. 89/469, Edinburgh University, 1989.
• [24] B. D. Ripley, Pattern recognition and neural networks, Cambridge: Cambridge Univ. Press, 2009.DOI: 10.1017/CBO9780511812651.
• [25]Y. Y. Dorogiy, “Accelerated learning algorithm of Convolutional neural networks”, Visnik NTUU “KPI”, Informatics, operation and computer science, vol. 57, 2012, 150–154.
• [26] Y. Y. Dorohyy, “The algorithm of algorithmic optimization of the structural neural network is based on classification of data”, Visnyk NTUU “KPI”, Informatics, operation and computer science, vol. 62, 2015, 169–173.
• [27] S. D. Halloway, Programming Clojure, The Pragmatic Bookshelf, 2 edition, 2012.
• [28] B. Goetz, Java Concurrency in Practice, AddisonWesley Professional,1 edition, 2006.
• [29] S. Thrun et al., The monk’s problems: A performance comparison of different learning algorithms.Technical Report CMU-CS-91-197, Carnegie Mellon University, 1991.
• [30] P. Sadowski, Notes on backpropagation, homepage: https://www.ics.uci.edu/~pjsadows/notes.pdf (online).
• [31] K. J. Lang, M. Witbrock, Learning to Tell Two Spirals Apart In: Proceedings of 1988 Connectionists Models Summer School. Morgan Kaufmann, San Mateo CA, 1989, 52-59. DOI: 10.13140/2.1.3459.2329.
• [32] Yale Face Database, homepage: http://vision. ucsd.edu/~iskwak/ExtYaleDatabase/Yale/Face/Database.htm (online).
• [33] Y. Bengio, “Practical recommendations for gradient-based training of deep architectures”,arXiv:1206.5533v2, 2012.
• [34] M. Hüsken, Y. Jin, B. Sendhoff, Soft Computing (2005) 9: 21. DOI: 10.1007/s00500-003-0330-y.
• [35] Y. Y. Dorogyy et al.,“Qualitative evaluation method of IT-infrastructure elements functioning”. IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), 170–174, 2014.

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).