Wyniki wyszukiwania - BazTech

1

Fast computational approach to the Levenberg-Marquardt algorithm for training feedforward neural networks

Bilski Jarosław, Smoląg Jacek, Kowalczyk Bartosz, Grzanek Konrad, Izonin Ivan

Journal of Artificial Intelligence and Soft Computing Research

|

2023

|

Vol. 13, No. 2

45--61

EN

This paper presents a parallel approach to the Levenberg-Marquardt algorithm (LM). The use of the Levenberg-Marquardt algorithm to train neural networks is associated with significant computational complexity, and thus computation time. As a result, when the neural network has a big number of weights, the algorithm becomes practically ineffective. This article presents a new parallel approach to the computations in Levenberg-Marquardt neural network learning algorithm. The proposed solution is based on vector instructions to effectively reduce the high computational time of this algorithm. The new approach was tested on several examples involving the problems of classification and function approximation, and next it was compared with a classical computational method. The article presents in detail the idea of parallel neural network computations and shows the obtained acceleration for different problems.

2

Towards a very fast feedforward multilayer neural networks training algorithm

Bilski Jarosław, Kowalczyk Bartosz, Kisiel-Dorohinicki Marek, Siwocha Agnieszka, Żurada Jacek

Journal of Artificial Intelligence and Soft Computing Research

|

2022

|

Vol. 12, No. 3

181--195

EN

This paper presents a novel fast algorithm for feedforward neural networks training. It is based on the Recursive Least Squares (RLS) method commonly used for designing adaptive filters. Besides, it utilizes two techniques of linear algebra, namely the orthogonal transformation method, called the Givens Rotations (GR), and the QR decomposition, creating the GQR (symbolically we write GR + QR = GQR) procedure for solving the normal equations in the weight update process. In this paper, a novel approach to the GQR algorithm is presented. The main idea revolves around reducing the computational cost of a single rotation by eliminating the square root calculation and reducing the number of multiplications. The proposed modification is based on the scaled version of the Givens rotations, denoted as SGQR. This modification is expected to bring a significant training time reduction comparing to the classic GQR algorithm. The paper begins with the introduction and the classic Givens rotation description. Then, the scaled rotation and its usage in the QR decomposition is discussed. The main section of the article presents the neural network training algorithm which utilizes scaled Givens rotations and QR decomposition in the weight update process. Next, the experiment results of the proposed algorithm are presented and discussed. The experiment utilizes several benchmarks combined with neural networks of various topologies. It is shown that the proposed algorithm outperforms several other commonly used methods, including well known Adam optimizer.

3

A novel fast feedforward neural networks training algorithm

Bilski Jarosław, Kowalczyk Bartosz, Marjański Andrzej, Gandor Michał, Zurada Jacek

Journal of Artificial Intelligence and Soft Computing Research

|

2021

|

Vol. 11, No. 4

287--306

EN

In this paper1 a new neural networks training algorithm is presented. The algorithm originates from the Recursive Least Squares (RLS) method commonly used in adaptive filtering. It uses the QR decomposition in conjunction with the Givens rotations for solving a normal equation - resulting from minimization of the loss function. An important parameter in neural networks is training time. Many commonly used algorithms require a big number of iterations in order to achieve a satisfactory outcome while other algorithms are effective only for small neural networks. The proposed solution is characterized by a very short convergence time compared to the well-known backpropagation method and its variants. The paper contains a complete mathematical derivation of the proposed algorithm. There are presented extensive simulation results using various benchmarks including function approximation, classification, encoder, and parity problems. Obtained results show the advantages of the featured algorithm which outperforms commonly used recent state-of-the-art neural networks training algorithms, including the Adam optimizer and the Nesterov’s accelerated gradient.

4

SmartX - platforma do zarządzania urządzeniami IoT

Kowalczyk Bartosz, Szeląg Piotr

Rynek Energii

|

2020

|

Nr 2

54--60

PL

W pracy zaprezentowano platformę SmartX umożliwiającą zarządzanie urządzeniami IoT. Przedstawiono skrócony opis współczesnych systemów automatyki domowej oraz Internetu Rzeczy (ang. Internet of Things - IoT). Opisano funkcjonalność platformy SmartX w odniesieniu do architektury, a w szczególności funkcje dotyczące konektorów platformy z urządzeniami i serwisami zewnętrznymi (ang. bindings). Dodatkowo, w pracy opisano zastosowania platformy SmartX na przykładzie serwisu pogodowego Open Weather Map oraz monitora zużycia energii elektrycznej.

EN

At the present times we are observing the global transformation of human mankind. It is happening due to the immense growth of number of devices used to support, guide or monitor everyday human activities and their environment. Such devices are still evolving in order to communicate between each other and the user itself. The best examples are the areas of consumer electronics and domestic appliances. The data processed by those devices are driving the segments of data gathering and analysis for further improvements in scope of usability. The given paper presents the SmartX platform which provides the unified solution for household devices management. The main architectural demand is to gather, accumulate and process data directly from the devices and external data sources such as weather forecasts, etc. The proposed system is flexible enough to work with devices of various manufacturers and what is most important with multiple nodes (locations such as private house, office, etc.) and the devices that are connected to them from a single UI panel. It also gives a user an opportunity to build cross-nodes rules for automating interactions between devices based on various triggers and conditions. Additionally, due to the integration of all devices in the respective node the SmartX platform offers tools to analyse energy usage profile. In a broad scale it might be used by the media providers as a tool for very detailed manipulations of medias demand. Also, the shortened descriptions of the modern household automation systems and Internet of Things (IoT) devices are provided. The functionality of the SmartX platform is discussed in scope of architecture, especially in terms of so called bindings which acts as a data flow controllers between services/devices and the users. The paper is concluded with example usage of the Open Weather Map binding, the well-known internet service which provides the weather data for many regions and the locally consumed energy monitor.

5

Zastosowanie platformy SmartX do akwizycji danych pomiarowych i zarządzania zużyciem energii elektrycznej

Kowalczyk Bartosz, Szeląg Piotr

Pomiary Automatyka Robotyka

|

2020

|

R. 24, nr 3

51--60

PL

W pracy zaprezentowano platformę SmartX umożliwiającą akwizycję danych pomiarowych pochodzących z inteligentnych liczników energii elektrycznej. Platforma pozwala na integrację urządzeń IoT różnych producentów. Dzięki implementacji dużej liczby protokołów komunikacyjnych możliwe jest stworzenie inteligentnego systemu automatyki domowej. W artykule przedstawiono system pomiarowy składający się z platformy wraz z inteligentnymi licznikami energii. Stworzony system został wykorzystany do akwizycji danych reprezentujących zużycie energii elektrycznej dla wybranych obwodów elektrycznych w pomieszczeniach biurowych. W pracy zaprezentowano wyniki analizy danych pomiarowych rejestrowanych w półrocznym okresie rozliczeniowym. Na podstawie wyników badań sformułowano wnioski pozwalające na optymalizację profilu zużycia energii i zmniejszenie kosztów po stronie końcowego odbiorcy energii elektrycznej.

EN

From the electricity consumer point of view, its usage cost reduction is very important. To achieve that the dedicated software systems are required. They are capable of acquiring data directly from smart energy meters and deliver rule engines and solutions for maintaining household devices. In the paper, the SmartX platform is presented. It is capable of acquisition of data directly from the smart energy meters. The platform integrates the IoT (Internet of Things) devices of various manufacturers. This is possible due to its flexible architecture which supports many communication protocols and can be easily extended by so-called protocol bindings. Thanks to that a generic home automation system can be created. In this paper, the dedicated measurement system based on the SmartX platform and Sonoff energy meters are presented. The created system has been used for energy data acquisition from a small office. The measurements took half of the year and the most interesting data is shown in the results section of the paper. The data analysis has been held with the Python programming language. Based on the achieved results several conclusions have been made. Based on them the usage of the SmartX platform and the dedicated measurement system leads to the optimization of the energy consumption and the cost reduction by the energy end-user.

6

Local Levenberg-Marquardt algorithm for learning feedforwad neural networks

Bilski Jarosław, Kowalczyk Bartosz, Marchlewska Alina, Zurada Jacek M.

Journal of Artificial Intelligence and Soft Computing Research

|

2020

|

Vol. 10, No. 4

299--316

EN

This paper presents a local modification of the Levenberg-Marquardt algorithm (LM). First, the mathematical basics of the classic LM method are shown. The classic LM algorithm is very efficient for learning small neural networks. For bigger neural networks, whose computational complexity grows significantly, it makes this method practically inefficient. In order to overcome this limitation, local modification of the LM is introduced in this paper. The main goal of this paper is to develop a more complexity efficient modification of the LM method by using a local computation. The introduced modification has been tested on the following benchmarks: the function approximation and classification problems. The obtained results have been compared to the classic LM method performance. The paper shows that the local modification of the LM method significantly improves the algorithm’s performance for bigger networks. Several possible proposals for future works are suggested.