Wyniki wyszukiwania - BazTech

1

Artificial Neural Networks vs Long Short-Term Memory Prediction of Solid Flow in Tafna Basin (North-West Algeria)

Medfouni Mohamed Nadjib, Korichi Khaled, Marouf Nadir

Ecological Engineering & Environmental Technology

|

2024

|

Vol. 25, iss. 3

275--286

EN

The main objective of this work is to select the most reliable machine learning model to predict the generated solid flow in the Tafna basin (North-West of Algeria). It is about the artificial neural networks (ANN) and long short-term memory (LSTM). The sediment load is recorded through three hydrometric stations. The efficiency and performance of the two models is verified using the correlation coefficient (R2), the Nash-Sutcliffe coefficient (NSC) and the root mean square error (RMSE). The obtained simulated solids load shows a very good correlation in terms of precision although the ANN model gave relatively better results compared to the LSTM model where low RMSE values were recorded, which confirms that the artificial intelligence models remain also effective for the treatment and the prediction of hydrological phenomena such as the estimation of the solid load in a such watershed.

2

Application of artificial neural networks in the development of the PM10 air pollution prediction system

Wiktorzak Aneta, Sawicki Andrzej

TASK Quarterly : scientific bulletin of Academic Computer Centre in Gdansk

|

2023

|

Vol. 27, No 1

1--9

EN

This article presents research on the model of forecasting the average daily air pollution levels focused mainly on two solutions, artificial neural networks: the NARX model and the LSTM model. The research used an air quality monitoring system. This system includes individually designed and implemented sensors to measure the concentration of pollutants such as PM10, PM2.5, SO2, NO2 and to record weather conditions such as temperature, humidity, pressure, wind strength and speed. Data is sent to a central database server based on the MQTT protocol. Additional weather information in the area covered by pollution monitoring is collected from the weather services of the IMGW and openwethermap.org. The artificial neural network models were built in the MATLAB environment, the process of learning neural networks was performed and the results of pollution prediction for the level of PM10 dust were tested. The models showed good and acceptable results when forecasting the state of PM10 dust concentration in the next 24 hours. The LSTM prediction model were more accurate than the NARX model. The future work will be related to the use of artificial intelligence algorithms to predict the concentration of other harmful substances, e.g. PM2.5, NO2, SO2 etc. A very important task in the future will be to frame the entire system of monitoring and predicting smog in a given area.

3

Correlation between Cone Penetration Test parameters, soil type, and soil liquidity index using long short-term memory neural network

Jocz Mateusz, Lefik Marek

Studia Geotechnica et Mechanica

|

2023

|

Vol. 45, nr s1

405--415

EN

Accuracy and quality of recognizing soil properties are crucial for optimal building design and for ensuring safety in the construction and exploitation stages. This article proposes use of long short-term memory (LSTM) neural network to establish a correlation between Cone Penetration Test (CPTU) results, the soil type, and the soil liquidity index IL. LSTM artificial neural network belongs to the class of networks requiring deep machine learning and is qualitatively different from artificial neural networks of the multilayer perceptron type, which have long been widely used to interpret the results of geotechnical experiments. The article outlines the methodology of CPTU testing and laboratory testing of the liquidity index, as well as construction and preparation of data for the network. The proposed network achieved good results when considering a database consisting of the parameters of eight CPTU soundings, soil stratifications, and laboratory test results.

4

Cuban consumer price index forecasting through transformer with attention

Rosado Reynaldo, Toledano-López Orlando G., González Hector R., Abreu Aldis J., Hernandez Yanio

Journal of Automation Mobile Robotics and Intelligent Systems

|

2023

|

Vol. 17, No. 2

12--17

EN

Recently, time series forecasting modelling in the Con‐ sumer Price Index (CPI) has attracted the attention of the scientific community. Several research projects have tackled the problem of CPI prediction for their countries using statistical learning, machine learning and deep neural networks. The most popular approach to CPI in several countries is the Autoregressive Integrated Mov‐ ing Average (ARIMA) due to the nature of the data. This paper addresses the Cuban CPI forecasting problem using Transformer with attention model over univariate dataset. The fine tuning of the lag parameter shows that Cuban CPI has better performance with small lag and that the best result was in 𝑝 = 1. Finally, the comparative results between ARIMA and our proposal show that the Transformer with attention has a very high performance despite having a small data set.

5

Deep Learning Approach for Runoff Prediction – Evaluating the Long-Short-Term Memory Neural Network Architectures for Capturing Extreme Discharge Events in the Ouergha Basin, Morocco

Karmouda Nourelhouda, Bouramtane Tarik, Tahiri Mounia, Kacimi Ilias, Leblanc Marc, Kassou Nadia

Ecological Engineering & Environmental Technology

|

2023

|

Vol. 24, iss. 9

271--290

EN

Rainfall-runoff modeling plays a crucial role in achieving efficient water resource management and flood forecasting, particularly in the context of increasing intensity and frequency of extreme meteorological events induced by climate change. Therefore, the aim of this research is to assess the accuracy of the Long-Short-Term Memory (LSTM) neural networks and the impact of its architecture in predicting runoff, with a particular focus on capturing extreme hydrological discharges in the Ouergha basin; a Moroccan Mediterranean basin with historical implications in many cases of flooding; using solely daily rainfall and runoff data for training. For this purpose, three LSTM models of different depths were constructed, namely LSTM 1 single-layer, LSTM 2 bi-layer, and LSTM 3 tri-layer, their window size and hyperparameters were first tuned, and on seven years of daily data they were trained, then validated and tested on two separate years to ensure the generalization on unseen data. The performance of the three models was compared using hydrogram-plots, Scatter-plots, Taylor diagrams, and several statistical metrics. The results indicate that the single-layer LSTM 1 outperforms the other models, it consistently achieves higher overall performance on the training, validation, and testing periods with a coefficient of determination R-squared of 0.92, 0.97, and 0.95 respectively; and with Nash-Sutcliffe efficiency metric of 0.91, 0.94 and 0.94 respectively, challenging the conventional beliefs about the direct link between complexity and effectiveness. Furthermore, all the models are capable of capturing the extreme discharges, although, with a moderate und erprediction trend for LSTM 1 and 2 as it does not exceed -25% during the test period. For LSTM 3, even if its underestimation is less pronounced, its increased error rate reduces the confidence in its performance. This study highlights the importance of aligning model complexity with data specifications and suggests the necessity of considering unaccounted factors like upstream dam releases to enhance the efficiency in capturing the peaks of extreme events.

6

Comparative analysis of machine learning algorithms based on an air pollution prediction model

Wiktorzak Aneta, Kaczorowski B.

TASK Quarterly : scientific bulletin of Academic Computer Centre in Gdansk

|

2022

|

Vol. 26, No 4

1--10

EN

In this paper it has been assumed that the use of artificial intelligence algorithms to predict the level of air quality gives good results. Our goal was to perform a comparative analysis of machine learning algorithms based on an air pollution prediction model. By repeatedly performing tests on a number of models, it was possible to establish both the positive and negative influence of the parameters on the result generated by the ANN model. The research was based on some selected both current and historical data of the air pollution concentration altitude and weather data. The research was carried out with the help of the Python 3 programming language, along with the necessary libraries such as TensorFlow and Jupyter Notebook. The analysis of the results showed that the optimal solution was to use the Long Stort Term Memory LSTM algorithm in smog prediction. It is a recursive model of an artificial neural network that is ideally suited for prediction tasks. Further research on the models may develop in various directions, ranging from increasing the number of trials which would be linked to more reliable data, ending with increasing the number of types of algorithms studied. Developing the models by testing other types of activation and optimization functions would also be able to improve the understanding of how they affect the data presented. A very interesting developmental task may be to focus on a self-learning artificial intelligence algorithm, so that the algorithm can learn on a regular basis, not only on historical data. These studies would contribute significantly to the amount of data collected, its analysis and prediction quality in the future.

7

Multimodal Sarcasm Detection via Hybrid Classifier with Optimistic Logic

Bavkar Dnyaneshwar Madhukar, Kashyap Ramgopal, Khairnar Vaishali

Journal of Telecommunications and Information Technology

|

2022

|

nr 3

97--114

EN

This work aims to provide a novel multimodal sarcasm detection model that includes four stages: pre-processing, feature extraction, feature level fusion, and classification. The pre-processing uses multimodal data that includes text, video, and audio. Here, text is pre-processed using tokenization and stemming, video is pre-processed during the face detection phase, and audio is pre-processed using the filtering technique. During the feature extraction stage, such text features as TF-IDF, improved bag of visual words, n-gram, and emojis as well on the video features using improved SLBT, and constraint local model (CLM) are extraction. Similarly the audio features like MFCC, chroma, spectral features, and jitter are extracted. Then, the extracted features are transferred to the feature level fusion stage, wherein an improved multilevel canonical correlation analysis (CCA) fusion technique is performed. The classification is performer using a hybrid classifier (HC), e.g. bidirectional gated recurrent unit (Bi-GRU) and LSTM. The outcomes of Bi-GRU and LSTM are averaged to obtain an effective output. To make the detection results more accurate, the weight of LSTM will be optimally tuned by the proposed opposition learning-based aquila optimization (OLAO) model. The MUStARD dataset is a multimodal video corpus used for automated sarcasm Discovery studies. Finally, the effectiveness of the proposed approach is proved based on various metrics.

8

Autonomous anomaly detection system for crime monitoring and alert generation

Kukad Jyoti, Soner Swapnil, Pandya Sagar

Journal of Automation Mobile Robotics and Intelligent Systems

|

2022

|

Vol. 16, No. 1

62--71

EN

Nowadays, violence has a major impact in society. Violence metrics increasing very rapidly reveal a very alarming situation. Many violent events go unnoticed. Over the last few years, autonomous vehicles have been used to observe and recognize abnormalities in human behavior and to classify them as crimes or not. Detecting crime on live streams requires classifying an event as a crime or not a crime and generating alerts to designated authorities, who can in turn take the required actions and assess the security of the city. There is currently a need for this kind of effective techniques for live video stream processing in computer vision. There are many techniques that can be used, but Long Short-Term Memory (LSTM) networks and OpenCV provide the most accurate prediction for this task. OpenCV is used for the task of object detection in computer vision, which will take the input from either a drone or any autonomous vehicle. LSTM is used to classify any event or behavior as a crime or not. This live stream is also encrypted using the Elliptic curve algorithm for more security of data against any manipulation. Through its ability to sense its surroundings, an autonomous vehicle is able to operate itself and execute critical activities without the need for human interaction. Much crowd-based crimes like mob lynching and individual crimes like murder, burglary, and terrorism can be protected against with advanced deep learning-based Anamoly detection techniques. With this proposed system, object detection is possible with approximately 90% accuracy. After analyzing all the data, it is sent to the nearest concern department to provide the remedial approach or protect from any crime. This system helps to enhance surveillance and decrease the crime rate in society.

9

Roller damage detection method based on the measurement of transverse vibrations of the conveyor belt

Bortnowski Piotr, Król Robert, Ozdoba Maksymilian

Eksploatacja i Niezawodność

|

2022

|

Vol. 24, no. 3

510--521

EN

The article presents the detection of damage to rollers based on the transverse vibration signal measured on the conveyor belt. A solution was proposed for a wireless measuring device that moves with the conveyor belt along of the route, which records the signal of transverse vibrations of the belt. In the first place, the research was conducted in laboratory conditions, where a roller with prepared damage was used. Subsequently, the process of validating the adopted test procedure under real conditions was performed. The approach allowed to verify the correctness of the adopted technical assumptions of the measuring device and to assess the reliability of the acquired test results. In addition, an LSTM neural network algorithm was proposed to automate the process of detecting anomalies of the recorded diagnostic signal based on designated time series. The adopted detection algorithm has proven itself in both laboratory and in-situ tests.

10

Predicting hospital emergency department visits with deep learning approaches

Zhao Xinxing, Lai Joel Weijia, Ho Andrew Fu Wah, Liu Nan, Ong Marcus Eng Hock, Cheong Kang Hao

Biocybernetics and Biomedical Engineering

|

2022

|

Vol. 42, no. 3

1051--1065

EN

Overcrowding in emergency department (ED) causes lengthy waiting times, reduces adequate emergency care and increases rate of mortality. Accurate prediction of daily ED visits and allocating resources in advance is one of the solutions to ED overcrowding problem. In this paper, a deep stacked architecture is being proposed and applied to the daily ED visits prediction problem with deep components such as Long Short Term Memory (LSTM), Gated Recurrent Units (GRU) and simple Recurrent Neural Network (RNN). The proposed architecture achieves very high mean accuracy level (94.28–94.59%) in daily ED visits predictions. We have also compared the performance of this architecture with non-stacked deep models and traditional prediction models. The results indicate that deep stacked models outperform (4–7%) the traditional prediction models and other non-stacked deep learning models (1–2%) in our prediction tasks. The application of deep neural network in ED visits prediction is novel as this is one of the first studies to apply a deep stacked architecture in this field. Importantly, our models have achieved better prediction accuracy (in one case comparable) than the state-of-the-art in the literature.

11

ECG signals-based automated diagnosis of congestive heart failure using Deep CNN and LSTM architecture

Kusuma S., Jothi K. R.

Biocybernetics and Biomedical Engineering

|

2022

|

Vol. 42, no. 1

247--257

EN

In humans, Congestive Heart Failure (CHF) refers to the chronic progressive condition that drastically influences the pumping potentiality of the heart muscle. This CHF has the possibility of increasing health expenditure, morbidity, mortality and minimized quality of life. In this context, Electrocardiogram (ECG) is considered as the simplest and a non-invasive diagnosis method that aids in detecting and demonstrating the realizable changes in CHF. However, diagnosing CHF based on manual exploration of ECG signals is frequently impacted by errors as duration and small amplitude of the signals either investigated separately or in the integration is determined to neither specific nor sensitive. At this juncture, the reliability and diagnostic objectivity of ECG signals during the CHF detection process may be enhanced through the inclusion of automated computer-aided system. In this paper, Deep CNN and LSTM Architecture (DCNN-LSTM)-based automated diagnosis system is proposed for detecting CHF using ECG signals. In specific, CNN is included for the purpose of extracting deep features and LSTM is used for attaining the objective of CHF detection using the extracted features. This proposed DCNN-LSTM is evolved with minimal pre-processing of ECG signals and does not involve any classification process or manual engineered features during diagnosis. The experimentation of the proposed DCNN-LSTM conducted using the real time ECG signals datasets confirmed an accuracy of 99.52, sensitivity of 99.31%, specificity of 99.28%, F-Score of 98.94% and AUC of 99.9%, respectively.

12

Capsule network assisted electrocardiogram classification model for smart healthcare

Jiao Ying, Qi Huamei, Wu Jia

Biocybernetics and Biomedical Engineering

|

2022

|

Vol. 42, no. 2

543--555

EN

Improving the classification accuracy of electrocardiogram (ECG) signals is of great significance for diagnosing heart abnormalities and arrhythmias and preventing cardiovascular diseases (CVDs). The traditional classification method depends on medical experience to select and extract features artificially, lacks the generalization ability to deal with big medical data. The vital feature extraction ability of neural networks has become a hot topic to solve this problem. Based on this, the capsule network is applied to ECG signal classification in this paper. Based on the original network architecture, long short-term memory (LSTM) network and 1D convolutional neural network (CNN) are added as a parallel feature extraction layer to extract the spatial and temporal features of the ECG signal. In addition, the enhanced routing algorithm is proposed, which uses the prior probability of subcapsules as a weighting factor for routing algorithm classification to weaken the influence of noise capsules. The proposed model is superior to the existing state-of-the-art techniques when tested on the MIT-BIH arrhythmia database.

13

A smart IoT-enabled heart disease monitoring system using meta-heuristic-based Fuzzy-LSTM model

Krishna Munagala N.V.L.M., Langoju Lakshmi Rajeswara Rao, Rani A. Daisy, Reddy D.V. Rama Koti

Biocybernetics and Biomedical Engineering

|

2022

|

Vol. 42, no. 4

1183--1204

EN

A continuous heart disease monitoring system is one of the significant applications specified by the Internet of Things (IoT). This goal might be achieved by combining sophisticated expert systems with extensive healthcare data on heart diseases. Several machine learning-based methods have recently been proven for predicting and diagnosing cardiac illness. However, these algorithms are unable to manage high-dimensional information due to the lack of a smart framework that can combine several sources to anticipate cardiac illness. The Fuzzy-Long Short Term Memory (LSTM) model is used in this work to present a unique IoT-enabled heart disease prediction method. The benchmark data for the experiment came from public sources and collected via wearable IoT devices. An improved Harris Hawks Optimization (HHO) called Population and Fitness-based HHO (PF-HHO) is utilized to select the best features, with the objective function of correlation maximization within the same class and correlation minimization among different classes. The scientific contributions of the health care monitoring system are depicted here that help to improve heart disease healthcare efficiency and also it can be reducing the death rate in the current world. The important section of this persistent healthcare mode is the real-world monitoring system. The simulation outcomes proved that the recommended approach is more successful at predicting heart illness than existing technologies.

14

Web intrusion detection using character level machine learning approaches with upsampled data

Sivri Talya Tümer, Akman Nergis Pervan, Berkol Ali, Peker Can

Annals of Computer Science and Information Systems

|

2022

|

Vol. 32

269--274

EN

Today, people fulfill their needs in many areas such as shopping, health, and finance online. Besides many well-meaning people who use websites for their own needs, there are also people who send attack requests to get these people's personal data, get website owners' information, and damage the application. The attack types such as SQL injection and XSS can seriously harm web applications and users. Detecting these cyber-attacks manually is very time-consuming and difficult to adapt to new attack types. Our proposed study performs attack detection using different machine learning and deep learning approaches with a larger dataset obtained by combining CSIC 2012 and ECML/PKDD datasets. In this study, we evaluated our classification results which experimented with different algorithms based on computation time and accuracy. In addition to applying different algorithms, experiments on various learning models were applied with our data upsample method for balancing the dataset labels. As a result of the binary classification, LSTM achieves the best result in terms of accuracy, and a positive effect of the upsampled data on accuracy has been observed. LightGBM was the algorithm with the highest performance in terms of computation time.

15

Performance analysis of LSTM model with multi-step ahead strategies for a short-term traffic flow prediction

Doğan Erdem

Zeszyty Naukowe. Transport / Politechnika Śląska

|

2021

|

z. 111

15--31

EN

In this study, the effect of direct and recursive multi-step forecasting strategies on the short-term traffic flow forecast performance of the Long Short-Term Memory (LSTM) model is investigated. To increase the reliability of the results, analyses are carried out with various traffic flow data sets. In addition, databases are clustered using the k-means++ algorithm to reduce the number of experiments. Analyses are performed for different time periods. Thus, the contribution of strategies to LSTM was examined in detail. The results of the recursive based strategy performances are not satisfactory. However, different versions of the direct strategy performed better at different time periods. This research makes an important contribution to clarifying the compatibility of LSTM and forecasting strategies. Thus, more efficient traffic flow prediction models will be developed and systems such as Intelligent Transportation System (ITS) will work more efficiently. A practical implication for researchers that forecasting strategies should be selected based on time periods.

16

Application of long short term memory neural networks for GPS Satellite clock bias prediction

Gnyś Piotr, Przestrzelski Paweł

TASK Quarterly : scientific bulletin of Academic Computer Centre in Gdansk

|

2021

|

Vol. 25, No 4

381--395

EN

Satellite-based localization systems like GPS or Galileo are one of the most com-monly used tools in outdoor navigation. While for most applications, like car navigation orhiking, the level of precision provided by commercial solutions is satisfactory it is not alwaysthe case for mobile robots. In the case of long-time autonomy and robots that operate in re-mote areas battery usage and access to synchronization data becomes a problem. In this paper,a solution providing a real-time onboard clock synchronization is presented. Results achievedare better than the current state-of-the-art solution in real-time clock bias prediction for mostsatellites.

17

Methods of process mining and prediction using deep learning

Cieplak Tomasz, Rymarczyk Tomasz, Kłosowski Grzegorz, Maj Michał, Pliszczuk Damian, Rymarczyk Paweł

Przegląd Elektrotechniczny

|

2021

|

R. 97, nr 3

146--149

EN

The first part of the article presents analytical methods to understand how processes (security or business) occur and function over time. The second part presents the concept of a predictive system using deep learning methods that would enable the prediction of subsequent operations or steps that are part of the process under consideration. The article was supplemented with a review of scientific publications related to the content and theoretical foundations were provided. The research was of an applied nature, therefore the considerations are based on the example of analysis and forecasts based on historical data contained in process logs.

PL

Pierwsza część artykułu przedstawia metody analityczne pozwalające zrozumieć, w jaki sposób procesy (dotyczące bezpieczeństwa lub biznesu) zachodzą i funkcjonują w czasie. W drugiej części przedstawiono koncepcję systemu predykcyjnego wykorzystującego metody głębokiego uczenia, które umożliwiałyby przewidywanie kolejnych operacji lub kroków wchodzących w skład rozważanego procesu. Uzupełnieniem artykułu był przegląd publikacji naukowych pod kątem merytorycznym oraz podano podstawy teoretyczne. Badania miały charakter aplikacyjny, dlatego rozważania opierają się na przykładzie analiz i prognoz opartych na danych historycznych zawartych w logach procesów.

18

A complete system for an automated ECG diagnosis

Wójcik Dariusz, Woś Michał, Kłosowski Grzegorz, Rymarczyk Tomasz

Przegląd Elektrotechniczny

|

2021

|

R. 97, nr 1

162--165

EN

We present a very simple LSTM neural network capable of categorizing heart diseases from the ECG signal. With the use of the ECG simulator we ware able to obtain a large data-set of ECG signal for different diseases that was used for neural network training and validation.

PL

W artykule prezentujemy bardzo prostą sieć LSTM zdolną do rozpoznawania jednostek chorobowych przy chorobach serca. Dodatkowo pokazujemy w jaki sposób stworzyliśmy bazę danych sygnałów pomiarowych użytych do nauki i walidacji sieci neuronowej przy użyciu symulatora EKG.

19

Time series forecasting using the LSTM network

Pęczek Mateusz, Litawa Grzegorz

Journal of Engineering, Energy and Informatics

|

2021

|

No. 1

116--120

EN

Predicting time series is currently one of the problems that can be solved through the use of machine learning methods. A time series is a set of data points in which the sequence is measured at equal time intervals. Predicting the value of the time series can influence your decisions or help you achieve better results. Stock quotes are an example of a time series - the purpose of the created model is attempt to predict their value. One solution to the problem of predicting the results of the time series is the LSTM network. The network contains layered LSTM cells that have the ability to use previously observed relationships in the data set. The number of LSTM layers and cells in each layer is dependent on the designer and is selected based on expert knowledge. The results obtained from the model may seem correct and close to the real ones. Regardless of what values we get and how high the accuracy of the model will be, it should be remembered that stock prices are influenced by parameters and events that cannot be predicted. The predicted values obtained from the model should be treated as a guide or reference information. Stock quotes may change under the influence of geopolitical situations, company involvement, armed conflict or other random and unpredictable phenomenon, therefore, when making decisions, the results of the model should not be taken for granted.

PL

Przewidywanie szeregów czasowych jest obecnie jednym z problemów, które mogą zostać rozwiązane poprzez zastosowanie metod uczenia maszynowego. Szeregiem czasowym nazwiemy zbiór danych, w których pomiar odbywał się w jednakowych odstępach czasu. Przewidywanie wartości szeregu czasowego może wpłynąć na podejmowane decyzje lub pomóc w osiąganiu lepszych wyników. Przykładem szeregu czasowego są notowania giełdowe - celem utworzonego modelu jest próba przewidywania ich wartości. Jednym z rozwiązań problemu przewidywania wyników szeregów czasowych jest sieć LSTM. Sieć zawiera warstwowo ułożone komórki LSTM, które mają zdolność do wykorzystywania wcześniej zaobserwowanych zależności występujących w zbiorze danych. Liczba warstw i komórek LSTM w każdej warstwie jest zależna od projektanta i dobiera się ją w oparciu o wiedzę ekspercką. Wyniki otrzymane z modelu mogą wydawać się poprawne i zbliżone do rzeczywistych. Niezależnie od tego, jakie wartości otrzymamy i jak duża będzie dokładność modelu, należy pamiętać, że na notowania giełdowe wpływ mają parametry i zdarzenia, których nie da się przewidzieć. Wartości przewidywane, otrzymane z modelu, należy traktować jako pomoc lub informacje poglądowe. Notowania giełdowe mogą zmieniać się pod wpływem sytuacji geopolitycznej, upadku firmy, konfliktu zbrojnego lub innego losowego i niemożliwego do przewidzenia zjawiska, dlatego przy podejmowaniu decyzji nie należy traktować wyników modelu jako pewne.

20

An optimized parallel implementation of non-iteratively trained recurrent neural networks

El Zini Julia, Rizk Yara, Awad Mariette

Journal of Artificial Intelligence and Soft Computing Research

|

2021

|

Vol. 11, No. 1

33--50

EN

Recurrent neural networks (RNN) have been successfully applied to various sequential decision-making tasks, natural language processing applications, and time-series predictions. Such networks are usually trained through back-propagation through time (BPTT) which is prohibitively expensive, especially when the length of the time dependencies and the number of hidden neurons increase. To reduce the training time, extreme learning machines (ELMs) have been recently applied to RNN training, reaching a 99% speedup on some applications. Due to its non-iterative nature, ELM training, when parallelized, has the potential to reach higher speedups than BPTT. In this work, we present Opt-PR-ELM, an optimized parallel RNN training algorithm based on ELM that takes advantage of the GPU shared memory and of parallel QR factorization algorithms to efficiently reach optimal solutions. The theoretical analysis of the proposed algorithm is presented on six RNN architectures, including LSTM and GRU, and its performance is empirically tested on ten time-series prediction applications. Opt- PR-ELM is shown to reach up to 461 times speedup over its sequential counterpart and to require up to 20x less time to train than parallel BPTT. Such high speedups over new generation CPUs are extremely crucial in real-time applications and IoT environments.