Deep back propagation–long short-term memory network based upper-limb sEMG signal classification for automated rehabilitation

• Autorzy: Wang Ying , Wu Qun , Dey Nilanjan , Fong Simon , Ashour Amira S.

Identyfikatory

DOI

10.1016/j.bbe.2020.05.003

• Języki publikacji: EN

Abstrakty

EN

Autonomous rehabilitation training for assisted patients with injured upper-limbs promotes the regenerative communication between muscle signals and brain consciousness. Surface electromyographic (sEMG) is a type of electrical signals of neuromuscular activity recorded by electrodes on the surface of the human body, which is widely applied for detecting gestures and stimuli reactions. Experimental results proved the importance of the sEMG signals for extracting such reactions, in which, the segmentation and classification of the sEMG are vital tasks. The objective of the present work is to segment and classify the sEMG signals of patients to assist the design of clinical rehabilitation devices based on the classification of sEMG signals. In the pre-processing stage, a dual-tone multi-frequency signaling is designed for signal coding; subsequently, the pre-processed sEMG signal is transformed by the Fast Fourier Transfer. Afterward, a time-series frequency analysisis performed by applyingHiddenMarkov Models.A basic traditional longshort- term memory (LSTM) model is addressed for waveform-based classification to be compared to the proposed improved deep BP (back-propagation)–LSTM for sEMG signal classification. Seventeen performance features are selected for evaluating the proposed multi-classification, deep learning model for classifying six actions, namely moving gesture of grip, slowly moving, flexor, straight lift, stretch, and up-high lift; which were proposed by rehabilitation physician. The experiment results indicated the superiority of the proposed method compared to other well-known classifiers, such as the neural network, support vector machine, decision trees, Bayes inference, and recurrent neural network. The proposed deep BP–LSTM network achieved 92% accuracy, 89% specificity, 91% precision, and 96% F1-score, in the multi-classification of the sEMG signals.

Słowa kluczowe

EN

electromyographic signal; long short-term memory neural network; back propagation; waveform segmentation; deep learning

PL

sygnał elektromiograficzny; sieć neuronowa; propagacja wsteczna; uczenie głębokie

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2020
• Tom: Vol. 40, no. 3
• Strony: 987--1001
• Opis fizyczny: Bibliogr. 54 poz., rys., tab., wykr.

Twórcy

autor

Wang Ying

• Department of Industrial Design at College of Art and Design, Zhejiang Sci-Tech University, Hangzhou, PR China

autor

Wu Qun

• Institute of Universal Design, Zhejiang Sci-Tech University, Hangzhou, PR China; Collaborative Innovation Center of Culture, Creative Design and Manufacturing Industry of China Academy of Art, Hangzhou, PR China; Zhejiang Provincial Key Laboratory of Integration of Healthy Smart Kitchen System, Hangzhou, PR China

autor

Dey Nilanjan

• Department of Information Technology, Techno International New Town, West Bengal, India

autor

Fong Simon

• Department of Computer and Information Science, University of Macau, Taipa, Macau

autor

Ashour Amira S.

• Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, Tanta, Egypt

Bibliografia

• [1] Qi S, Wu X, Wang J, Zhang J. Recognition of composite motions based on sEMG via deep learning. 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA); 2019. pp. 31–6.
• [2] Guo S, Pang M, Gao B, Hirata H, Ishihara H. Comparison of sEMG-based feature extraction and motion classification methods for upper-limb movement. Sensors (Basel) 2015;15 (4):9022–38.
• [3] Beretta-Piccoli M, Cescon C, Barbero M, D'Antona G. Reliability of surface electromyography in estimating muscle fiber conduction velocity: a systematic review. J Electromyogr Kinesiol 2019;48:53–68.
• [4] Tinkhauser G, Shah SA, Fischer P, Peterman K, Debove I, Nygyuen K, et al. Electrophysiological differences between upper and lower limb movements in the human subthalamic nucleus. Clin Neurophysiol 2019;130(5):727–38.
• [5] Liu H, Tao J, Lyu P, Tian F. Human–robot cooperative control based on sEMG for the upper limb exoskeleton robot. Robot Autonom Syst 2020;125:103350.
• [6] Gámez AB, Hernandez Morante JJ, Martínez Gil JL, et al. The effect of surface electromyography biofeedback on the activity of extensor and dorsiflexor muscles in elderly adults: a randomized trial. Sci Rep 2019;9:13153.
• [7] Feldner HA, Howell D, Kelly VE, McCoy SW, Steele KM. Look, your muscles are firing!: a qualitative study of clinician perspectives on the use of surface electromyography in neurorehabilitation. Arch Phys Med Rehab 2019;100(4):663–75.
• [8] Ng CL, Reaz MBI. Evolution of a capacitive electromyography contactless biosensor: design and modelling techniques. Measurement 2019;145:460–71.
• [9] Simão M, Mendes N, Gibaru O, Neto P. A review on electromyography decoding and pattern recognition for human-machine interaction. IEEE Access 2019;7:39564–82.
• [10] Hazam Majid MS, Khairunizam W, Shahriman AB, Zunaidi I, Sahyudi BN, Zuradzman MR. EMG feature extractions for upper-limb functional movement during rehabilitation. International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS). 2018. pp. 314–20.
• [11] Lascurain-Aguirrebeña I, Newham DJ, Irazusta J, Seco J, Critchley DJ. Reliability of a method to measure neck surface electromyography, kinematics, and pain occurrence in participants with neck pain. J Manipul Physiol Therap 2018;41(5):413–24.
• [12] Sleutjes BT, Drenthen J, Boskovic E, van Schelven LJ, Kovalchuk MO, Lumens PG, et al. Excitability tests using high-density surface-EMG: a novel approach to studying single motor units. Clin Neurophysiol 2018;129(8):1634–41.
• [13] Zhang L, Liu G, Han B, Wang Z, Zhang T. sEMG based human motion intention recognition. J Robot 2019;3679174.
• [14] Gallagher RM, Marquez J, Osmotherly P. Cognitive and upper limb symptom changes from a tap test in idiopathic normal pressure hydrocephalus. Clin Neurol Neurosurg 2018;174:92–6.
• [15] Cai S, Chen Y, Huang S, Wu Y, Zheng H, Li X, et al. SVM-based classification of sEMG signals for upper-limb self-rehabilitation training. Front Neurorobot 2019;13:31.
• [16] Liang S, Wu Y, Chen J, Zhang L. Identification of gesture based on combination of raw sEMG and sEMG envelope using supervised learning and univariate feature selection. J Bionic Eng 2019;16:647–62.
• [17] Côté-Allard U, Campbell E, Phinyomark A, Laviolette F, Gosselin B, Scheme E. Interpreting deep learning features for myoelectric control: a comparison with handcrafted features. Front Bioeng Biotechnol 2020;8:158.
• [18] Bashford J, Wickham A, Iniesta R, Drakakis E, Boutelle M, Mills K, et al. Preprocessing surface EMG data removes voluntary muscle activity and enhances SPiQE fasciculation analysis. Clin Neurophysiol 2020;131(1):265–73.
• [19] Tavakoli M, Benussi C, Lourenco JL. Single channel surface EMG control of advanced prosthetic hands: a simple, low cost and efficient approach. Expert Syst Appl 2017;79:322–32.
• [20] Tavakoli M, Benussi C, Lopes PA, Osorio LB, de Almeida AT. Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier. Biomed Signal Process Contr 2018;46:121–30.
• [21] Amanpreet K. Machine learning-based novel approach to classify the shoulder motion of upper limb amputees. Biocybern Biomed Eng 2019;39(3):857–67.
• [22] Dey N, Ashour AS, Mohamed WS, Nguyen NG. Biomedical signals. Acoustic sensors for biomedical applications. Cham: Springer; 2019. p. 7–20.
• [23] Zhang J, Ling C, Li S. EMG signals based human action recognition via deep belief networks. IFAC-Papers Online 2019;52(19):271–6.
• [24] Jin Y, Qin C, Huang Y, Zhao W, Liu C. Multi-domain modeling of atrial fibrillation detection with twin attentional convolutional long short-term memory neural networks. Knowledge-Based Syst 2020;193:105460.
• [25] Khowailed IA, Abotabl A. Neural muscle activation detection: a deep learning approach using surface electromyography. J Biomech 2019;95:109322.
• [26] Wu Y, Fang Y, Liu C, Fan Z, Wang C. Gated recurrent unit based frequency-dependent hysteresis modeling and end- to-end compensation. Mech Syst Sig Process 2020;136:106501.
• [27] Marrie RA, Cutter GR, Tyry T, Cofield SS, Fox R, Salter A. Upper limb impairment is associated with use of assistive devices and unemployment in multiple sclerosis. Multiple Sclerosis Relat Disord 2017;13:87–92.
• [28] Mukhopadhyay AK, Samui S. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed Sig Process Cont 2020;55:101669.
• [29] Roberts A, Nguyen P, Orange JB, Jog M, Nisbet KA, McRae K. Differential impairments of upper and lower limb movements influence action verb processing in Parkinson disease. Cortex 2017;97:49–59.
• [30] Oliveira AS, Andersen CØ, Grimstrup CB, Pretzmann F, Mortensen NH, Castro MN, et al. A software for testing and training visuo-motor coordination for upper limb control. J Neurosci Methods 2019;324:108310.
• [31] Shao J, Niu Y, Xue C, Wu Q, Zhou X, Xie Y, et al. Single- channel SEMG using wavelet deep belief networks for upper limb motion recognition. Int J Indus Ergonom 2020;76:102905.
• [32] Yang Z, Chen Y, Tang Z, Wang J. Surface EMG based handgrip force predictions using gene expression programming. Neurocomputing 2016;207:568–79.
• [33] Parajuli N, Sreenivasan N, Bifulco P, Cesarelli M, Savino S, Niola V, et al. Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation. Sensors 2019;19 (20):4596.
• [34] Kanitz G, Cipriani C, Edin BB. Classification of transient myoelectric signals for the control of multi-grasp hand prostheses. IEEE Trans Neural Syst Rehab Eng 2018;26 (9):1756–64.
• [35] Bhavanam SN, Siddaiah P, Reddy PR. Area and power optimized DTMF detection by using different FPGA's. Procedia Comp Sci 2016;85:331–44.
• [36] Trinler U, Leboeuf F, Hollands K, Jones R, Baker R. Estimation of muscle activation during different walking speeds with two mathematical approaches compared to surface EMG. Gait Posture 2018;64:266–73.
• [37] Cordoba CG, Daly CJ. The organisation of vascular smooth muscle cells; a quantitative fast Fourier transform (FFT) based assessment. Trans Res Anatomy 2019;16:100047.
• [38] Palaz D, Magimai-Doss M, Collobert R. End-to-end acoustic modeling using convolutional neural networks for HMM-based automatic speech recognition. Speech Commun 2019;108:15–32.
• [39] Smith NJ, Kutas M. Regression-based estimation of ERP waveforms: I. The rERP framework. Psychophysiology 2015;52(2):157–68.
• [40] Larivière C, Gagnon D, Loisel P. An application of pattern recognition for the comparison of trunk muscles EMG waveforms between subjects with and without chronic low back pain during flexion–extension and lateral bending tasks. J Electromyogr Kinesiol 2000;10(4):261–73.
• [41] Cao H, Zhang D. Soft robotic glove with integrated sEMG sensing for disabled people with hand paralysis. IEEE International Conference on Robotics and Biomimetics (ROBIO), December. 2016. pp. 714–8.
• [42] Waris A, Kamavuako EN. Effect of threshold values on the combination of EMG time domain features: surface versus intramuscular EMG. Biomed Sig Process Contr 2018;45:267–73.
• [43] Clancy EA, Morin EL, Merletti R. Sampling, noise-reduction and amplitude estimation issues in surface electromyography. J Electromyogr Kinesiol 2002;12(1):1–16.
• [44] Schmidhuber J, Hochreiter S. Long short-term memory. Neural Comput 1997;9(8):1735–80.
• [45] Hsieh C, Liou D, Lee D. A real time hand gesture recognition system using motion history image. 2nd International Conference on Signal Processing Systems, Dalian; 2010. pp. V2-394–V2-398, doi: 10.1109/ICSPS.2010.5555462.
• [46] Shanthakumar VA, Peng C, Hansberger J, Cao L. Design and evaluation of a hand gesture recognition approach for real-time interactions. Multimed Tools Appl 2020. http://dx.doi.org/10.1007/s11042-019-08520-1.
• [47] Nazmi N, Rahman A, Azizi M, Yamamoto SI, Ahmad SA, Zamzuri H, et al. A review of classification techniques of EMG signals during isotonic and isometric contractions. Sensors 2016;16(8):1304.
• [48] Too J, Abdullah AR, Mohd Saad N, Tee W. EMG feature selection and classification using a Pbest-guide binary particle swarm optimization. Computation 2019;7:12.
• [49] Gokgoz E, Subasi A. Comparison of decision tree algorithms for EMG signal classification using DWT. Biomed Sig Process Cont 2015;18:138–44.
• [50] Falih ADI, Dharma WA, Sumpeno S. Classification of EMG signals from forearm muscles as automatic control using Naive Bayes. International Seminar on Intelligent Technology and its Applications (ISITIA); 2017. pp. 346–51.
• [51] Saikia A, Mazumdar S, Sahai N, et al. Performance analysis of artificial neural network for hand movement detection from EMG signals. IETE J Res 2019;1–10.
• [52] Bhusari A, Gupta N, Kambli T, Kulkami S. Comparison of SVM an/kNN classifiers for palm movements using sEMG signals with different features. 3rd International Conference on Computing Methodologies and Communication (ICCMC); 2019. pp. 881–5.
• [53] Hu Y, Wong Y, Wei W, Du Y, et al. A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition. PLOS ONE 2018;13(10):e0206049.
• [54] Oktay AB, Kocer A. Differential diagnosis of Parkinson and essential tremor with convolutional LSTM networks. Biomed Sig Process Cont 2020;56:101683.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).