Deep learning and machine learning techniques to improve hand movement classification in myoelectric control system

• Autorzy: Rajapriya R. , Rajeswari K. , Thiruvengadam S. J.

Identyfikatory

DOI

10.1016/j.bbe.2021.03.006

• Języki publikacji: EN

Abstrakty

EN

A robust myoelectric control system (MCS) is essential for the design of electromyography (EMG) based human–machine interface (HMI) designs such as prosthetics, exoskeleton, wheelchair and humanoid robots. The functionality of the current pattern recognition (PR) technique in MCS is limited by factors such as variation in the user’s limb position. To overcome the effect of this dynamic variation, an invariant higher order statistics – frequency domain feature set (HOS-FD) is proposed in this paper. EMG data from eight hand movements in five limb positions are considered. When trained with three limb position data, the HOS-FD with three hidden layers deep neural network (DNN) achieved a significantly high classification accuracy of 97.84%±0.22 compared to other classifiers viz., single layer artificial neural network (ANN), linear discriminant analysis (LDA), support vector machine (SVM), k nearest neighbor (kNN), decision tree (DT) and Naive Bayes (NB) classifiers with accuracies of 94.11%±1.63, 95.02%±1.89, 94.63%±2.33, 90.05 ± 4.11, 86.66 ± 4.72 and 78.78%±5.02 respectively. Further, when trained with data from all five limb positions, the proposed feature set with DNN had an accuracy of 99.16%±0.14. The statistical significance of the high classification accuracy obtained using the proposed feature set is also proven using multiple analysis of variance tests (p < 0.001). These results indicate that the proposed method is a promising technique for HMI.

Słowa kluczowe

EN

human machine interface; biomedical signal processing; electromyography; machine learning; myoelectric control system

PL

interfejs człowiek-maszyna; przetwarzanie sygnałów biomedycznych; elektromiografia; uczenie maszynowe

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2021
• Tom: Vol. 41, no. 2
• Strony: 554--571
• Opis fizyczny: Bibliogr. 59 poz., rys., tab., wykr.

Twórcy

autor

Rajapriya R.

• Department of Electronics and Communication, Thiagarajar College of Engineering, Madurai, India

autor

Rajeswari K.

• Department of Electronics and Communication, Thiagarajar College of Engineering, Madurai, India

autor

Thiruvengadam S. J.

• Department of Electronics and Communication, Thiagarajar College of Engineering, Madurai, India

Bibliografia

• [1] Zhang J, Wang B, Zhang C, Xiao Y, Wang MY. An EEG/EMG/ EOG-based multimodal human-machine interface to realtime control of a soft robot hand. Frontiers in Neurorobotics 2019;13:00007.
• [2] Hakonen M, Piitulainen H, Visala A. Current state of digital signal processing in myoelectric interfaces and related applications. Biomed Signal Process and Control 2015;18:334–59.
• [3] Kuiken TA, Miller LA, Turner K, Hargrove LJ. A comparison of pattern recognition control and direct control of a multiple degree-of-freedom transradial prosthesis. IEEE J Transl Eng Heal Med 2016;4:1–8.
• [4] Shi WT, Lyu ZJ, Tang ST, Chia TL, Yang CY. A bionic hand controlled by hand gesture recognition based on surface EMG signals: a preliminary study. Biocybern Biomed Eng 2018;38 (1):126–35.
• [5] Benatti S, Milosevic B, Farella E, Gruppioni E, Benini L. A prosthetic hand body area controller based on efficient pattern recognition control strategies. Sensors 2017;17(4):869.
• [6] Amanpreet K. Machine learning-based novel approach to classify the shoulder motion of upper limb amputees. Biocybern Biomed Eng 2019;39(3):857–67.
• [7] Hudgins B, Parker P, Scott RN. A new strategy for multifunction myoelectric control. IEEE Trans Biomed Eng 1993;40(1):82–94.
• [8] Karabulut D, Ortes F, Arslan YZ, Adli MA. Comparative evaluation of EMG signal features for myoelectric controlled human arm prosthetics. Biocybern Biomed Eng 2017;37 (2):326–35.
• [9] Palkowski A, Redlarski G. Basic hand gestures classification based on surface electromyography. Comput Math Methods Med 2016;2016:1–7.
• [10] Phinyomark A, Scheme E. EMG pattern recognition in the era of big data and deep learning. Big Data Cogn Comput 2018; 2(3):21.
• [11] Leone F, Gentile C, Ciancio AL, Gruppioni E, Davalli A, Sacchetti R, et al. Simultaneous sEMg classification of hand/ wrist gestures and forces. Front. Neurorobot 2019;13:42.
• [12] Pancholi S, Joshi AM. Time derivative moments based feature extraction approach for recognition of upper limb motions using EMG. IEEE Sensors Lett 2019;2(3):1–4.
• [13] Jaramillo-Yánez A, Benalcázar ME, Mena-Maldonado E. Realtime hand gesture recognition using surface electromyography and machine learning: a systematic literature review. Sensors 2020;20(9):2464.
• [14] Zhang Z, Yang K, Qian J, Zhang L. Real-time surface EMG pattern recognition for hand gestures based on an artificial neural network. Sensors 2019;19(14):3170.
• [15] Phukpattaranont P, Thongpanja S, Anam K, Al-Jumaily A, Limsakul C. Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal. Med Biol Eng Compu 2018;56 (12):2259–71.
• [16] Wang Y, Wu Q, Dey N, Fong S, Ashour AS. Deep back propagation–long short-term memory network based upperlimb sEMG signal classification for automated rehabilitation. Biocybern Biomed Eng 2020;40(3):987–1001.
• [17] Shao J, Niu Y, Xue C, Wu Q, Zhou X, Xie Yi, et al. Singlechannel sEMG using wavelet deep belief networks for upper limb motion recognition. Int J Ind Ergon 2020;76.
• [18] Chen J, Bi S, Zhang G, Cao G. High-density surface EMG-based gesture recognition using a 3D convolutional neural network. Sensors 2020;20.
• [19] Jia G, Lam HK, Liao J, Wang R. Classification of electromyographic hand gesture signals using machine learning techniques. Neurocomputing 2020;401:236–48.
• [20] Campanini I, Disselhorst-Klug C, Rymer WZ, Merletti R. Surface EMG in clinical assessment and neurorehabilitation: barriers limiting its use. Front Neurol 2020;11:934.
• [21] Chadwell A, Kenney L, Thies S, Galpin A, Head J. The reality of myoelectric prostheses: Understanding what makes these devices difficult for some users to control. Front Neurorobot 2016;10:7.
• [22] Samuel OW, Asogbon MG, Geng Y, Al-Timemy AH, Pirbhulal S, Ji N, et al. Intelligent EMG pattern recognition control method for upper-limb multifunctional prostheses: advances, current challenges, and future prospects. IEEE Access 2019;7:10150–65.
• [23] Masters MR, Smith RJ, Soares AB, Thakor NV. Towards better understanding and reducing the effect of limb position on myoelectric upper-limb prostheses. In: Proc. IEEE Eng. Med. Biol. Soc. 2014, 2577-2580.
• [24] Parajuli N, Sreenivasan N, Bifulco P, Cesarelli M, Savino S, Niola V, et al. Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation. Sensors 2019;19 (20):4596.
• [25] Kupa EJ, Roy SH, Kandarian SC, De Luca CJ. Effects of muscle fiber type and size on EMG median frequency and conduction velocity. J Appl Physiol 1995;79(1):23–32.
• [26] Kyranou I, Vijayakumar S, Erden MS. Causes of performance degradation in non-invasive electromyographic pattern recognition in upper limb prostheses. Front. Neurorobot 2018;12:58.
• [27] Beaulieu RJ, Masters MR, Betthauser J, Smith RJ, Kaliki R, Thakor NV, et al. Multi-position training improves robustness of pattern recognition and reduces limb-position effect in prosthetic control. J Prosthetics Orthot 2017;29(2):54–62.
• [28] Jiang N, Muceli S, Graimann B, Farina D. Effect of arm position on the prediction of kinematics from EMG in amputees. Med Biol Eng Compu 2013;51(1-2):143–51.
• [29] Fougner A, Scheme E, Chan ADC, Englehart K, Stavdahl Ø. Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng 2011;19 (6):644–51.
• [30] Geng Y, Zhou P, Li G. Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees. J NeuroEng Rehabil 2012;9(1):74.
• [31] Betthauser JL, Hunt CL, Osborn LE, Masters MR, Levay G, Kaliki RR, et al. Limb position tolerant pattern recognition for myoelectric prosthesis control with adaptive sparse representations from extreme learning. IEEE Trans Biomed Eng 2018;65(4):770–8.
• [32] Khushaba RN, Takruri M, Miro JV, Kodagoda S. Towards limb position invariant myoelectric pattern recognition using time-dependent spectral features. Neural Networks 2014;55:42–58.
• [33] Mukhopadhyay AK, Samui S. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed Signal Process and Control 2020;55:101669.
• [34] Rampichini S, Vieira TM, Castiglioni P, Merati G. Complexity analysis of surface electromyography for assessing the myoelectric manifestation of muscle fatigue: a review. Entropy 2020;22(5):529.
• [35] Chen CC, Kilner JM, Friston KJ, Kiebel SJ, Jolly RK, Ward NS. Nonlinear coupling in the human motor system. J Neurosci 2010;30(25):8393–9.
• [36] Sezgin N. Analysis of EMG signals in aggressive and normal activities by using higher-order spectra. Sci World J 2012;7.
• [37] Kaplanis PA, Pattichis CS, Hadjileontiadis LJ, Panas SM. Bispectral analysis of surface EMG. In: Proc. Mediterr Electrotech Conf 2000:770–3.
• [38] Chen X, Zhu X, Zhang D. A discriminant bispectrum feature for surface electromyogram signal classification. Med Eng Phys 2010;32(2):126–35.
• [39] Kaplanis PA, Pattichis CS, Hadjileontiadis LJ, Roberts VC. Surface EMG analysis on normal subjects based on isometric voluntary contraction. J Electromyogr Kinesiol 2009;19 (1):157–71.
• [40] Yang Y, Solis-Escalante T, van de Ruit M, van der Helm FCT, Schouten AC. Nonlinear coupling between cortical oscillations and muscle activity during isotonic wrist flexion. Front Comput Neurosci 2016;10.
• [41] Macisaac DT, Parker PA, Scott RN, Englehart KB, Duffley C. Influences of dynamic factors on myoelectric parameters considering the effects of joint angle and muscle force when estimating mean frequency and conduction velocity. IEEE Eng Med Biol Mag 2001;20(6):82–9.
• [42] Sadler BM, Giannakis GB, Lii KS. Estimation and detection in non-Gaussian noise using higher order statistics. IEEE Trans Signal Process 1994;42(10):2729–41.
• [43] Sadler BM, Giannakis GB. Shift and rotation invariant object reconstruction using the bispectrum. J Optical Soc Am A 1992;9(1):57–69.
• [44] Yellott JI, Iverson GJ. Uniqueness properties of higher-order autocorrelation functions. J Opt Soc Am A 1992;9(3):388–404.
• [45] Khushaba RN. Towards limb position invariant myoelectric pattern recognition data 2017. https://www.ramikhushaba.com/electromyogram-emg-repository.html.
• [46] He J, Zhang D, Jiang N, Sheng X, Farina D, Zhu X. User adaptation in long-term, open-loop myoelectric training: Implications for EMG pattern recognition in prosthesis control. J Neural Eng 2015;12(4).
• [47] Zhou X, Qi W, Ovur SE, Zhang L, Hu Y, Su H, et al. A novel muscle-computer interface for hand gesture recognition using depth vision. J Ambient Intell Humaniz Comput 2020;11(11):5569–80.
• [48] Su H, Ovur SE, Zhou X, Qi W, Ferrigno G, De Momi E. Depth vision guided hand gesture recognition using electromyographic signals. Adv Robot 2020;34(15):985–97.
• [49] Rabin N, Kahlon M, Malayev S, Ratnovsky A. Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques. Expert Syst Appl 2020;149.
• [50] Chen C, Yu Y, Ma S, Sheng X, Lin C, Farina D, et al. Hand gesture recognition based on motor unit spike trains decoded from high-density electromyography. Biomed Signal Process Control 2020;55.
• [51] Vieira TM, Botter A, Muceli S, Farina D. Specificity of surface EMG recordings for gastrocnemius during upright standing. Sci Rep 2017;7(1).
• [52] Vieira TM, Bisi MC, Stagni R, Botter A. Changes in tibialis anterior architecture affect the amplitude of surface electromyograms. J NeuroEng Rehabil 2017;14(1).
• [53] Del Vecchio A, Bazzucchi I, Felici F. Variability of estimates of muscle fiber conduction velocity and surface EMG amplitude across subjects and processing intervals. J Electromyogr Kinesiol 2018;40:102–9.
• [54] Peters KM, Kelly VE, Chang T, Weismann MC, Westcott McCoy S, Steele KM. Muscle recruitment and coordination during upper-extremity functional tests. J Electromyogr Kinesiol 2018;38:143–50.
• [55] Akgun G, Cetin AE, Kaplanoglu E. Exoskeleton design and adaptive compliance control for hand rehabilitation. Trans Inst Meas Control 2020;42(3):493–502.
• [56] Qi W, Su H, Aliverti A. A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Trans Human-Machine Syst 2020;50(5):414–23.
• [57] Su H, Hu Y, Karimi HR, Knoll A, Ferrigno G, De Momi E. Improved recurrent neural network-based manipulator control with remote center of motion constraints: experimental results. Neural Networks 2020;131:291–9.
• [58] Su H, Qi W, Yang C, Sandoval J, Ferrigno G, Momi ED. Deep neural network approach in robot tool dynamics identification for bilateral teleoperation. IEEE Robot Autom Lett 2020;5(2):2943–9.
• [59] Gámez AB, Hernandez Morante JJ, Martínez Gil JL, Esparza F, Martínez CM. The effect of surface electromyography biofeedback on the activity of extensor and dorsiflexor muscles in elderly adults: a randomized trial. Sci Rep 2019;9(1).

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).