Classification of motor imagery EEG signals using wavelet scattering transform and Bi-directional long short-term memory networks

• Autorzy: Zhang Hongyuan , Zhao Zijian , Liu Chong , Duan Miao , Lu Zhiguo , Wang Hong

Identyfikatory

DOI

10.1016/j.bbe.2024.11.003

• Języki publikacji: EN

Abstrakty

EN

A brain-computer interface (BCI) is a technology that creates a communication path between the brain and external devices. Raw EEG data in BCI contain a large amount of complex information, but only some of it needs to be focused on in research. So Feature extraction and classification play an important role in BCI by reducing the data dimensionality and improving the accuracy of subsequent classification. Wavelet scattering transform is an emerging feature extraction method that generates time-shift invariant representations of EEG signals. We applied the wavelet scattering transform to extract features from motor imagery EEG signals, and utilized these features for classification purposes. To achieve this, we proposed a new method that combines wavelet scattering transform with a bidirectional long short-term memory (BiLSTM) network in a fusion deep learning network. Wavelet scattering transform can deeply mine the feature information in EEG signals. In the classification stage, multiple time window features obtained in the scattering transform are sent to the BiLSTM network for classification. The final result will be determined by a vote. In addition, for the processing of raw EEG data, we proposed a time-step based time window strategy that can better utilize the small dataset. This operation can obtain EEG data of multiple time steps. The proposed method was validated using BCI competition II dataset III and BCI competition IV dataset 2b. The results show that the proposed method in this paper can effectively improve the accuracy of motor imagery EEG and provide a new idea for the feature extraction and classification research of motor imagery brain-computer interface.

Słowa kluczowe

EN

motor imagery; electroencephalogram; EEG; signal classification; wavelet scattering transform; WST; Bi-directional long short term memory; BiLSTM

PL

obrazowanie motoryczne; elektroencefalogram; EEG; klasyfikacja sygnału; transformata falkowa; BiLSTM

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2024
• Tom: Vol. 44, no. 4
• Strony: 874--884
• Opis fizyczny: Bibliogr. 42 poz., rys., tab., wykr.

Twórcy

autor

Zhang Hongyuan

• School of Mechanical Engineering & Automation, Northeastern University, Shenyang, 110819, China

autor

Zhao Zijian

• School of Mechanical Engineering & Automation, Northeastern University, Shenyang, 110819, China

autor

Liu Chong

• School of Mechanical Engineering & Automation, Northeastern University, Shenyang, 110819, China

• https://orcid.org/0000-0001-9352-4994

autor

Duan Miao

• China FAW Co.,Ltd., Changchun, 130000, China

autor

Lu Zhiguo

• School of Mechanical Engineering & Automation, Northeastern University, Shenyang, 110819, China

autor

Wang Hong

• School of Mechanical Engineering & Automation, Northeastern University, Shenyang, 110819, China

Bibliografia

• [1] Wolpaw JR. Chapter 6 - brain-computer interfaces. In: Barnes MP, Good DC, editors. Neurological rehabilitation. Handbook of clinical neurology, vol. 110, Elsevier; 2013, p. 67-74. http://dx.doi.org/10.1016/B978-0-444-52901-5.00006-X.
• [2] Fatourechi M, Bashashati A, Ward RK, Birch GE. EMG and EOG artifacts in brain computer interface systems: A survey. Clin Neurophysiol 2007;118(3):480-94. http://dx.doi.org/10.1016/j.clinph.2006.10.019.
• [3] Pfurtscheller G, Aranibar A. Event-related cortical desynchronization detected by power measurements of scalp EEG. Electroencephalogr Clin Neurophysiol 1977;42(6):817-26. http://dx.doi.org/10.1016/0013-4694(77)90235-8.
• [4] Birbaumer N. Brain-computer-interface research: Coming of age. Clin Neurophysiol 2006;117(3):479-83. http://dx.doi.org/10.1016/j.clinph.2005.11.002.
• [5] Bang J-S, Lee M-H, Fazli S, Guan C, Lee S-W. Spatio-spectral feature representation for motor imagery classification using convolutional neural networks. IEEE Trans Neural Netw Learn Syst 2022;33(7):3038-49. http://dx.doi.org/10.1109/ TNNLS.2020.3048385.
• [6] Koles ZJ, Lazar MS, Zhou SZ. Spatial patterns underlying population differences in the background EEG. Brain Topogr 1990;2(4):275-84. http://dx.doi.org/10.1007/BF01129656.
• [7] Ang KK, Chin ZY, Zhang H, Guan C. Filter bank common spatial pattern (FBCSP) in brain-computer interface. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). 2008, p. 2390-7. http://dx.doi.org/10.1109/IJCNN.2008.4634130.
• [8] Huang E, Zheng X, Fang Y, Zhang Z. Classification of motor imagery EEG based on time-domain and frequency-domain dual-stream convolutional neural network. IRBM 2022;43(2):107-13. http://dx.doi.org/10.1016/j.irbm.2021.04.004.
• [9] Xu B, Zhang L, Song A, Wu C, Li W, Zhang D, Xu G, Li H, Zeng H. Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification. IEEE Access 2019;7:6084-93. http://dx.doi.org/10. 1109/ACCESS.2018.2889093.
• [10] Chaudhary S, Taran S, Bajaj V, Sengur A. Convolutional neural network based approach towards motor imagery tasks EEG signals classification. IEEE Sens J 2019;19(12):4494-500. http://dx.doi.org/10.1109/JSEN.2019.2899645.
• [11] Li F, He F, Wang F, Zhang D, Xia Y, Li X. A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl Sci 2020;10(5). http://dx.doi.org/10.3390/app10051605.
• [12] Bruna J, Mallat S. Invariant scattering convolution networks. IEEE Trans Pattern Anal Mach Intell 2013;35(8):1872-86. http://dx.doi.org/10.1109/TPAMI.2012.230.
• [13] Mallat S. Group invariant scattering. Comm Pure Appl Math 2012;65(10):1331-98. http://dx.doi.org/10.1002/cpa.21413.
• [14] Ahmad MZ, Kamboh AM, Saleem S, Khan AA. Mallat’s scattering transform based anomaly sensing for detection of seizures in scalp EEG. IEEE Access 2017;5:16919-29. http://dx.doi.org/10.1109/ACCESS.2017.2736014.
• [15] Liu Z, Yao G, Zhang Q, Zhang J, Zeng X, et al. Wavelet scattering transform for ECG beat classification. Comput Math Methods Med 2020;2020. http://dx.doi. Org/10.1155/2020/3215681.
• [16] Buriro AB, Ahmed B, Baloch G, Ahmed J, Shoorangiz R, Weddell SJ, Jones RD. Classification of alcoholic EEG signals using wavelet scattering transform-based features. Comput Biol Med 2021;139:104969. http://dx.doi.org/10.1016/ j.compbiomed.2021.104969.
• [17] Kumar S, Sharma A, Mamun K, Tsunoda T. A deep learning approach for motor imagery EEG signal classification. In: 2016 3rd Asia-Pacific world congress on computer science and engineering (APWC on CSE). 2016, p. 34-9. http: //dx.doi.org/10.1109/APWC-on-CSE.2016.017.
• [18] Salimpour S, Kalbkhani H, Seyyedi S, Solouk V. Stockwell transform and semi-supervised feature selection from deep features for classification of BCI signals. Sci Rep 2022;12(1):11773. http://dx.doi.org/10.1038/s41598-022-15813-3.
• [19] Ghezaiel W, Brun L, Lézoray O. Wavelet scattering transform and CNN for closed set speaker identification. In: 2020 IEEE 22nd international workshop on multimedia signal processing. MMSP, 2020, p. 1-6. http://dx.doi.org/10.1109/ MMSP48831.2020.9287061.
• [20] Leeb R, Brunner C, Müller-Putz G, Schlögl A, Pfurtscheller G. BCI competition 2008-Graz data set B. Austria: Graz University of Technology; 2008, p. 1-6.
• [21] Andén J, Mallat S. Deep scattering spectrum. IEEE Trans Signal Process 2014;62(16):4114-28. http://dx.doi.org/10.1109/TSP.2014.2326991.
• [22] Oyallon E, Belilovsky E, Zagoruyko S. Scaling the scattering transform: Deep hybrid networks. In: Proceedings of the IEEE international conference on computer vision. 2017, p. 5618-27.
• [23] Mikolov T, Karafiát M, Burget L, Cernocky J, Khudanpur S. Recurrent neural network based language model. In: Interspeech. vol. 2, Makuhari; 2010, p. 1045-8.
• [24] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735-80. http://dx.doi.org/10.1162/neco.1997.9.8.1735.
• [25] Schuster M, Paliwal K. Bidirectional recurrent neural networks. IEEE Trans Signal Process 1997;45(11):2673-81. http://dx.doi.org/10.1109/78.650093.
• [26] Baldi P, Brunak S, Frasconi P, Soda G, Pollastri G. Exploiting the past and the future in protein secondary structure prediction. Bioinformatics 1999;15(11):937-46. http://dx.doi.org/10.1093/bioinformatics/15.11.937.
• [27] Li H, Ding M, Zhang R, Xiu C. Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network. Biomed Signal Process Control 2022;72:103342. http://dx.doi.org/10.1016/j.bspc.2021.103342.
• [28] Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Map 2017;38(11):5391-420. http://dx.doi.org/10.1002/hbm.23730.
• [29] Sepúlveda A, Castillo F, Palma C, Rodriguez-Fernandez M. Emotion recognition from ECG signals using wavelet scattering and machine learning. Appl Sci 2021;11(11). http://dx.doi.org/10.3390/app11114945.
• [30] Andreux M, Angles T, Exarchakisgeo G, Leonardu R, Rochette G, Thiry L, Zarka J, Mallat S, Andén J, Belilovsky E, et al. Kymatio: Scattering transforms in python. J Mach Learn Res 2020;21(1):2256-61.
• [31] Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett 2006;27(8):861-74. http://dx.doi.org/10.1016/j.patrec.2005.10.010.
• [32] Ramoser H, Muller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 2000;8(4):441-6. http://dx.doi.org/10.1109/86.895946.
• [33] Lemm S, Blankertz B, Curio G, Muller K-R. Spatio-spectral filters for improving the classification of single trial EEG. IEEE Trans Biomed Eng 2005;52(9):1541-8. http://dx.doi.org/10.1109/TBME.2005.851521.
• [34] Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. Eegnet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng 2018;15(5):056013. http://dx.doi.org/10.1088/1741-2552/aace8c.
• [35] Xu G, Shen X, Chen S, Zong Y, Zhang C, Yue H, Liu M, Chen F, Che W. A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access 2019;7:112767-76. http://dx.doi.org/10.1109/ACCESS.2019. 2930958.
• [36] He H, Wu D. Transfer learning for brain-computer interfaces: A euclidean space data alignment approach. IEEE Trans Biomed Eng 2020;67(2):399-410. http://dx.doi.org/10.1109/TBME.2019.2913914.
• [37] Sun B, Wu Z, Hu Y, Li T. Golden subject is everyone: A subject transfer neural network for motor imagery-based brain computer interfaces. Neural Netw 2022;151:111-20. http://dx.doi.org/10.1016/j.neunet.2022.03.025.
• [38] Wang J, Cheng S, Tian J, Gao Y. A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification. Biomed Signal Process Control 2023;83:104627. http://dx.doi.org/10.1016/j.bspc.2023.104627.
• [39] Huang E, Zheng X, Fang Y, Zhang Z. Classification of motor imagery EEG based on time-domain and frequency-domain dual-stream convolutional neural network. IRBM 2022;43(2):107-13. http://dx.doi.org/10.1016/j.irbm.2021.04.004, URL: https://www.sciencedirect.com/science/article/pii/S1959031821000506.
• [40] Lee HK, Choi Y-S. Application of continuous wavelet transform and convolutional neural network in decoding motor imagery brain-computer interface. Entropy 2019;21(12). http://dx.doi.org/10.3390/e21121199, URL: https://www. mdpi.com/1099-4300/21/12/1199.
• [41] Chen Z, Wang Y, Song Z. Classification of motor imagery electroencephalography signals based on image processing method. Sensors 2021;21(14). http://dx.doi. org/10.3390/s21144646, URL: https://www.mdpi.com/1424-8220/21/14/4646.
• [42] Zhang G, Luo J, Han L, Lu Z, Hua R, Chen J, Che W. A dynamic multi-scale network for EEG signal classification. Front Neurosci 2021;14. http://dx.doi.org/10.3389/fnins.2020.578255, URL: https://www.frontiersin.org/articles/10.3389/ fnins.2020.578255.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).