Compact convolutional neural network (CNN) based on SincNet for end-to-end motor imagery decoding and analysis

• Autorzy: Izzuddin Tarmizi Ahmad , Safri Norlaili Mat , Othman Mohd Afzan

Identyfikatory

DOI

10.1016/j.bbe.2021.10.001

• Języki publikacji: EN

Abstrakty

EN

In the field of human-computer interaction, the detection, extraction and classification of the electroencephalogram (EEG) spectral and spatial features are crucial towards developing a practical and robust non-invasive EEG-based brain-computer interface. Recently, due to the popularity of end-to-end deep learning, the applicability of algorithms such as convolutional neural networks (CNN) has been explored to achieve the mentioned tasks. This paper presents an improved and compact CNN algorithm for motor imagery decoding based on the adaptation of SincNet, which was initially developed for speaker recognition task from the raw audio input. Such adaptation allows for a compact end-to-end neural network with state-of-the-art (SOTA) performances and enables network interpretability for neurophysiological validation in cortical rhythms and spatial analysis. In order to validate the performance of proposed algorithms, two datasets were used; the first is the publicly available BCI Competition IV dataset 2a, which was often used as a benchmark in validating motor imagery classification algorithms, and the second is a dataset consists of primary data initially collected to study the difference between motor imagery and mental-task associated motor imagery BCI and was used to test the plausibility of the proposed algorithm in highlighting the differences in terms of cortical rhythms. Competitive decoding performance was achieved in both datasets in comparisons with SOTA CNN models, albeit with the lowest number of trainable parameters. In addition, it was shown that the proposed architecture performs a cleaner band-pass, highlighting the necessary frequency bands that were crucial and neurophysiologically plausible in solving the classification tasks.

Słowa kluczowe

EN

brain-computer interface; BCI; convolutional neural network; CNN; electroencephalogram; EEG; motor imagery

PL

interfejs człowiek-komputer; BCI; sieć neuronowa konwolucyjna; CNN; elektroencefalogram; EEG; obrazowanie ruchowe

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2021
• Tom: Vol. 41, no. 4
• Strony: 1629--1645
• Opis fizyczny: Bibliogr. 48 poz., rys., tab., wykr.

Twórcy

autor

Izzuddin Tarmizi Ahmad

• Department of Control, Instrumentation and Automation, Faculty of Electrical Engineering, Universiti Teknikal Malaysia Melaka, Hang Tuah Jaya, 76100 Durian Tunggal, Melaka, Malaysia; Department of Electronic and Computer Engineering, School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Johor, Malaysia

autor

Safri Norlaili Mat

• Department of Electronic and Computer Engineering, School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Johor, Malaysia

autor

Othman Mohd Afzan

• Department of Electronic and Computer Engineering, School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Johor, Malaysia

Bibliografia

• [1] Rao RPN. Brain-computer interfacing: an introduction. Cambridge University Press; 2011. 10.1017/CBO9781139032803.
• [2] Lebedev MA, Nicolelis MAL. Brain-machine interfaces: from basic science to neuroprostheses and neurorehabilitation. Physiol Rev 2017;97:767–837. https://doi.org/10.1152/physrev.00027.2016.
• [3] Abdulkader SN, Atia A, Mostafa M-S. Brain computer interfacing: applications and challenges. Egypt Informatics J 2015;16(2):213–30. https://doi.org/10.1016/j.eij.2015.06.002.
• [4] Amiri S, Fazel-Rezai R, Asadpour V. A review of hybrid brain-computer interface systems. Adv Human-Computer Interact 2013;2013:1–8. https://doi.org/10.1155/2013/187024.
• [5] Brodu N, Lotte F, Lecuyer A. Comparative study of band-power extraction techniques for Motor Imagery classification. In: IEEE Symp. Comput. Intell Cogn. Algorithms, Mind, Brain. IEEE; 2011. p. 1–6. 10.1109/CCMB.2011.5952105.
• [6] Wang Y, Gao S, Gao X. Common spatial pattern method for channel selection in motor imagery based brain-computer. Interface 2006:5392–5. https://doi.org/10.1109/iembs.2005.1615701.
• [7] Jin J, Miao Y, Daly I, Zuo C, Hu D, Cichocki A. Correlation-based channel selection and regularized feature optimization for MI-based BCI. Neural Networks 2019;118:262–70. https://doi.org/10.1016/j.neunet.2019.07.008.
• [8] Jin J, Fang H, Daly I, Xiao R, Miao Y, Wang X, et al. Optimization of model training based on iterative minimum covariance determinant in motor-imagery BCI. Int J Neural Syst 2021;31:2150030. https://doi.org/10.1142/S0129065721500301.
• [9] Ang KK, Chin ZY, Wang C, Guan C, Zhang H. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Front Neurosci 2012;6:1–9. https://doi.org/10.3389/fnins.2012.00039.
• [10] Kai Keng Ang, Zheng Yang Chin, Haihong Zhang, Cuntai Guan, Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface, in: 2008 IEEE Int. Jt. Conf. Neural Networks (IEEE World Congr. Comput. Intell., 2008: pp. 2390–2397. 10.1109/IJCNN.2008.4634130.
• [11] Jin J, Xiao R, Daly I, Miao Y, Wang X, Cichocki A. Internal feature selection method of CSP based on L1-norm and dempster-shafer theory. IEEE Trans Neural Networks Learn Syst PP 2020:1–12. https://doi.org/10.1109/TNNLS.2020.3015505.
• [12] Wang H, Tang C, Xu T, Li T, Xu L, Yue H, et al. An approach of one-vs-rest filter bank common spatial pattern and spiking neural networks for multiple motor imagery decoding. IEEE Access 2020;8:86850–61. https://doi.org/10.1109/ACCESS.2020.2992631.
• [13] Barachant A, Bonnet S, Congedo M, Jutten C. Multiclass brain-computer interface classification by riemannian geometry. IEEE Trans Biomed Eng 2012;59(4):920–8. https://doi.org/10.1109/TBME.2011.2172210.
• [14] Barachant A, Bonnet S, Congedo M, Jutten C. Classification of covariance matrices using a Riemannian-based kernel for BCI applications. Neurocomputing 2013;112:172–8. https://doi.org/10.1016/j.neucom.2012.12.039.
• [15] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: IEEE Conf. Comput. Vis. Pattern Recognit. IEEE; 2016. p. 770–8. 10.1109/CVPR.2016.90.
• [16] Sainath TN, Kingsbury B, Saon G, Soltau H, Mohamed AR, Dahl G, et al. Deep convolutional neural networks for large-scale speech tasks. Neural Networks 2015;64:39–48. https://doi.org/10.1016/j.neunet.2014.08.005.
• [17] Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 2017;38(11):5391–420. https://doi.org/10.1002/hbm.23730.
• [18] Tayeb Z, Fedjaev J, Ghaboosi N, Richter C, Everding L, Qu X, et al. Validating deep neural networks for online decoding of motor imagery movements from eeg signals. Sensors (Switzerland) 2019;19. https://doi.org/10.3390/s19010210.
• [19] Amin SU, Alsulaiman M, Muhammad G, Bencherif MA, Hossain MS. Multilevel weighted feature fusion using convolutional neural networks for EEG motor imagery classification. IEEE Access 2019;7:18940–50. https://doi.org/10.1109/ACCESS.2019.2895688.
• [20] Tang X, Wang T, Du Y, Dai Y. Motor imagery EEG recognition with KNN-based smooth auto-encoder. Artif Intell Med 2019;101. https://doi.org/10.1016/j.artmed.2019.101747101747.
• [21] Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng 2018;15. https://doi.org/10.1088/1741-2552/aace8c 056013.
• [22] Mousavi M, de Sa VR. Temporally adaptive common spatial patterns with deep convolutional neural networks. In: 41st Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. IEEE; 2019. p. 4533–6. 10.1109/EMBC.2019.8857423.
• [23] Tang Z, Li C, Sun S. Single-trial EEG classification of motor imagery using deep convolutional neural networks. Optik (Stuttg) 2017;130:11–8. https://doi.org/10.1016/j.ijleo.2016.10.117.
• [24] Ravanelli M, Bengio Y, Recognition S. Speaker recognition from raw waveform with SincNet. In: IEEE Spok. Lang. Technol. Work. IEEE; 2018. p. 1021–8. 10.1109/SLT.2018.8639585.
• [25] Olivas-Padilla BE, Chacon-Murguia MI. Classification of multiple motor imagery using deep convolutional neural networks and spatial filters. Appl Soft Comput J 2019;75:461–72. https://doi.org/10.1016/j.asoc.2018.11.031.
• [26] JEL (Hanoi) H, CG, Roberts A. DDSP:Differentiable Digital Signal Processing, in: Int. Conf. Learn. Represent., 2020: pp. 1–19.
• [27] Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: The all convolutional net, in: 3rd Int. Conf. Learn. Represent. ICLR 2015 – Work. Track Proc., 2015.
• [28] Wolpaw JR, McFarland DJ, Vaughan TM. Brain-computer interface research at the Wadsworth Center. IEEE Trans Rehabil Eng 2000. https://doi.org/10.1109/86.847823.
• [29] Tangermann M, Mu¨ ller K-R, Aertsen A, Birbaumer N, Braun C, Brunner C, et al. Review of the BCI competition IV. Front Neurosci 2012;6. https://doi.org/10.3389/fnins.2012.00055.
• [30] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15: 1929–58.
• [31] Jeunet C, N’Kaoua B, Subramanian S, Hachet M, Lotte F, Friedman D. Predicting mental imagery-based BCI performance from personality, cognitive profile and neurophysiological patterns. PLoS ONE 2015;10(12):e0143962. https://doi.org/10.1371/journal.pone.0143962.
• [32] Kubler A, Holz E, Kaufmann T, Zickler C. A User Centred Approach for Bringing BCI Controlled Applications to End-Users, in: Brain-Computer Interface Syst. - Recent Prog. Futur. Prospect., InTech, 2013. https://doi.org/10.5772/55802.
• [33] Ma X, Qiu S, Wei W, Wang S, He H. Deep channel-correlation network for motor imagery decoding from the same limb. IEEE Trans Neural Syst Rehabil Eng 2020;28(1):297–306. https://doi.org/10.1109/TNSRE.2019.2953121.
• [34] Garcia-Rill E. The 10Hz Fulcrum, in: Waking Reticular Act. Syst. Heal. Dis., Elsevier, 2015: pp. 157–170. 10.1016/B978-0-12-801385-4.00008-2.
• [35] Kübler A, Mattia D. Brain-computer interface based solutions for end-users with severe communication disorders. In: Neurol Conciousness. Elsevier; 2016. p. 217–40. 10.1016/B978-0-12-800948-2.00014-5.
• [36] Abhang PA, Gawali BW, Mehrotra SC. Technical Aspects of Brain Rhythms and Speech Parameters, in: Introd. to EEGSpeech-Based Emot. Recognit., Elsevier, 2016: pp. 51–79. https://doi.org/10.1016/B978-0-12-804490-2.00003-8.
• [37] Satapathy SK, Dehuri S, Jagadev AK, Mishra S. Introduction. In: EEG Brain Signal Classif Epileptic Seizure Disord Detect. Elsevier; 2019. p. 1–25. 10.1016/B978-0-12-817426-5.00001-6.
• [38] Izzuddin TA, Safri NM, Othman MA. Mental imagery classification using one-dimensional convolutional neural network for target selection in single-channel BCI-controlled mobile robot. Neural Comput Appl 2020. https://doi.org/10.1007/s00521-020-05393-6.
• [39] Lim S, Yeo M, Yoon G. Comparison between concentration and immersion based on EEG analysis. Sensors 2019;19:1669. https://doi.org/10.3390/s19071669.
• [40] Zeng H, Wu Z, Zhang J, Yang C, Zhang H, Dai G, et al. EEG emotion classification using an improved sincnet-based deep learning model. Brain Sci 2019;9. https://doi.org/10.3390/brainsci9110326.
• [41] Emami A, Kunii N, Matsuo T, Shinozaki T, Kawai K, Takahashi H. Seizure detection by convolutional neural network-based analysis of scalp electroencephalography plot images. NeuroImage Clin 2019;22. https://doi.org/10.1016/j.nicl.2019.101684 101684.
• [42] Zhou M, Tian C, Cao R, Wang B, Niu Y, Hu T, et al. Epileptic seizure detection based on EEG signals and CNN. Front Neuroinform 2018;12. https://doi.org/10.3389/fninf.2018.00095.
• [43] Xu M, Han J, Wang Y, Jung T-P, Ming D. Implementing Over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features. IEEE Trans Biomed Eng 2020;67(11):3073–82. https://doi.org/10.1109/TBME.2020.2975614.
• [44] Acharya UR, Oh SL, Hagiwara Y, Tan JH, Adam M, Gertych A, et al. A deep convolutional neural network model to classify heartbeats. Comput Biol Med 2017;89:389–96. https://doi.org/10.1016/j.compbiomed.2017.08.022.
• [45] Porumb M, Iadanza E, Massaro S, Pecchia L. Biomedical signal processing and control a convolutional neural network approach to detect congestive heart failure. Biomed Signal Process Control 2020;55. https://doi.org/10.1016/j.bspc.2019.101597 101597.
• [46] Abdul-Kadir NA, Mat Safri N, Othman MA. Atrial fibrillation classification and association between the natural frequency and the autonomic nervous system. Int J Cardiol 2016;222:504–8. https://doi.org/10.1016/j.ijcard.2016.07.196.
• [47] Sengur A, Akbulut Y, Guo Y, Bajaj V. Classification of amyotrophic lateral sclerosis disease based on convolutional neural network and reinforcement sample learning algorithm. Heal Inf Sci Syst 2017;5:9. https://doi.org/10.1007/s13755-017-0029-6.
• [48] Li Y, Wang F, Chen Y, Cichocki A, Sejnowski T. The effects of audiovisual inputs on solving the cocktail party problem in the human brain: an fMRI study. Cereb Cortex 2018;28:3623–37. https://doi.org/10.1093/cercor/bhx235.