Hybrid EEG-fNIRS brain-computer interface based on the non-linear features extraction and stacking ensemble learning

• Autorzy: Maher Asmaa , Mian Qaisar Saeed , Salankar N. , Jiang Feng , Tadeusiewicz Ryszard , Pławiak Paweł , Abd El-Latif Ahmed A. , Hammad Mohamed

Identyfikatory

DOI

10.1016/j.bbe.2023.05.001

• Języki publikacji: EN

Abstrakty

EN

The Brain-computer interface (BCI) is used to enhance the human capabilities. The hybridBCI (hBCI) is a novel concept for subtly hybridizing multiple monitoring schemes to maximize the advantages of each while minimizing the drawbacks of individual methods. Recently, researchers have started focusing on the Electroencephalogram (EEG) and ‘‘Functional Near-Infrared Spectroscopy” (fNIRS) based hBCI. The main reason is due to the development of artificial intelligence (AI) algorithms such as machine learning approaches to better process the brain signals. An original EEG-fNIRS based hBCI system is devised by using the non-linear features mining and ensemble learning (EL) approach. We first diminish the noise and artifacts from the input EEG-fNIRS signals using digital filtering. After that, we use the signals for non-linear features mining. These features are ‘‘Fractal Dimension” (FD), ‘‘Higher Order Spectra” (HOS), ‘‘Recurrence Quantification Analysis” (RQA) features, and Entropy features. Onward, the Genetic Algorithm (GA) is employed for Features Selection (FS). Lastly, we employ a novel Machine Learning (ML) technique using several algorithms namely, the ‘‘Naïve Bayes” (NB), ‘‘Support Vector Machine” (SVM), ‘‘Random Forest” (RF), and ‘‘K-Nearest Neighbor” (KNN). These classifiers are combined as an ensemble for recognizing the intended brain activities. The applicability is tested by using a publicly available multi-subject and multiclass EEG-fNIRS dataset. Our method has reached the highest accuracy, F1-score, and sensitivity of 95.48%, 97.67% and 97.83% respectively.

Słowa kluczowe

EN

hybrid BCI; electroencephalogram; ensemble learning; genetic algorithm; motor imagery tasks; nonlinear features extraction

PL

BCI; elektroencefalogram; uczenie zespołowe; algorytm genetyczny; ekstrakcja cech nieliniowych

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2023
• Tom: Vol. 43, no. 2
• Strony: 463--475
• Opis fizyczny: Bibliogr. 40 poz., rys., tab., wykr.

Twórcy

autor

Maher Asmaa

• School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China

autor

Mian Qaisar Saeed

• LINEACT CESI, Lyon, France
• Dept. of Electrical and Computer Engineering, Effat University, Jeddah, Saudi Arabia

autor

Salankar N.

• Persistent Systems Limited, Nagpur, India

autor

Jiang Feng

• School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China

autor

Tadeusiewicz Ryszard

• AGH University of Science and Technology, Department of Biocybernetics and Biomedical Engineering, Krakow, Poland

autor

Pławiak Paweł

• Department of Computer Science, Faculty of Computer Science and Telecommunications, Cracow University of Technology, Krakow, Poland
• Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Gliwice, Poland

autor

Abd El-Latif Ahmed A.

• EIAS Data Science Lab, College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia
• Department of Mathematics and Computer Science, Faculty of Science, Menoufia University, Al-Minufya, Egypt

autor

Hammad Mohamed

• EIAS Data Science Lab, College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia
• Department of Information Technology, Faculty of Computers and Information, Menoufia University, Al-Minufya, Egypt

Bibliografia

• [1] Mudgal SK, Sharma SK, Chaturvedi J, Sharma A. Brain computer interface advancement in neurosciences: Applications and issues. Interdiscipl Neurosurg 2020;20 100694.
• [2] Rashid M, Sulaiman N, Abdul Majeed PPA, Musa RM, Bari BS, Khatun S. Current status, challenges, and possible solutions of EEG-based brain-computer interface: A comprehensive review. Front Neurorob 2020;14:25.
• [3] Liu Z, Shore J, Wang M, Yuan F, Buss A, Zhao X. A systematic review on hybrid EEG/fNIRS in brain-computer interface. Biomed Signal Process Control 2021;68 102595.
• [4] Gao W, Guan J, Gao J, Zhou D. Multi-ganglion ANN based feature learning with application to P300-BCI signal classification. Biomed Signal Process Control 2015;18:127-37.
• [5] Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, et al. How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett 2009;462 (1):94-8.
• [6] Judith AM, Priya SB. Multiset task related component analysis (M-TRCA) for SSVEP frequency recognition in BCI. J Ambient Intell Hum Comput 2021;12(5):5117-26.
• [7] Pillette L, Roc A, N’kaoua B, Lotte F. Experimenters’ Influence on Mental-Imagery based Brain-Computer Interface User Training. Int J Hum Comput Stud 2021;149 102603.
• [8] Vasilyev A, Liburkina S, Yakovlev L, Perepelkina O, Kaplan A. Assessing motor imagery in brain-computer interface training: Psychological and neurophysiological correlates. Neuropsychologia 2017;97:56-65.
• [9] Mora N, De Munari I, Ciampolini P. A Multi-modal BCI System for Active and Assisted Living; 2016. pp. 345-355.
• [10] Bhattacharyya S, Mukul MK. Reactive frequency band based movement imagery classification. J Ambient Intell Hum Comput 2018:1-14.
• [11] Zarei R, He J, Siuly S, Zhang Y. A PCA aided cross-covariance scheme for discriminative feature extraction from EEG signals. Comput Methods Programs Biomed 2017;146:47-57.
• [12] Lee C-S, Wang M-H, Kuan W-K, Huang S-H, Tsai Y-L, Ciou ZH, et al. BCI-based hit-loop agent for human and AI robot colearning with AIoT application. J Ambient Intell Hum Comput 2021:1-25.
• [13] Neuper CU, Pfurtscheller G. Event-related dynamics of cortical rhythms: Frequency-specific features and functional correlates. Int J Psychophysiol 2001;43(1):41-58.
• [14] Foong R, Ang KK, Quek C, Guan C, Phua KS, Kuah CWK, et al. Assessment of the efficacy of EEG-based MI-BCI with visual feedback and EEG correlates of mental fatigue for upper-limb stroke rehabilitation. IEEE Trans Biomed Eng 2019;67 (3):786-95.
• [15] Valente G, Kaas AL, Formisano E, Goebel R. Optimizing fMRI experimental design for MVPA-based BCI control: Combining the strengths of block and event-related designs. Neuroimage 2019;186:369-81.
• [16] Li C, Su M, Xu J, Jin H, Sun L. A between-subject fNIRS-BCI study on detecting self-regulated intention during walking. IEEE Trans Neural Syst Rehabil Eng 2020;28(2):531-40.
• [17] Zhang S, Zheng Y, Wang D, Wang L, Ma J, Zhang J, et al. Application of a common spatial pattern-based algorithm for an fNIRS-based motor imagery brain-computer interface. Neurosci Lett 2017;655:35-40.
• [18] Li R, Potter T, Huang W, Zhang Y. Enhancing performance of a hybrid EEG-fNIRS system using channel selection and early temporal features. Front Hum Neurosci 2017;11:462.
• [19] Shin J, Kwon J, Im C-H. A ternary hybrid EEG-NIRS brain-computer interface for the classification of brain activation patterns during mental arithmetic, motor imagery, and idle state. Front Neuroinf 2018;12:5.
• [20] Chaudhary U, Xia B, Silvoni S, Cohen LG, Birbaumer N. Brain-computer interface-based communication in the completely locked-in state. PLoS Biol 2017;15(1) e1002593.
• [21] Cabañero L, Hervás R, Bravo J, Rodríguez-Benitez L, Nugent C. eeglib: Computational analysis of cognitive performance during the use of video games. J Ambient Intell Hum Comput 2019:1-12.
• [22] Zhu G, Li R, Zhang T, Lou D, Wang R, Zhang Y. A simplified hybrid EEG-fNIRS Brain-Computer Interface for motor task classification. 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER); 2017. pp. 134-137.
• [23] Ge S, Yang Q, Wang R, Lin P, Gao J, Leng Y, et al. A brain-computer interface based on a few-channel EEG-fNIRS bimodal system. IEEE Access 2017;5:208-18.
• [24] Saadati M, Nelson J, Ayaz H. Convolutional neural network for hybrid fNIRS-EEG mental workload classification. International Conference on Applied Human Factors and Ergonomics 2019:221-32.
• [25] Khan MU, Hasan MA. Hybrid EEG-fNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD). Front Hum Neurosci 2020;14, 599802.
• [26] Han C-H, Mu¨ ller K-R, Hwang H-J. Enhanced Performance of a Brain Switch by Simultaneous Use of EEG and NIRS Data for Asynchronous Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2020;28(10):2102-12.
• [27] Shin J, Kim D-W, Müller K-R, Hwang H-J. Improvement of information transfer rates using a hybrid EEG-NIRS brain-computer interface with a short trial length: Offline and pseudo-online analyses. Sensors 2018;18(6):1827.
• [28] Ghaffar MSBA, Khan US, Iqbal J, Rashid N, Hamza A, Qureshi WS, et al. Improving classification performance of four class FNIRS-BCI using Mel Frequency Cepstral Coefficients (MFCC). Infrared Phys Technol 2021;112 103589.
• [29] Ho TKK, Gwak J, Park CM, Khare A, Song J-I. Deep learning-based approach for mental workload discrimination from multi-channel fNIRS. In: Recent trends in communication, computing, and electronics. Springer; 2019. p. 431-40.
• [30] Shin J, Müller K-R, Schmitz CH, Kim D-W, Hwang H-J. Evaluation of a compact hybrid brain-computer interface system. BioMed Res Int 2017.
• [31] Dehais F, Dupres A, Di Flumeri G, Verdiere K, Borghini G, Babiloni F, Roy R. Monitoring pilot’s cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI. 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC); 2018. pp. 544-549.
• [32] Sharma M, Acharya UR. Automated detection of schizophrenia using optimal wavelet-based $$ l_1 $$ l 1 norm features extracted from single-channel EEG. Cogn Neurodyn 2021:1-14.
• [33] Bairy GM, Lih OS, Hagiwara Y, Puthankattil SD, Faust O, Niranjan U, et al. Automated diagnosis of depression electroencephalograph signals using linear prediction coding and higher order spectra features. J Med Imag Health Informat 2017;7(8):1857-62.
• [34] Zbilut JP, Webber Jr CL. Embeddings and delays as derived from quantification of recurrence plots. Phys Lett A 1992;171 (3-4):199-203.
• [35] Hammad M, Rajesh KN, Abdelatey A, Abdar M, Zomorodi-Moghadam M, San Tan R, et al. Automated detection of Shockable ECG signals: A review. Information Sci 2021;571:580-604.
• [36] Acharya UR, Hagiwara Y, Deshpande SN, Suren S, Koh JEW, Oh SL, et al. Characterization of focal EEG signals: A review. Futur Gener Comput Syst 2019;91:290-9.
• [37] Ksiaôżek W, Hammad M, Pławiak P, Acharya UR, Tadeusiewicz R. Development of novel ensemble model using stacking learning and evolutionary computation techniques for automated hepatocellular carcinoma detection. Biocybern Biomed Eng 2020;40(4):1512-24.
• [38] Zeng X, Martinez TR. Distribution-balanced stratified cross-validation for accuracy estimation. J Exp Theor Artif Intell 2000;12(1):1-12.
• [39] Khan B, Wildey C, Francis R, Tian F, Delgado MR, Liu H, et al. Improving optical contact for functional near-infrared brain spectroscopy and imaging with brush optodes. Biomed Opt Express 2012;3(5):878-98.
• [40] Mian Qaisar S. Signal-piloted processing and machine learning based efficient power quality disturbances recognition. PLoS One 2021;16(5):e0252104.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023)