Machine Learning Based System Identification with Binary Output Data Using Kernel Methods

• Autorzy: Fateh Rachid , Oualla Hicham , Azougaghe Es-said , Darif Anouar , Boumezzough Ahmed , Safi Said , Pouliquen Mathieu , Frikel Miloud

Identyfikatory

DOI

10.26636/jtit.2024.1.1430

• Języki publikacji: EN

Abstrakty

EN

Within the realm of machine learning, kernel methods stand out as a prominent class of algorithms with widespread applications, including but not limited to classification, regression, and identification tasks. Our paper addresses the challenging problem of identifying the finite impulse response (FIR) of single-input single-output nonlinear systems under the influence of perturbations and binary-valued measurements. To overcome this challenge, we exploit two algorithms that leverage the framework of reproducing kernel Hilbert spaces (RKHS) to accurately identify the impulse response of the Proakis C channel. Additionally, we introduce the application of these kernel methods for estimating binary output data of nonlinear systems. We showcase the effectiveness of kernel adaptive filters in identifying nonlinear systems with binary output measurements, as demonstrated through the experimental results presented in this study.

Słowa kluczowe

EN

finite impulse response; kernel adaptive filtering; nonlinear systems identification; Proakis C channe

• Wydawca: Instytut Łączności - Państwowy Instytut Badawczy
• Czasopismo: Journal of Telecommunications and Information Technology
• Rocznik: 2024
• Tom: nr 1
• Strony: 17--25
• Opis fizyczny: Bibliogr. 38 poz., rys., wykr.

Twórcy

autor

Fateh Rachid

• Laboratory of Innovation in Mathematics, Applications, and Information Technologies, Polydisciplinary Faculty Sultan Moulay Slimane University, Beni Mellal, Morocco

• https://orcid.org/0000-0002-0574-2105

autor

Oualla Hicham

• Akkodis, Paris, France

autor

Azougaghe Es-said

• Laboratory of Innovation in Mathematics, Applications, and Information Technologies, Polydisciplinary Faculty Sultan Moulay Slimane University, Beni Mellal, Morocco

• https://orcid.org/0000-0002-0233-1132

autor

Darif Anouar

• Laboratory of Innovation in Mathematics, Applications, and Information Technologies, Polydisciplinary Faculty Sultan Moulay Slimane University, Beni Mellal, Morocco

• https://orcid.org/0000-0001-8026-9189

autor

Boumezzough Ahmed

• Laboratory of Innovation in Mathematics, Applications, and Information Technologies, Polydisciplinary Faculty Sultan Moulay Slimane University, Beni Mellal, Morocco

autor

Safi Said

• Laboratory of Innovation in Mathematics, Applications, and Information Technologies, Polydisciplinary Faculty Sultan Moulay Slimane University, Beni Mellal, Morocco

• https://orcid.org/0000-0003-3390-9037

autor

Pouliquen Mathieu

• Normandie University, Caen, France

autor

Frikel Miloud

• Normandie University, Caen, France

• https://orcid.org/0000-0003-2178-1814

Bibliografia

• [1] S. Haykin, Adaptive Filter Theory, 4th ed., Hoboken: Prentice Hall, 920 p., 2002 (ISBN: 9780130901262).
• [2] M.M. Sondhi, “The History of Echo Cancellation”, IEEE Signal Processing Magazine, vol. 23, no. 5, pp. 95 –102, 2006 (https//doi.org/10.1109/MSP.2006.1708416).
• [3] A.H. Sayed, Fundamentals of Adaptive Filtering, Hoboken: John Wiley & Sons, 1168 p., 2003 (ISBN: 9780471461265).
• [4] P.S.R. Diniz, Adaptive Filtering: Algorithms and Practical Implementations, 3 rd ed., New York: Springer, 652 p., 2008 (https://doi.org/10.1007/978-1-4614-4106-9).
• [5] R. Fateh, A. Darif, and S. Safi, “Kernel and Linear Adaptive Methods for the BRAN Channels Identification”, in: International Conference on Advanced Intelligent Systems for Sustainable Development, Tangier, Morocco, pp. 579– 591, 2020 (https://doi.org/ 10.1007/978-3-030-90639-9_47).
• [6] R. Fateh, A. Darif, and S. Safi, “Identification of the Linear Dynamic Parts of Wiener Model Using Kernel and Linear Adaptive”, in: International Conference on Advanced Intelligent Systems for Sustainable Development, Tangier, Morocco, pp. 387– 400, 2020 (https://doi.org/10.1007/978-3-030-90639-9_31).
• [7] E. Ferrara, “Fast Implementations of LMS Adaptive Filters”, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 28, no. 4, pp. 474–475, 1980 (https://doi.org/ 10. 1109/tassp.1980.1163432).
• [8] S. Sanei and J.A. Chambers, EEG Signal Processing, Hoboken: John Wiley & Sons, 289 p., 2013 (https://doi.org/ 10. 1002/978047 0511923).
• [9] M. Krstic, I. Kanellakopoulos, and P.V. Kokotovic, Nonlinear and Adaptive Control Design, Hoboken: John Wiley & Sons, 592 p., 1995 (ISBN: 9780471127321).
• [10] R. Fateh, A. Darif, and S. Safi, “Channel Identification of Non-linear Systems with Binary-Valued Output Observations Based on Positive Definite Kernels”, E3S Web of Conferences, vol. 297, art. no. 01020, 2021 (https://doi.org/10.1051/e3sconf/202129701020).
• [11] R. Fateh, A. Darif, and S. Safi, “Hyperbolic Functions Impact Evaluation on Channel Identification Based on Recursive Kernel Algorithm”, 2022 8th International Conference on Optimization and Applications (ICOA), Genoa, Italy, 2022 (https://doi.org/ 10. 1109/ICOA55659.2022.9934118).
• [12] J.W. Xu, A.R. Paiva, I. Park, and J.C. Principe, “A Reproducing Kernel Hilbert Space Framework for Information-theoretic Learning”, IEEE Transactions on Signal Processing, vol. 56, no. 12, pp. 5891 –5902 ,2008 (https://doi.org/10.1109/TSP.2008.2005085).
• [13] W. Liu, J.C. Principe, and S. Haykin, Kernel Adaptive Filtering: A Comprehensive Introduction, Hoboken: John Wiley & Sons, 240 p., 2010 (ISBN: 9780470447536).
• [14] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 462 p., 2004 (https://doi.org/10.1017/CBO9780511809682).
• [15] N. Aronszajn, “Theory of Reproducing Kernels”, Transactions of the American Mathematical Society, vol. 68, no. 3, pp. 337–404, 1950 (https://doi.org/10.2307/1990404).
• [16] B. Scholkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Cambridge: MIT Press, 2002 (https://doi.org/ 10.7551/mitpress/4175.001.0001).
• [17] C. Cortes and V. Vapnik, “Support-vector Networks”, Machine Learning, vol. 20, no. 3, pp. 273–297, 1995 (https://doi.org/ 10.1007/bf00994018).
• [18] B. Scholkopf, A. Smola, and K.R. Muller, “Kernel Principal Component Analysis”, in: International Conference on Artificial Neural Networks (ICANN), Lausanne, Switzerland, pp. 583– 588, 1997 (https://doi.org/10.1007/bfb0020217).
• [19] M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, “Support Vector Machines”, IEEE Intelligent Systems and Their Applications, vol. 13 , no. 4, pp. 18–28, 1998 (https://doi.org/10.1109/5254.708428).
• [20] M. Zidane and R. Dinis, “A New Combination of Adaptive Channel Estimation Methods and TORC Equalizer in MC-CDMA Systems”, International Journal of Communication Systems, vol. 33, no. 11, art. no. 4429, 2020 (https://doi.org/10.1002/dac.4429).
• [21] M. Zidane, S. Safi, and M. Sabri, “Measured and Estimated Data of Non-linear BRAN Channels Using HOS in 4G Wireless Communications”, Data in Brief, vol. 17 , pp. 1136–1148 , 2018 (https: //doi.org/10.1016/j.dib.2018.02.005).
• [22] R. Fateh, A. Darif, and S. Safi, “Performance Evaluation of MC-CDMA Systems with Single User Detection Technique using Kernel and Linear Adaptive Method”, Journal of Telecommunications and Information Technology, no. 4, pp. 1–11, 2021 (https://doi.org/10.26636/jtit.2021.151621).
• [23] M. Zidane, S. Safi, and M. Sabri, “Compensation of Fading Channels Using Partial Combining Equalizer in MC-CDMA Systems”, Journal of Telecommunications and Information Technology, no. 1, pp. 5– 11, 2017 (http://dlibra.itl.waw.pl/dlibra-webapp/Content/1962/ISSN_1509-4553_1_2017_5.pdf).
• [24] S. Safi, M. Frikel, A. Zeroual, and M. M’Saad, “Higher Order Cumulants for Identification and Equalization of Multicarrier Spreading Spectrum Systems”, Journal of Telecommunications and Information Technology, no. 1, pp. 74–84, 2011 (https://doi.org/ 10.26636 /jtit.2022.161122).
• [25] W. Liu, P.P. Pokharel, and J.C. Principe, “The Kernel Least-mean square Algorithm”, IEEE Transactions on Signal Processing, vol. 56, no. 2 , pp. 543– 554, 2008 (https://doi.org/ 10.1109/tsp.2007.907881).
• [26] C. Richard, J. Bermudez, and P. Honeine, “Online Prediction of Time Series Data with Kernels”, IEEE Transactions on Signal Processing, vol. 57, no. 3, pp. 1058 –1067, 2009 (https://doi.org/ 10. 1109/TSP.2008.2009895).
• [27] R. Fateh, A. Darif, and S. Safi, “An Extended Version of the Proportional Adaptive Algorithm Based on Kernel Methods for Channel Identification with Binary Measurements”, Journal of Telecommunications and Information Technology, no. 3, pp. 47 –58, 2022 (https://doi.org/10.26636/jtit.2022.161122).
• [28] S. Ciochina, C. Paleologu, and J. Benesty, “An Optimized NLMS Algorithm for System Identification”, Signal Processing, vol. 118, pp. 115– 121, 2016 (https://doi.org/ 10.1016/j.sigpro.201 5.06.016).
• [29] L. Ljung, System Identification: Theory for the User, 2nd ed., Hoboken: Prentice Hall, 640 p., 1999 (ISBN: 9780136566953).
• [30] F. Ding, System Identification-Performances Analysis for Identification Methods, Beijing: Science Press, 2014.
• [31] X. Zhang, F. Ding, F.E. Alsaadi, and T. Hayat, “Recursive Parameter Identification of the Dynamical Models for Bilinear State Space Systems”, Nonlinear Dynamics, vol. 89 , pp. 2415–2429 , 2017 (https://doi.org/10.1007/s11071-017-3594-y).
• [32] L. Xu, “Application of the Newton Iteration Algorithm to the Parameter Estimation for Dynamical Systems”, Journal of Computational and Applied Mathematics, vol. 288, pp. 33–43, 2015 (https: //doi.org/10.1016/j.cam.2015.03.057).
• [33] Q. Song, “Recursive Identification of Systems with Binary-valued Outputs and with ARMA Noises”, Automatica, vol. 93, pp. 106–113, 2018 (https://doi.org/ 10. 1016/j.automatica. 2018. 03. 059).
• [34] J. Guo, X. Wang, W. Xue, and Y. Zhao, “System Identification with Binary-valued Observations under Data Tampering Attacks”, IEEE Transactions on Automatic Control, vol. 66, no. 8, pp. 3825– 3832, 2020 (https://doi.org/10.1109/TAC.2020.3029325).
• [35] L. Li, F. Wang, H. Zhang, and X. Ren, “A Novel Recursive Learning Estimation Algorithm of Wiener Systems with Quantized Observations”, ISA Transactions, vol. 112, pp. 23–34 , 2021 (https://doi.org/10.1016/j.isatra.2020.11.032).
• [36] L. Zhang, Y. Zhao, and L. Guo, “Identification and Adaptation with Binary-valued Observations under Non-persistent Excitation Condition”, Automatica, vol. 138, art. no. 110158, 2022 (https: //doi.org/10.1016/j.automatica.2022.110158).
• [37] R. Fateh and A. Darif, “Mean Square Convergence of Reproducing Kernel for Channel Identification: Application to Bran D Channel Impulse Response”, in: International Conference on Business Intelligence, Beni-Mellal, Morocco, 2021 (https://doi.org/ 10.1007/978-3-030-76508-8_20).
• [38] W. Liu and J.C. Principe, “Kernel Affine Projection Algorithms”, EURASIP Journal on Advances in Signal Processing, vol. 2008 , art. no. 784292, 2008 (https://doi.org/10.1155/2008/784292).