Comparison of entropy rate measures for the evaluation of time series complexity: Simulations and application to heart rate and respiratory variability

• Autorzy: Barà Chiara , Pernice Riccardo , Catania Cristina Angela , Hilal Mirvana , Porta Alberto , Humeau-Heurtier Anne , Faes Luca

Identyfikatory

DOI

10.1016/j.bbe.2024.04.004

• Języki publikacji: EN

Abstrakty

EN

Most real-world systems are characterised by dynamics and correlations emerging at multiple time scales, and are therefore referred to as complex systems. In this work, the complexity of time series produced by complex systems was investigated in the frame of information theory computing the entropy rate via the conditional entropy (CE) measure. A comparative investigation of several CE estimators, based on linear parametric and non-linear model-free representations of the process dynamics, was performed considering simulated linear autoregressive (AR) and mixed non-linear deterministic and linear stochastic dynamics processes, as well as physiological time series reflecting short-term cardiorespiratory dynamics. In simulations, the estimated CE values decreased when reducing the system complexity through an increase in the pole radius of the AR process or with the predominance of the deterministic behaviour in the mixed dynamics. In the application to cardiorespiratory dynamics, a reduction in physiological complexity was observed resulting from a regularization of the time series of heart rate and respiratory volume when decreasing the breathing rate. Our results evidence how simple and fast approaches based on linear parametric or permutation-based modelfree estimators allow efficient discrimination of complexity changes in the short-term evolution of complex dynamic systems. However, in the presence of non-linear dynamics, the superiority of the more general but computationally expensive nearest-neighbour method is highlighted. These findings have implications for the assessment of complex dynamics both in clinical settings and in physiological monitoring.

Słowa kluczowe

EN

conditional entropy; cardiorespiratory dynamics; linear parametric entropy estimation; nonlinear mode free entropy estimation

PL

entropia warunkowa; dynamika krążeniowo oddechowa; estymacja entropii

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2024
• Tom: Vol. 44, no. 2
• Strony: 380--392
• Opis fizyczny: Bibliogr. 115 poz., tab., wykr.

Twórcy

autor

Barà Chiara

• Department of Engineering, University of Palermo, Palermo, 90128, Italy

autor

Pernice Riccardo

• Department of Engineering, University of Palermo, Palermo, 90128, Italy

autor

Catania Cristina Angela

• Department of Engineering, University of Palermo, Palermo, 90128, Italy

autor

Hilal Mirvana

• Univ Angers, LARIS, SFR MATHSTIC, F-49000, Angers, France

autor

Porta Alberto

• Department of Cardiothoracic, Vascular Anesthesia and Intensive Care, IRCCS Policlinico San Donato, San Donato Milanese, Milan, 20097, Italy
• Department of Biomedical Sciences for Health, University of Milan, Milan, 20133, Italy

autor

Humeau-Heurtier Anne

• Univ Angers, LARIS, SFR MATHSTIC, F-49000, Angers, France

autor

Faes Luca

• Department of Engineering, University of Palermo, Palermo, 90128, Italy

Bibliografia

• [1] Ivanov PC. The new field of network physiology: building the human physiolome. Front Netw Physiol 2021;1:711778.
• [2] Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D-U. Complex networks: Structure and dynamics. Phys Rep 2006;424(4–5):175-308.
• [3] Siegenfeld AF, Bar-Yam Y. An introduction to complex systems science and its applications. Complexity 2020;2020:1-16.
• [4] Ladyman J, Lambert J, Wiesner K. What is a complex system? Eur J Philos Sci 2013;3:33-67.
• [5] Foote R. Mathematics and complex systems. Science 2007;318(5849):410-2.
• [6] Goldberger AL. Giles f. filley lecture. complex systems. Proc Am Thorac Soc 2006;3(6):467-71.
• [7] Mihailović DT, Mimić G, Nikolić-Djorić E, Arsenić I. Novel measures based on the kolmogorov complexity for use in complex system behavior studies and time series analysis. Open Phys 2015;13(1).
• [8] Arshinov V, Fuchs C. Causality, emergence, self-organisation. NIA-Priroda Moscow; 2003.
• [9] Cannon J. The fractal geometry of nature, by benoit b. mandelbrot. Amer Math Monthly 1984;91(9):594-8.
• [10] Kaplan D, Glass L. Understanding nonlinear dynamics. Springer Science & Business Media; 1997.
• [11] Glass L. Synchronization and rhythmic processes in physiology. Nature 2001;410(6825):277-84.
• [12] Valente M, Javorka M, Porta A, Bari V, Krohova J, Czippelova B, Turianikova Z, Nollo G, Faes L. Univariate and multivariate conditional entropy measures for the characterization of short-term cardiovascular complexity under physiological stress. Physiol Meas 2018;39(1):014002.
• [13] Goldberger AL, Peng C-K, Lipsitz LA. What is physiologic complexity and how does it change with aging and disease? Neurobiol Aging 2002;23(1):23-6.
• [14] Shaffer F, McCraty R, Zerr CL. A healthy heart is not a metronome: an integrative review of the heart’s anatomy and heart rate variability. Front Psychol 2014;5:1040.
• [15] Lehrer P, Eddie D. Dynamic processes in regulation and some implications for biofeedback and biobehavioral interventions. Appl Psychophysiol biofeedback 2013;38:143-55.
• [16] Javorka M, Krohova J, Czippelova B, Turianikova Z, Lazarova Z, Wiszt R, Faes L. Towards understanding the complexity of cardiovascular oscillations: Insights from information theory. Comput biol Med 2018;98:48-57.
• [17] Javorka M, Turianikova Z, Tonhajzerova I, Javorka K, Baumert M. The effect of orthostasis on recurrence quantification analysis of heart rate and blood pressure dynamics. Physiol Meas 2008;30(1):29.
• [18] Porta A, Bari V, Ranuzzi G, De Maria B, Baselli G. Assessing multiscale complexity of short heart rate variability series through a model-based linear approach. Chaos 2017;27(9).
• [19] Heffernan KS, Fahs CA, Shinsako KK, Jae SY, Fernhall B. Heart rate recovery and heart rate complexity following resistance exercise training and detraining in young men. Am J Physiol-Heart Circ Physiol 2007;293(5):H3180-6.
• [20] Takahashi AC, Porta A, Melo RC, Quitério RJ, da Silva E, Borghi-Silva A, Tobaldini E, Montano N, Catai AM. Aging reduces complexity of heart rate variability assessed by conditional entropy and symbolic analysis. Intern Emerg Med 2012;7:229-35.
• [21] Porta A, Faes L, Bari V, Marchi A, Bassani T, Nollo G, Perseguini NM, Milan J, Minatel V, Borghi-Silva A, et al. Effect of age on complexity and causality of the cardiovascular control: comparison between model-based and model-free approaches. PLoS One 2014;9(2):e89463.
• [22] Jia Y, Gu H, Luo Q. Sample entropy reveals an age-related reduction in the complexity of dynamic brain. Sci Rep 2017;7(1):7990.
• [23] Pincus SM. Greater signal regularity may indicate increased system isolation. Math Biosci 1994;122(2):161-81.
• [24] Romero-Ortuño R, Martínez-Velilla N, Sutton R, Ungar A, Fedorowski A, Galvin R, Theou O, Davies A, Reilly RB, Claassen J, et al. Network physiology in aging and frailty: the grand challenge of physiological reserve in older adults. 2021.
• [25] Méndez MA, Zuluaga P, Hornero R, Gómez C, Escudero J, Rodríguez-Palancas A, Ortiz T, Fernández A. Complexity analysis of spontaneous brain activity: effects of depression and antidepressant treatment. J Psychopharmacol 2012;26(5):636-43.
• [26] Fernández A, Quintero J, Hornero R, Zuluaga P, Navas M, Gómez C, Escudero J, García-Campos N, Biederman J, Ortiz T. Complexity analysis of spontaneous brain activity in attention-deficit/hyperactivity disorder: diagnostic implications. Biol Psychiatry 2009;65(7):571-7.
• [27] Hornero R, Abásolo D, Jimeno N, Sánchez CI, Poza J, Aboy M. Variability, regularity, and complexity of time series generated by schizophrenic patients and control subjects. IEEE Trans Biomed Eng 2006;53(2):210-8.
• [28] Chen C, Jin Y, Lo IL, Zhao H, Sun B, Zhao Q, Zheng J, Zhang XD. Complexity change in cardiovascular disease. Int J Biol Sci 2017;13(10):1320.
• [29] Trunkvalterova Z, Javorka M, Tonhajzerova I, Javorkova J, Lazarova Z, Javorka K, Baumert M. Reduced short-term complexity of heart rate and blood pressure dynamics in patients with diabetes mellitus type 1: multiscale entropy analysis. Physiol Meas 2008;29(7):817.
• [30] Tobaldini E, Nobili L, Strada S, Casali KR, Braghiroli A, Montano N. Heart rate variability in normal and pathological sleep. Front Physiol 2013;4:294.
• [31] Theiler J. Estimating fractal dimension. J Opt Soc Amer A 1990;7(6):1055-73.
• [32] Wolf A, Swift JB, Swinney HL, Vastano JA. Determining lyapunov exponents from a time series. Physica D 1985;16(3):285-317.
• [33] Lempel A, Ziv J. On the complexity of finite sequences. IEEE Trans Inform Theory 1976;22(1):75-81.
• [34] Baranger M, Institute NECS. Chaos, complexity, and entropy: A physics talk for non-physicists. New England Complex Systems Institute; 2001, URL https: //books.google.sk/books?id=duzsMgEACAAJ.
• [35] Faes L, Porta A. Conditional entropy-based evaluation of information dynamics in physiological systems. Direct Inf Meas Neurosci 2014;61-86.
• [36] Lizier JT. The local information dynamics of distributed computation in complex systems. Springer Science & Business Media; 2012.
• [37] Faes L, Erla S, Nollo G. Measuring connectivity in linear multivariate processes: definitions, interpretation, and practical analysis. Comput Math Methods Med 2012;2012.
• [38] Kozachenko LF, Leonenko NN. Sample estimate of the entropy of a random vector. Probl Peredachi Inform 1987;23(2):9-16.
• [39] Richman JS, Moorman JR. Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol-Heart Circ Physiol 2000.
• [40] Azami H, Faes L, Escudero J, Humeau-Heurtier A, Silva LE. Entropy analysis of univariate biomedical signals: Review and comparison of methods. Front Entropy across Discipl: Panorama Entropy: Theory Comput Appl 2023;233-86.
• [41] Bandt C, Pompe B. Permutation entropy: a natural complexity measure for time series. Phys Rev Lett 2002;88(17):174102.
• [42] Cuesta-Frau D. Slope entropy: A new time series complexity estimator based on both symbolic patterns and amplitude information. Entropy 2019;21(12):1167.
• [43] Shannon CE. A mathematical theory of communication. Bell Syst Tech J 1948;27(3):379-423.
• [44] Cover TM. Elements of information theory. John Wiley & Sons; 1999.
• [45] Kolmogorov AN. A new metric invariant of transitive dynamical systems and automorphisms of lebesgue spaces. Trudy Mat Inst Imeni VA Steklova 1985;169:94-8.
• [46] Sinai YG. On the notion of entropy of a dynamical system. In: Doklady of Russian academy of sciences. Vol. 124, 1959, p. 768-71.
• [47] Porta A, De Maria B, Bari V, Marchi A, Faes L. Are nonlinear model-free conditional entropy approaches for the assessment of cardiac control complexity superior to the linear model-based one? IEEE Trans Biomed Eng 2016;64(6):1287-96.
• [48] Xiong W, Faes L, Ivanov PC. Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations. Phys Rev E 2017;95(6):062114.
• [49] Barà C, Sparacino L, Pernice R, Antonacci Y, Porta A, Kugiumtzis D, Faes L. Comparison of discretization strategies for the model-free information-theoretic assessment of short-term physiological interactions. Chaos 2023;33(3):033127.
• [50] Runge J, Heitzig J, Petoukhov V, Kurths J. Escaping the curse of dimensionality in estimating multivariate transfer entropy. Phys Rev Lett 2012;108(25):258701.
• [51] Porta A, Baselli G, Liberati D, Montano N, Cogliati C, Gnecchi-Ruscone T, Malliani A, Cerutti S. Measuring regularity by means of a corrected conditional entropy in sympathetic outflow. Biol Cybern 1998;78:71-8.
• [52] Barrett AB, Barnett L, Seth AK. Multivariate granger causality and generalized variance. Phys Rev E 2010;81(4):041907.
• [53] Mardia KV, Marshall RJ. Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika 1984;71(1):135-46.
• [54] Akaike H. A new look at the statistical model identification. IEEE Trans Autom Control 1974;19(6):716-23.
• [55] Schwarz G. Estimating the dimension of a model. Ann Stat 1978;461-4.
• [56] Barnett L, Barrett AB, Seth AK. Granger causality and transfer entropy are equivalent for gaussian variables. Phys Rev Lett 2009;103(23):238701.
• [57] Kraskov A, Stögbauer H, Grassberger P. Estimating mutual information. Phys Rev E 2004;69(6):066138.
• [58] Pincus SM. Approximate entropy as a measure of system complexity. Proc Natl Acad Sci 1991;88(6):2297-301.
• [59] Delgado-Bonal A, Marshak A. Approximate entropy and sample entropy: A comprehensive tutorial. Entropy 2019;21(6):541.
• [60] Unakafov AM, Keller K. Conditional entropy of ordinal patterns. Physica D 2014;269:94-102.
• [61] Kugiumtzis D. Partial transfer entropy on rank vectors. Eur Phys J Spec Top 2013;222(2):401-20.
• [62] Pernice R, Javorka M, Krohova J, Czippelova B, Turianikova Z, Busacca A, Faes L, Member I. Comparison of short-term heart rate variability indexes evaluated through electrocardiographic and continuous blood pressure monitoring. Med Biol Eng Comput 2019;57:1247-63.
• [63] Faes L, Porta A, Nollo G. Information decomposition in bivariate systems: theory and application to cardiorespiratory dynamics. Entropy 2015;17(1):277-303.
• [64] Cover TM, Thomas JA. Differential entropy. Elem Inf Theory 1991;224-38.
• [65] Santamaría-Bonfil G, Fernández N, Gershenson C. Measuring the complexity of continuous distributions. Entropy 2016;18(3):72.
• [66] May RM. Simple mathematical models with very complicated dynamics. Nature 1976;261:459-67.
• [67] Schreiber T, Schmitz A. Improved surrogate data for nonlinearity tests. Phys Rev Lett 1996;77(4):635.
• [68] Porta A, Bassani T, Bari V, Pinna GD, Maestri R, Guzzetti S. Accounting for respiration is necessary to reliably infer granger causality from cardiovascular variability series. IEEE Trans Biomed Eng 2011;59(3):832-41.
• [69] Cairo B, Bari V, Gelpi F, De Maria B, Porta A. Assessing cardiorespiratory interactions via lagged joint symbolic dynamics during spontaneous and controlled breathing. Front Netw Physiol 2023;3.
• [70] Darbellay GA, Wuertz D. The entropy as a tool for analysing statistical dependences in financial time series. Phys A 2000;287(3-4):429-39.
• [71] Zhou R, Cai R, Tong G. Applications of entropy in finance: A review. Entropy 2013;15(11):4909-31.
• [72] Leung L-Y, North GR. Information theory and climate prediction. J Clim 1990;3(1):5-14.
• [73] Hlinka J, Hartman D, Vejmelka M, Runge J, Marwan N, Kurths J, Paluš M. Reliability of inference of directed climate networks using conditional mutual information. Entropy 2013;15(6):2023-45.
• [74] Berntson GG, Cacioppo JT, Quigley KS. Respiratory sinus arrhythmia: Autonomic origins, physiological mechanisms, and psychophysiological implications. Psychophysiology 1993;30(2):183-96.
• [75] Saul JP, Berger RD, Chen M, Cohen RJ. Transfer function analysis of autonomic regulation, ii. respiratory sinus arrhythmia. Am J Physiol-Heart Circ Physiol 1989;256(1):H153-61.
• [76] Tzeng Y, Larsen P, Galletly D. Cardioventilatory coupling in resting human subjects. Exp Physiol 2003;88(6):775-82.
• [77] Elstad M, O’Callaghan EL, Smith AJ, Ben-Tal A, Ramchandra R. Cardiorespiratory interactions in humans and animals: rhythms for life. Am J Physiol-Heart Circ Physiol 2018;315(1):H6-17.
• [78] Porta A, Guzzetti S, Furlan R, Gnecchi-Ruscone T, Montano N, Malliani A. Complexity and nonlinearity in short-term heart period variability: comparison of methods based on local nonlinear prediction. IEEE Trans Biomed Eng 2006;54(1):94-106.
• [79] Porta A, Bari V, Gelpi F, Cairo B, De Maria B, Tonon D, Rossato G, Faes L. On the different abilities of cross-sample entropy and k-nearest-neighbor cross-unpredictability in assessing dynamic cardiorespiratory and cerebrovascular interactions. Entropy 2023;25(4):599.
• [80] Nuzzi D, Stramaglia S, Javorka M, Marinazzo D, Porta A, Faes L. Extending the spectral decomposition of granger causality to include instantaneous influences: application to the control mechanisms of heart rate variability. Phil Trans R Soc A 2021;379(2212):20200263.
• [81] Volpes G, Barà C, Busacca A, Stivala S, Javorka M, Faes L, Pernice R. Feasibility of ultra-short-term analysis of heart rate and systolic arterial pressure variability at rest and during stress via time-domain and entropy-based measures. Sensors 2022;22(23):9149.
• [82] Lombardi D, Pant S. Nonparametric k-nearest-neighbor entropy estimator. Phys Rev E 2016;93(1):013310.
• [83] Trujillo LT. K-th nearest neighbor (knn) entropy estimates of complexity and integration from ongoing and stimulus-evoked electroencephalographic (eeg) recordings of the human brain. Entropy 2019;21(1):61.
• [84] Kaplan DT, Furman MI, Pincus SM, Ryan SM, Lipsitz LA, Goldberger AL. Aging and the complexity of cardiovascular dynamics. Biophys J 1991;59(4):945-9.
• [85] Lewis M, Short A. Sample entropy of electrocardiographic rr and qt time-series data during rest and exercise. Physiol Meas 2007;28(6):731.
• [86] Lake DE, Richman JS, Griffin MP, Moorman JR. Sample entropy analysis of neonatal heart rate variability. Am J Physiol-Regul Integr Comp Physiol 2002;283(3):R789-97.
• [87] Porta A, Gnecchi-Ruscone T, Tobaldini E, Guzzetti S, Furlan R, Montano N. Progressive decrease of heart period variability entropy-based complexity during graded head-up tilt. J Appl Physiol 2007;103(4):1143-9.
• [88] Porta A, Faes L, Masé M, D’addio G, Pinna G, Maestri R, Montano N, Furlan R, Guzzetti S, Nollo G, et al. An integrated approach based on uniform quantization for the evaluation of complexity of short-term heart period variability: application to 24h holter recordings in healthy and heart failure humans. Chaos 2007;17(1).
• [89] Cuesta-Frau D, Miró-Martínez P, Oltra-Crespo S, Molina-Picó A, Dakappa PH, Mahabala C, Vargas B, González P. Classification of fever patterns using a single extracted entropy feature: A feasibility study based on sample entropy. Math Biosci Eng 2020;17:235.
• [90] Morales J, Borzée P, Testelmans D, Buyse B, Van Huffel S, Varon C. Linear and non-linear quantification of the respiratory sinus arrhythmia using support vector machines. Front Physiol 2021;12:623781.
• [91] Porta A, Guzzetti S, Montano N, Furlan R, Pagani M, Malliani A, Cerutti S. Entropy, entropy rate, and pattern classification as tools to typify complexity in short heart period variability series. IEEE Trans Biomed Eng 2001;48(11):1282-91.
• [92] Arya S, Mount DM, Netanyahu NS, Silverman R, Wu AY. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J ACM 1998;45(6):891-923.
• [93] Merkwirth C, Parlitz U, Lauterborn W. Fast nearest-neighbor searching for nonlinear signal processing. Phys Rev E 2000;62(2):2089.
• [94] Samet H. K-nearest neighbor finding using maxnearestdist. IEEE Trans Pattern Anal Mach Intell 2007;30(2):243-52.
• [95] Manis G. Fast computation of approximate entropy. Comput Methods Programs Biomed 2008;91(1):48-54.
• [96] Pan Y-H, Wang Y-H, Liang S-F, Lee K-T. Fast computation of sample entropy and approximate entropy in biomedicine. Comput Methods Programs Biomed 2011;104(3):382-96.
• [97] Manis G, Aktaruzzaman M, Sassi R. Low computational cost for sample entropy. Entropy 2018;20(1):61.
• [98] Liu W, Jiang Y, Xu Y. A super fast algorithm for estimating sample entropy. Entropy 2022;24(4):524.
• [99] Dias D, Paulo Silva Cunha J. Wearable health devices - vital sign monitoring, systems and technologies. Sensors 2018;18(8):2414.
• [100] Kakria P, Tripathi N, Kitipawang P. A real-time health monitoring system for remote cardiac patients using smartphone and wearable sensors. Int. J Telemed Appl 2015;2015:8.
• [101] Georgiou K, Larentzakis AV, Khamis NN, Alsuhaibani GI, Alaska YA, Giallafos EJ. Can wearable devices accurately measure heart rate variability? a systematic review. Folia Med 2018;60(1):7-20.
• [102] Pernice R, Javorka M, Krohova J, Czippelova B, Turianikova Z, Busacca A, Faes L. A validity and reliability study of conditional entropy measures of pulse rate variability. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society. EMBC, IEEE; 2019, p. 5568-71.
• [103] Citi L, Guffanti G, Mainardi L. Rank-based multi-scale entropy analysis of heart rate variability. In: Computing in cardiology 2014. IEEE; 2014, p. 597-600.
• [104] Manis G, Aktaruzzaman M, Sassi R. Bubble entropy: An entropy almost free of parameters. IEEE Trans Biomed Eng 2017;64(11):2711-8.
• [105] Omidvarnia A, Mesbah M, Pedersen M, Jackson G. Range entropy: A bridge between signal complexity and self-similarity. Entropy 2018;20(12):962.
• [106] Wang X, Si S, Li Y. Multiscale diversity entropy: A novel dynamical measure for fault diagnosis of rotating machinery. IEEE Trans Ind Inf 2020;17(8):5419-29.
• [107] Chen W, Zhuang J, Yu W, Wang Z. Measuring complexity using fuzzyen, apen, and sampen. Med Eng Phys 2009;31(1):61-8.
• [108] Li P, Liu C, Li K, Zheng D, Liu C, Hou Y. Assessing the complexity of short-term heartbeat interval series by distribution entropy. Med Biol Eng Comput 2015;53:77-87.
• [109] Rostaghi M, Azami H. Dispersion entropy: A measure for time-series analysis. IEEE Signal Process Lett 2016;23(5):610-4.
• [110] Rohila A, Sharma A. Phase entropy: A new complexity measure for heart rate variability. Physiol Meas 2019;40(10):105006.
• [111] Platiša MM, Radovanović NN, Pernice R, Barà C, Pavlović SU, Faes L. Information-theoretic analysis of cardio-respiratory interactions in heart failure patients: Effects of arrhythmias and cardiac resynchronization therapy. Entropy 2023;25(7):1072.
• [112] Nollo G, Faes L, Antolini R, Porta A. Assessing causality in normal and impaired short-term cardiovascular regulation via nonlinear prediction methods. Phil Trans R Soc A 2009;367(1892):1423-40.
• [113] Cerutti S, Corino VD, Mainardi L, Lombardi F, Aktaruzzaman M, Sassi R. Nonlinear regularity of arterial blood pressure variability in patient with atrial fibrillation in tilt-test procedure. Europace 2014;16(suppl_4):iv141-7.
• [114] Chen Y, Yang H. Multiscale recurrence analysis of long-term nonlinear and nonstationary time series. Chaos Solitons Fractals 2012;45(7):978-87.
• [115] Faes L, Porta A, Nollo G, Javorka M. Information decomposition in multivariate systems: definitions, implementation and application to cardiovascular networks. Entropy 2016;19(1):5.